🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Will be presenting this work at #NeurIPS2024, today 11am, poster #2311. Come visit us!

Presenting this work at #NeurIPS2024 today 4:30pm session (poster #4807, east). Come by to hear about auto-regressive decision trees for language modeling!

The 4:30pm poster session today at #NeurIPS2024 will feature "The Evolution of Statistical Induction Heads: In-Context Learning Markov Chains," by Ezra Edelman, Nikolaos Tsilivis, Surbhi Goel, Ben Edelman and Eran Malach. #KempnerInstitute

New paper at #ICLR2025! Fast LLM inference = smart scheduling 🕒 but size-based scheduling (prioritizing short requests over long ones) requires knowing request sizes—a challenging task in LLM systems. So, how can we predict request sizes accurately? 🔗openreview.net/forum?id=7JhGd…)

To backtrack or not to backtrack? The answer depends on the nature of the reasoning problem! Check out our paper new paper, led by Sunny Qin, with David Alvarez Melis and Samy Jelassi: arxiv.org/abs/2504.07052 See thread below 👇👇

The best part of RLs focus in post-training right now is that the elicitation idea of post-training is a much better match to large-scale pretraining. Instruction and preference tuning are still crucial to good models, in tasteful amounts. More research on this:

New in the Deeper Learning blog: Kempner researchers dive into how LLMs reason with the help of "backtracking," and explore alternative ways to enhance LLM reasoning capabilities. bit.ly/KempnerBacktra… #AI #ML #LLMs By Sunny Qin, David Alvarez Melis, Samy Jelassi, and Eran Malach

New in the Deeper Learning blog: Kempner researchers show how VLMs speak the same semantic language across images and text. bit.ly/KempnerVLM by Isabel Papadimitriou, Chloe H. Su, Thomas Fel, Stephanie Gil, and Sham Kakade #AI #ML #VLMs #SAEs

Super excited to announce our ICML workshop on highlighting the power (and limitations?) of small-scale in the era of large-scale ML. You can submit just a Jupyter notebook, Jupyter notebook + paper, or a survey/position paper. Do submit your work and help us spread the word!

Excited to announce MOSS, our ICML workshop focused on discoveries at small scale! We believe there's tremendous potential & creativity in research done with limited resources and would love to hear your ideas. The submission (due May 22nd) can literally be a Jupyter notebook! :)

We have a new SSM theory paper, just accepted to COLT, revisiting recall properties of linear RNNs. It's surprising how much one can delve into, and how beautiful it can become. With (and only thanks to) the amazing Alexandre and Francis Bach arxiv.org/pdf/2502.09287

I'll be attending ICML this week; come stop by our poster on length generalization in LLMs on Tuesday morning (poster session 1 west)! Paper link: openreview.net/forum?id=S9LkB…