Imagine a community of LLM agents. Do they learn to cooperate with one another, or will they act selfishly? We know that human greediness can cause the “Tragedy of the Commons”, but what about #LLMs? 📜 ➡️ arxiv.org/abs/2404.16698 🧵👇

How do LLMs learn to reason from data? Are they ~retrieving the answers from parametric knowledge🦜? In our new preprint, we look at the pretraining data and find evidence against this: Procedural knowledge in pretraining drives LLM reasoning ⚙️🔢 🧵⬇️

How to Select Datapoints for Efficient Human Evaluation of NLG Models?. arxiv.org/abs/2501.18251

🚀 𝐇𝐨𝐰 𝐰𝐞𝐥𝐥 𝐜𝐚𝐧 𝐋𝐋𝐌𝐬 𝐭𝐞𝐚𝐜𝐡? Evaluating LLMs for education is key to making real progress, yet we lack a reliable and simple benchmark. Introducing 𝐌𝐚𝐭𝐡𝐓𝐮𝐭𝐨𝐫𝐁𝐞𝐧𝐜𝐡—an open-source benchmark designed to assess holistic tutoring capabilities in AI.

🌍 How do #LLMs handle trolley problems across cultures? We test them with 98K dilemmas in 107 languages, grounded in 40M+ human moral judgments. 💡 Spotlight @ICLR2025 in Singapore✈️| Best Paper Pluralistic Alignment Workshop Workshop #NeurIPS2024 📄 Paper: arxiv.org/abs/2407.02273 🧵👇

Very honored to be one of the 15,553 runners today in #SOLA Relay Zürich. And also super proud of our #NLProc team of 14 finishing 113km in total! Many many thanks to all the friends & our Prof Mrinmaya Sachan! It's such a meaningful day in my life. Yet to run for #EMNLP now ;)!

This paper introduces an online reinforcement learning framework using simulated student-tutor interactions. It trains LLMs to prioritize guiding students pedagogically instead of simply revealing solutions, aligning models with better teaching methods. This helps students

🤖⚛️Can AI truly see Physics? Test your model with the newly released SeePhys Benchmark! 🚀 🖼️Covering 2,000 vision-text multimodal physics problems spanning from middle school to doctoral qualification exams, the SeePhys benchmark systematically evaluates LLMs/MLLMs on tasks

I am excited to share that our paper, “Generating Pedagogically Meaningful Visuals for Math Word Problems: A New Benchmark and Analysis of Text-to-Image Models” has been accepted to Findings of ACL 2025. Preprint available: arxiv.org/abs/2506.03735 Code and dataset coming soon!

AI alignment for tutoring🎓 We use full online RL with conversation-level rewards—not just single-turn signals like DPO. Did the student actually learn by the end? Using GRPO, the model learns real teaching strategies like when to hint or when to correct. Explore models below⤵️

Check out our latest work on aligning language models with pedagogy.

1/9: Dense SAE Latents Are Features💡, Not Bugs🐛❌! In our new paper, we examine dense (ie. very frequently occuring) SAE latents. We find that dense latents are structured and meaningful, representing truly dense model signals.🧵

Check out our 3 papers on Testing LLM Moral Reasoning via Multi-Agent Simulations! ✍️ Our summary blogpost: lesswrong.com/posts/2WAire3L… 📑Our series of 3 papers:1️⃣GovSim (NeurIPS 2024) arxiv.org/abs/2404.16698 2️⃣SanctSim zhijing-jin.com/files/papers/2… 3️⃣MoralSim arxiv.org/abs/2505.19212

Human evaluation is still the holy grail in NLG. But, can you get more bang for your buck? Check out this amazing piece of work from Vilém Zouhar

New paper on detecting & correcting arithmetic errors in LLMs! We show that simple probes can recover correct answers from hidden states and trigger self-correction of reasoning errors. 📍 If you’re at #ICML2025 stop by our poster @ the Act Interp WS 📝arxiv.org/abs/2507.12379

Congrats again to our brilliant students David Guzman Yongjin Yang for receiving the "Oral Paper Award" at the #ACL2025NLP Workshop on Research on Agent Language Models (REALM)! Check out how Reasoning LLMs optimize self interests over collective success 📊 in our paper