🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

🚀 Introducing CAVA: The Comprehensive Assessment for Voice Assistants A new benchmark for evaluating end-to-end, speech-in-speech-out voice assistants in real-world scenarios. We go beyond single tasks or metrics to test the capabilities required for voice assistants:

Will AI agents be controlled by big tech companies? Or could they be controlled by users, safeguarding user autonomy and privacy? In a new position paper (accepted to ICML 2025), we outline the steps we need to take now to enable user-centric agents (w/Seth Lazar, Noam Kolt)🧶

🚨 Lucie-Aimée Kaffee and I are looking for a junior collaborator to research the Open Model Ecosystem! 🤖 Ideally, someone w/ AI/ML background, who can help w/ annotation pipeline + analysis. docs.google.com/forms/d/e/1FAI…

My PhD defense is this coming Monday (June 2) from 1-2 PM PT. It will be in-person at Stanford and also on Zoom. I tried my best to invite folks individually, but if you would like an invite, just send me an email or DM me and I can send you details!

🧵 Multilingual safety training/eval is now standard practice, but a critical question remains: Is multilingual safety actually solved? Our new survey with Cohere Labs answers this and dives deep into: - Language gap in safety research - Future priority areas Thread 👇

Can you train a performant language models without using unlicensed text? We are thrilled to announce the Common Pile v0.1, an 8TB dataset of openly licensed and public domain text. We train 7B models for 1T and 2T tokens and match the performance similar models like LLaMA 1&2

Happy to release the Common Pile, an 8TB, 1 Trillion Token Dataset of Public Domain and Openly Licensed Text in collaboration with EleutherAI, Vector Institute, Ai2, Hugging Face, and DPI by Shayne Longpre. We provisioned a subset of the Common Pile, consisting only of public

Only a fraction of data needed for LLM comes with identifiable licenses. But if you curate it all, can you train a model on in? We release Common Pile, a 1T tokens dataset, and train a 7B model on it! results are on par with open weights models trained on eq FLOPS

Two years in the making, we finally have 8 TB of openly licensed data with document-level metadata for authorship attribution, licensing details, links to original copies, and more. Hugely proud of the entire team.

🧵 Crazy verifier we have built: Simulate vision evolution by evolving embodied agents inside realistic simulators that simulate physics of light and use it as a verification engine. This enables us to re-run evolution computationally to test impossible questions like 'what if

🪄We made a 1B Llama BEAT GPT-4o by... making it MORE private?! LoCoMo results: 🔓GPT-4o: 80.6% 🔐1B Llama + GPT-4o (privacy): 87.7% (+7.1!⏫) 💡How? GPT-4o provides reasoning ("If X then Y"), the local model fills in the blanks with your private data to get the answer!

individual reporting for post-deployment evals — a little manifesto (& new preprints!) tldr: end users have unique insights about how deployed systems are failing; we should figure out how to translate their experiences into formal evaluations of those systems.

My PhD materials are now available! Dissertation: arxiv.org/abs/2506.23123 Slides: drive.google.com/file/d/13N2FRW… Folks should read the acknowledgements since so many people have been so important to me along this journey!