🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

I keep getting asked about my take on these CO2 estimates for the o3 model by the press and members of the community, so I'll interrupt my vacation to comment 🤓 TL;DR- any kind of estimate is a proxy, and instead of wasting our time and energy, we should demand accountability.

CMU LTI is hosting predoc interns this summer, centered around "Language Technologies for All"! Please apply and circulate! lti.cs.cmu.edu/news-and-event…

What do we want to know about LLMs in research? Out today in Nature Medicine, we take a stab at developing a comprehensive, living taxonomy of LLM use-cases and reporting.

#ICML2025 includes a new track on Application-Driven Machine Learning (innovative ML techniques, problems, and datasets driven by the needs of end-users in real-world)! If this fits your work, consider submitting to ICML (dl: Jan 30) and checking the ADML box ✅ in OpenReview ⬇️

Dear Climate and AI community! We are hiring 😀 a postdoc to join UVA Environmental Institute at UVA and work with Chirag Agarwal and myself, on using multimodal AI models and explainable AI to attribute extreme precipitation events! Fascinating stuff! Link below. Please RT!

Excited to share our work on keeping LLMs up-to-date by composing multiple post-training interventions was accepted to #ICLR2025 ICLR 2026! Great work led by Arinbjörn and Kyle O'Brien!

Medical licensing exams are convenient LLM benchmarks, but they don’t reflect real-world clinical tasks. With LLMs already in EHRs, we need benchmarks that match real-world needs. Let’s partner with hospitals piloting these tools to develop diverse, task-specific evaluations.

More SAE papers coming! We dived deeper, looking into what is the best way to gather the SAE features for downstream classifications and also what are the potential benefits 🧐.

I'm honored to have received a research award from Capital One to support our work developing models that reason about time series data! Thank you! Many exciting new results in this area coming soon :)

Excited to share new works on knowledge editing for LLMs! Many recent papers find cases where editing LLMs breaks them quickly, but we find the commonly-studied editing methods are needlessly destructive. With some easy-to-use tweaks, we avoid model degradation for WAY longer!

Our work on knowledge editing got an "Outstanding Paper Award"🏆🏆 at the AAAI KnowFM Workshop!! #AAAI2025 🥳🥳🥳 Congratulations to my amazing co-authors Tom Hartvigsen Ahmed Alaa Gopala Anumanchipalli

Really excited to share this work!

Super excited to share our new work on keeping LLMs factually up-to-date for long periods of time🎉

New #ICLR2025 paper to be presented by Sujay Nagaraj! It's the first method to capture how label noise can change over time in sequential classification tasks. Many cool implications, one being towards better models of labelers' behavior over time, even for non time series tasks🎉

Need a multilingual safety detector? 🚨Introducing PolyGuard🚨 ⚙️ supports 17 languages ⚙️ generates structured output for prompt safety, response safety, and model refusal 🚀 outperforms existing SOTA open and commercial safety detectors by 5.5% 📜 arxiv.org/abs/2504.04377🧵

Introducing TALES - Text Adventure Learning Environment Suite A benchmark of a few hundred text envs: science experiments and embodied cooking to solving murder mysteries. We test over 30 of the best LLM agents and pinpoint failure modes +how to improve 👨‍💻pip install tale-suite

Excited we have some papers accepted to ICML Conference in collaborations with some tremendous folks 🎉 Looking forward to Vancouver to discuss model editing for LLMs/VLMs and improving medical benchmarking!

🚨Happy to announce that one paper, "Understanding the Limits of Lifelong Knowledge Editing in LLMs", is accepted at #icml2025 ! Congrats to the wonderful authors Lukas Thede , Karsten Roth , Matthias Bethge ,Zeynep Akata , and Tom Hartvigsen. 👇 Highlights in the thread

Designing a hard but useful benchmark has always been a passion of mine. Here we present MedBrowseComp, a deep research + computer use benchmark that is easy to verify (like BrowseComp from OpenAI) but still very expandable 💊! Project page: moreirap12.github.io/mbc-browse-app/ 1/n

Just did a major revision to our paper on Lifelong Knowledge Editing!🔍 Key takeaway (+ our new title) - "Lifelong Knowledge Editing requires Better Regularization" Fixing this leads to consistent downstream performance! Tom Hartvigsen Ahmed Alaa Gopala Anumanchipalli Berkeley AI Research