🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Glad to see this piece out in National Academy of Engineering for their latest edition on "AI Promises & Risks". So many of the practical evaluation challenges with generative AI have already been encountered before. Not surprising how much we can learn from other fields! nae.edu/337862/Spring-…

Our multimodal AMIE study is out! This is another exciting step for conversational medical AI – many congrats to the AMIE team! A brief 🧵on what this means for both doctors and patients: x.com/GoogleAI/statu…

We have two papers accepted at #ICML2025! An oral on construct validity of medical LLM benchmarks and a poster on generalized Venn-Abers calibration. See you in Vancouver 🇨🇦!

Excited to share that our paper on uncertainty-aware model calibration using generalized Venn–Abers methods was accepted to #ICML2025! We show that Vovk’s Venn–Abers calibration and conformal prediction are both special cases of a unified framework.

Great to see our work with Maya Petersen, Emre Kıcıman, Chris Holmes and Mark van der Laan featured by Microsoft Research!

📢If you're interested in conformal prediction, algorithms w/predictions, robust stats & connections between them from a theory perspective, join us for a workshop at #COLT2025 in Lyon 🇫🇷 June 30! Submit a poster description by May 25, more here: vaidehi8913.github.io/predictions-an…

Our Rising Stars series is back with Paula Nicoleta Gradu sharing a control-theoretic perspective on medication tapering problems! Link: youtu.be/IP9unMmvbNE?si…

Super exciting. Come join us to work on reliable AI and community evaluations! There's a lot of exciting research to be done and products to be built. If you're interested in helping, apply to join, and my DMs/e-mail are always open :)

Just did a major revision to our paper on Lifelong Knowledge Editing!🔍 Key takeaway (+ our new title) - "Lifelong Knowledge Editing requires Better Regularization" Fixing this leads to consistent downstream performance! Tom Hartvigsen Ahmed Alaa Gopala Anumanchipalli Berkeley AI Research

Are you a health AI researcher thinking about founding a company? Are you a founder in health AI? Lucky you because we're holding an intimate gathering at UC Berkeley on June 25 with a great line-up and lots of built-in time for mingling

Honored to receive a Google Research Scholar Award. Thanks Google Research for supporting our work!

What more could we understand about the fractal, “jagged” edges of AI system deployments if we had better ways to listen to the people who interact with them? What a joy to work w jessica dai using individual experiences to inform AI evaluation (blog/ICML/arXiv links 👇)

individual reporting for post-deployment evals — a little manifesto (& new preprints!) tldr: end users have unique insights about how deployed systems are failing; we should figure out how to translate their experiences into formal evaluations of those systems.