📢 New paper from my internship at cohere with Seraphina Goldfarb-Tarrant @ FAccT 🇬🇷 ‼️ Are you interested in investigating the fairness of LLMs in hiring contexts? Take a look at our work 🧵 arxiv.org/abs/2501.04316

In 🇫🇷 for the AI Action Summit! Say hi if you're here!

Radar charts flame 🔥 reloaded, Gemma3 report edition. Axes/radials are rescaled for each capability, so the distance between points varies, not just between charts, but *within* a chart 😱. It would be challenging to make a bar plot this misleading, if trying. 🧪 👮 are coming.

GPT-4o level performance ✅ 75% faster ✅ 256k context length ✅ Open weight ✅ Fits on 2xH100s ✅ Add to basket

🔥 new model 🔥 Our approach to safety is pretty unique ✨. We have 3 focuses: controllability for context dependent safety (risks differ across use cases), enterprise fairness (equal treatement in tasks on human data), and open weights safety (baseline safety for everything).

223 (!!!) people worked on this tech report, it was definitely the biggest coordination effort of a scientific artifact that I've ever done. Totally worth it.

Everything you need to know about how to explain scientific concepts like merging clearly and concisely with just a couple of colours is in Figures 2-4 in this report, courtesy of Jay Alammar

This work was integral to Command A's development process described in the tech report (cohere.com/research/paper…), and enabled higher quality auto evaluations and fast iterations. Grateful to have been part of this cohere and mentored by Seraphina Goldfarb-Tarrant @ ACL 🇦🇹!

Charlie did this work with the safety team cohere on the robustness of LLM evaluators and found them to be surprisingly vulnerable to basic perturbations. This was really important research that was a prereq for creating evals at Cohere that are robust to spurious correlations!

🎉 Our Actionable Interpretability workshop has been accepted to #ICML2025! 🎉 >> Follow Actionable Interpretability Workshop ICML2025 Tal Haklay Anja Reusch Marius Mosbach Sarah Wiegreffe Ian Tenney (@[email protected]) Mor Geva Paper submission deadline: May 9th!

I will be in 🇬🇷 for ACM FAccT next Mon-Thurs, the first time I've been to one in 5 years! ☕️ chat with me about research, internships, collabs, etc -- I am largely there to talk to people :).