What would it take for AI evaluations to truly support our global experiences? 🌍 Our cross-institutional paper introduces INCLUDE, a multilingual LLM evaluation benchmark of local exams capturing in-language nuances & cultural context for truly localized AI evaluation.

Today, we are releasing MSTS, a new Multimodal Safety Test Suite for vision-language models! MSTS is exciting because it tests for safety risks *created by multimodality*. Each prompt consists of a text + image that *only in combination* reveal their full unsafe meaning. 🧵

My first ICLR paper 🥳🍻 congrats to the whole crew

Will be a pretty cool workshop! If you have relevant work, consider submitting there

Our updated Responsible Foundation Model Development Cheatsheet (250+ tools & resources) is now officially accepted to Transactions on Machine Learning Research 2025! It covers: - data sourcing, - documentation, - environmental impact, - risk eval - model release & licensing

🚀 Excited to announce that I’ll be teaching a course on "Large Language Models - Tune and Train Your Own Models" MethodsNET in Vienna from July 14-18 🤖💡 If you’re interested in adjusting LLMs for your tasks, this is for you. Can’t wait for an exciting week.

I am hiring a Student Researcher for our Modularity team at the Google DeepMind office in Zurich🇨🇭 Please fill out the interest form if you would like to work with us! The role would start mid/end 2025 and would be in-person in Zurich with 80-100% at GDM forms.gle/N94ViTmKHCCAcv…

Check out our very popular "Tutorial on Causality for LLMs" at #NeurIPS2024: 2-hour detailed intro + a super insightful panel w/ Yoshua Bengio Bernhard Schölkopf Swaroop Mishra Giambattista 'Gb' Parascandolo Tom Everitt on "The Role of Causality in Future LLMs" 📺youtube.com/watch?v=8j1fS_… with slides

🕵️‍♀️ Join us for the first "Validating Social Text-as-Data Method" workshop (June 23, 2025, Copenhagen) at ICWSM sites.google.com/view/valistad2…

🥁 It's the second half of our 🌱 speaker series this term, and we couldn't be more excited! Next week (Wednesday, April 30 at 5pm CET), Tanise Ceron will share insights on "Facilitating Information Access Through Language Models". More details tada.cool⬇️

I won’t be at #ICLR2025 unfortunately (no, im not FOMOing with all the twitter posts and friends sharing stories on IG 😅) but if you’re there, check the work on multimodal data provenance and chat with Shayne Longpre! Happy ICLRing! 🇸🇬 arxiv.org/abs/2412.17847

✨ Don’t miss the chance to meet Angelika Romanou and learn more about the INCLUDE project 🗺️ and check out our spotlight poster on Thu 3:00-5:30pm ⬇️

Fantastic first day at #COMPTEXT2025. Really enjoyed spreading the word about small & efficient LLMs 🌱✨🦙 in our workshop (w/ Julia Romberg) today. Many thanks to everyone who joined us 💚. Excited for more packed conference days ahead 🚀

🕔 We're excited to start in just 30 minutes (5pm Berlin time). Come join us for a fascinating talk by Tanise Ceron on "Evaluating Political Bias: Insights into Robustness and Multilinguality". We can’t wait to see many of you there ⬇️

🚨Just a month until the Computational Linguistics for the Political and Social Sciences (CPSS) workshop deadline (June 13, AoE) #KONVENS2025. Super excited for the upcoming round and can’t wait to see your submissions ✨ cpss-sig.github.io/CPSS-2025/inde…

Check it out! The newest version of AdapterHub is on 🔥

🚨 Lucie-Aimée Kaffee and I are looking for a junior collaborator to research the Open Model Ecosystem! 🤖 Ideally, someone w/ AI/ML background, who can help w/ annotation pipeline + analysis. docs.google.com/forms/d/e/1FAI…

Call for Papers: The 4th Annual Zurich Workshop in AI+Economics to be held Dec 5-6, 2025, hosted by ETH Zurich and University of Zurich. illuminating keynote to be given by tom cunningham (OpenAI) organized with S Galletta D. Yanagizawa-Drott Joachim Voth Info:

📢📢👇New job openings. Topic: social bias detection+analysis with LLMs across time (1950-now) & languages. There are 2 Post-Doc/PhD positions, supervised by Steffen Eger (Technische Universität Nürnberg)+Simone Ponzetto (Data and Web Science Group). Fully funded, up to 3 yrs. More infos: nl2g.github.io/positions