We are launching HALoGEN💡, a way to systematically study *when* and *why* LLMs still hallucinate. New work w/ Shrusti Ghela* David Wadden Yejin Choi 💫 🧵 [1/n]

At ICLR and interested in LLM evaluation? Go check out 𝕎𝕚𝕝𝕕𝔹𝕖𝕟𝕔𝕙, our work on benchmarking LLMs on complex real-world tasks 👇

Stoked that HALoGEN (non-archival version) won best paper award at the TrustNLP workshop @ #NAACL2025! Our work explores LLM hallucinations and their potential roots in training data. Excited to discuss more --- come find us!

I'm headed to #NAACL2025 ✈️ in Albuquerque 🏜️Looking for postdoc positions in the US, so if you're hiring (or know someone who is), let's chat at the conference! Also organizing #WNU2025 so make sure to swing by the workshop on May 4

I'll be presenting two papers NAACL HLT 2025! 1. Why LLMs can't write a question with the answer "468" 🤔🙅 2. A multi-agent LLM that balances opinions on "is pineapple good on pizza?" 🎭🍕 Let's also chat about: 📍 Helpfulness 📍 Why MCQA sucks 📍 Generating cute paper titles 👀

This is happening in 10min (first talk of the session) in Ballroom C!

I will present this paper this afternoon at 2pm in Hall 3 (Poster Session 5). Please stop by! #NAACL2025

Come catch our talk today at 5 PM in San Miguel!

A great PhD advisor helps you discover the best version of yourself! In doing so, they'll help set a research vision, set yearly plans, break the problem down to solvable pieces, how to measure success, when to pivot, how to communicate clearly & concisely (written/verbal), etc.

This is an awesome awesome-llm-unlearning resource, maintained by Chris Liu : github.com/chrisliu298/aw…

HALoGEN will appear at #ACL2025, see you in Vienna!

HALoGEN has been accepted to #ACL2025 main!! We also presented (non-archival) at trustnlp #NAACL2025 - and it won the Best Paper award!! 👑 My first paper — and all of this feels surreal, and if this is how it begins, I’m not complaining.

What would truly open-source AI look like? Not just open weights, open code/data, but *open development*, where the entire research and development process is public *and* anyone can contribute. We built Marin, an open lab, to fulfill this vision:

🥳Excited to share our paper RESTOR: Knowledge Recovery in Machine Unlearning is accepted to #TMLR! We introduce a benchmark where unlearning means not just forgetting target data, but recovering the behavior of an ideal model that never saw them. Paper: openreview.net/pdf?id=BbwlJpN…

𝐖𝐡𝐚𝐭 𝐇𝐚𝐬 𝐁𝐞𝐞𝐧 𝐋𝐨𝐬𝐭 𝐖𝐢𝐭𝐡 𝐒𝐲𝐧𝐭𝐡𝐞𝐭𝐢𝐜 𝐄𝐯𝐚𝐥𝐮𝐚𝐭𝐢𝐨𝐧? I'm happy to announce that the preprint release of my first project is online! Developed with the amazing support of Abhilasha Ravichander and Ana Marasović (Full link below 👇)