What do BPE tokenizers reveal about their training data?🧐 We develop an attack🗡️ that uncovers the training data mixtures📊 of commercial LLM tokenizers (incl. GPT-4o), using their ordered merge lists! Co-1⃣st Jonathan Hayase arxiv.org/abs/2407.16607 🧵⬇️

Doing good science is 90% finding a science buddy to constantly talk to about the project.

Super excited to announce that Microsoft Research's FATE group, Sociotechnical Alignment Center, and friends have several workshop papers at next week's NeurIPS Conference. A short thread about (some of) these papers below... #NeurIPS2024

Tim Vieira and I will be presenting this work at NeurIPS today! Join us from 16:40h at East Exhibition Hall A!

Super excited for the Evaluating Evaluations workshop at NeurIPS Conference today!!! evaleval.github.io #NeurIPS2024 Microsoft Research's FATE group, Sociotechnical Alignment Center, and friends will be presenting several papers there. See below for details...

I'll be giving a short talk on "Evaluating GenAI Systems is a Social Science Measurement Challenge" (arxiv.org/abs/2411.10939) in the 230--3pm oral session.

Tim Vieira and I were just discussing this interesting comment in the DeepSeek paper introducing GRPO: a different way of setting up the KL loss. It's a little hard to reason about what this does to the objective. 1/

Today we are launching a server dedicated to Tokenization research! Come join us! discord.gg/CDJhnSvU

New preprint on controlled generation from LMs! I'll be presenting at NENLP tomorrow 12:50-2:00pm Longer thread coming soon :)

Tackling complex problems with LMs requires search/planning, but how should test-time compute be structured? Introducing Self-Steering, a new meta-reasoning framework where LMs coordinate their own inference procedures by writing code!

A new technique from MIT can make AI-generated code adhere to whatever programming language or other format is being used, while remaining error-free: bit.ly/43U2Pua

#ICLR2025 Oral How can we control LMs using diverse signals such as static analyses, test cases, and simulations? In our paper “Syntactic and Semantic Control of Large Language Models via Sequential Monte Carlo” we: Cast controlled generation as an inference problem, with the LM

Excited to rep the team behind “Syntactic and Semantic Control of LLMs via Sequential Monte Carlo” ICLR 2026 #ICLR2025!🎲🎛️ Stop by our poster #634 from 10:00am-12:30pm today to chat with co-authors João Loula, Ben LeBrun, Alex Lew, Tim Vieira, Ryan Cotterell & more!

Current KL estimation practices in RLHF can generate high variance and even negative values! We propose a provably better estimator that only takes a few lines of code to implement.🧵👇 w/ Tim Vieira and Ryan Cotterell code: arxiv.org/pdf/2504.10637 paper: github.com/rycolab/kl-rb

This week on "How isn't everyone doing this already!?!?" ...

Many LM applications may be formulated as targeting some (Boolean) constraint. Generate a… - Python program that passes a test suite - PDDL plan that satisfies a goal - CoT trajectory that yields a positive reward The list goes on… How can we efficiently satisfy these? 🧵👇

Alright, people, let's be honest: GenAI systems are everywhere, and figuring out whether they're any good is a total mess. Should we use them? Where? How? Do they need a total overhaul?

Generative language systems are everywhere, and many of them stereotype, demean, or erase particular social groups.

📣 "Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems" is forthcoming at #ACL2025NLP---and you can read it now on arXiv! 🔗: arxiv.org/pdf/2506.04482 🧵: ⬇️

Has anyone tried running AI models (CNNs/LLMs, ViTs/ Diffusion) on weird chips? Edge: Qualcomm AR1, Ambarella, TensTorrent Cloud: Trainium, Inferentia, AMD Or even just porting Ampere → Hopper → Blackwell? Curious: how painful was it? Did it kill your project before it started?