Excited to share our latest research progress (joint work with Yang Song ): Consistency models can now scale stably to ImageNet 512x512 with up to 1.5B parameters using a simplified algorithm, and our 2-step samples closely approach the quality of diffusion models. See more

There’s a lot of awesome research about LLM reasoning right now. But how is  learning in the physical world 🤖different than in language 📚? In a new paper, show that imitation learning in continuous spaces can be exponentially harder than for discrete state spaces, even when

Accepted as a spotlight at #icml2025! See you in Vancouver 🎉

We had this vision a year ago, and it’s hard to believe how many dreams have come true since we filmed this video last summer. So much has changed—but one thing has stayed the same: our commitment to building a product for everyone, and giving people the power to create their own

Are modern large language models (LLMs) vulnerable to privacy attacks that can determine if given data was used for training? Models and dataset are quite large, what should we even expect? Our new paper looks into this exact question. 🧵 (1/10)

Another day without chatgpt OpenAI check your support email!!

Steering diffusion models with external rewards has recently led to exciting results, but what happens when the reward is inherently difficult? Introducing ReGuidance: a simple algorithm to (provably!) boost your favorite guidance method on hard problems! 🚀🚀🚀 A thread: (1/n)

A bit late to the party, but our paper on predictable inference-time / test-time scaling was accepted to #icml2025 🎉🎉🎉 TLDR: Best of N was shown to exhibit power (polynomial) law scaling (left), but maths suggest one should expect exponential scaling (center). We show how to

Announcing Ambient Diffusion Omni — a framework that uses synthetic, low-quality, and out-of-distribution data to improve diffusion models. State-of-the-art ImageNet performance. A strong text-to-image results in just 2 days on 8 GPUs. Filtering ❌ Clever data use ✅

1/8: The Emergent Misalignment paper showed LLMs trained on insecure code then want to enslave humanity...?! We're releasing two papers exploring why! We: - Open source small clean EM models - Show EM is driven by a single evil vector - Show EM has a mechanistic phase transition

1/6 Infinite-dim SGD in linear regression is the strawman model for studying scaling laws, critical batch sizes, and LR schedules. We revisit (and simplify) its analysis using just linear algebra, making it easier to derive and reason about. No PSD operators. No tensor calculus.

PufferLib 3.0: We trained reinforcement learning agents on 1 Petabyte / 12,000 years of data with 1 server. Now you can, too! Our latest release includes algorithmic breakthroughs, massively faster training, and 10 new environments. Live demos on our site. Volume on for trailer!

Excited to share that I’ll be presenting two oral papers in this ICML—see u guys in Vancouver!!🇨🇦 1️⃣ arxiv.org/abs/2502.06768 Understanding Masked Diffusion Models theoretically/scientifically 2️⃣ arxiv.org/abs/2502.09376 Theoretical analysis on LoRA training

Ecstatic to present an oral paper at ICML this year!!🎉 📚 “Blink of an Eye: a simple theory for feature localization in generative models” 🔗 arxiv.org/abs/2502.00921 Catch me at the poster session right after! See you there! 🚀

🚨 New preprint! 🚨 Everyone loves causal interp. It’s coherently defined! It makes testable predictions about mechanistic interventions! But what if we had a different objective: predicting model behavior not under mechanistic interventions, but on unseen input data?

Great to see this one finally out in PNAS! Asymptotic theory of in-context learning by linear attention pnas.org/doi/10.1073/pn… Many thanks to my amazing co-authors Yue Lu, Mary Letey, Jacob Zavatone-Veth and Anindita Maiti

🧵generative models are sweet, but navigating existing repositories can be overwhelming, particularly when starting a new research project so i built jax-interpolants, a clean & flexible implementation of the stochastic interpolant framework in jax github.com/nmboffi/jax-in…

New @Scale_AI paper! 🌟 LLMs trained with RL can exploit reward hacks but not mention this in their CoT. We introduce verbalization fine-tuning (VFT)—teaching models to say when they're reward hacking—dramatically reducing the rate of undetected hacks (6% vs. baseline of 88%).

Thrilled to share that our work received the Outstanding Paper Award at ICML! I will be giving the oral presentation on Tuesday at 4:15 PM. Jaeyeon (Jay) Kim @ICML and I both will be at the poster session shortly after the oral presentation. Please attend if possible!

Three weeks ago, we started building an AI game engine. But some models kept making things look... sloppy. So we turned finding the best one into a game. In three weeks, that game grew to 35K+ users across 135 countries. Introducing Design Arena, the fastest-growing