Discrete Diffusion Models (1.1B) 1. show competitive results in terms of zero-shot benchmarks. 2. x1.5 faster with comparable perf. 3. do not suffer _reverse curse_ thanks to bidirectional attn.

Glad to have worked with Jiaqi Han on this project. Very promising results for temporal generative models!

Excited to share our paper “Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-Based Decoding”. My amazing coauthors: Yulai Zhao, Chenyu (Monica) Wang, Gabriele Scalia, gokcen, Surag Nair, Tommaso Biancalani, Shuiwang Ji, Aviv Regev, Sergey Levine, Masatoshi Uehara

🌟 Excited to share our latest work on making diffusion language models (DLMs) faster than autoregressive (AR) models! ⚡ It’s been great to work on this with Caglar Gulcehre 😎 Lately, DLMs are gaining traction as a promising alternative to autoregressive sequence modeling 👀 1/14 🧵

Very exciting work from Justin Deschenaux on fast discrete diffusion models! I'm especially excited since I had lost hope for such distillation techniques.

📢Annoucing EDLM, our brand-new Energy-based Language Model embedded with Diffusion framework! Key results: 1. We (for the first time?) almost match AR perplexity. 2. Significantly improved generation quality. 3. Considerable sampling speedup without quality drop. 🧵1/n

A new Science study presents “Evo”—a machine learning model capable of decoding and designing DNA, RNA, and protein sequences, from molecular to genome scale, with unparalleled accuracy. Evo’s ability to predict, generate, and engineer entire genomic sequences could change the

🧬Evo, the first foundation model trained at scale on DNA, is a Rosetta Stone for biology. DNA, RNA, and proteins are the fundamental molecules of life—and cracking the code of their complex language is an ongoing grand challenge. 🔬Today in Science Magazine, the labs of Arc

Evo has been published in @Science! A true privilege to work with such an amazing team! So many exciting new experimental results in this emerging field of Generative Genomics, including AI generated and *validated* CRISPR-Cas systems and transposons

Tanishq Mathew Abraham, Ph.D. Dylan Patel SemiAnalysis Biden admin is in the Big Autoregression's pocket

It's tough times for science. 🥺 But we have to keep innovating to fight another day, and today I'm so proud to share Fred Zhangzhi Peng's new, groundbreaking sampling algorithm for generative language models, Path Planning (P2). 🌟 📜: arxiv.org/abs/2502.03540 💻: In the appendix!!

Large Language Diffusion Models Introduces LLaDA-8B, a large language diffusion model that pretrained on 2.3 trillion tokens using 0.13 million H800 GPU hours, followed by SFT on 4.5 million pairs. LLaDA 8B surpasses Llama-2 7B on nearly all 15 standard zero/few-shot learning

🚀Excited to announce Dream 7B (Diffusion reasoning model): the most powerful open diffusion large language model to date.

Life update: I started Radical Numerics with Stefano Massaroli, Armin Thomas, Eric Nguyen, and a fantastic team of engineers and researchers. We are building the engine for recursive self‑improvement (RSI): AI that designs and refines AI, accelerating discovery across science and

✨ Excited to share a few life updates! 🎤 My TED Talk is now live! I shared the origin story of Evo, titled: "How AI could generate new life forms" TED talk: ted.com/talks/eric_ngu… ✍️ I wrote a blog post about what it’s *really* like to deliver a TED talk blog:

1/n I’m really excited to share that our OpenAI reasoning system got a perfect score of 12/12 during the 2025 ICPC World Finals, the premier collegiate programming competition where top university teams from around the world solve complex algorithmic problems. This would have

Our general-purpose reasoning models solved all 12 problems at the 2025 International Collegiate Programming Contest (ICPC) World Finals, the world’s top university programming competition which was enough for a 1st-place human ranking.

Introducing RND1, the most powerful base diffusion language model (DLM) to date. RND1 (Radical Numerics Diffusion) is an experimental DLM with 30B params (3B active) with a sparse MoE architecture. We are making it open source, releasing weights, training details, and code to

Amazing work by Radical Numerics!

I turned Andrej Karpathy's baby GPT into a character-level text diffusion model, using Aaron Lou et al.'s score entropy-based training objective.