🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Happy to share that GIDD was accepted at ICML 2025! 🥳 See our thread to learn how we added self-correction capabilities to discrete diffusion models with a simple change in the noise schedule 👇

The switch from a hard to a soft reward going from o1 to o3 really shows when giving it math tasks. If it cannot solve the task, o3 will just make up some bullshit that tricks the verifier but is actually wrong. This never happened with o1.

you love to see it when the new run peeks below all the others

Huge step forward for non-AR models! Breaking away from the fixed sequence length and has been a big challenge for discrete diffusion/flow models due to the position-independence assumption. Only a question of time until AR LLMs become obsolete? 👀

*Generalized Interpolating Discrete Diffusion* by Dimitri von Rütte Antonio Orvieto & al. A class of discrete diffusion models combining standard masking with uniform noise to allow the model to potentially "correct" previously wrong tokens. arxiv.org/abs/2503.04482

Ever wonder why switching optimizers never actually helps? Introducing HeavyBall Benchmark

why we use BF16 in one picture

zuck reads arxiv like a shopping list

who doesn't love some good anti-AR propaganda on a sunny Friday afternoon!