PPO vs. DPO? 🤔 Our new paper proves that it depends on whether your models can represent the optimal policy and/or reward. Paper: arxiv.org/abs/2505.19770 Led by Ruizhe Shi Minhak Song

RLHF vs DPO under reward and/or policy model misspecification—when does each method succeed? Our new paper provides a fine-grained theoretical comparison. 📄 arxiv.org/abs/2505.19770

Very clear paper fleshing out different extensions of the story we outlined in arxiv.org/abs/2503.01067!

Interesting perspective. Misspecification does matter.

Schedule-Free methods, which forgo cosine/linear schedulers by averaging iterates and computing gradients at interpolated points, yield smoother training curves. It's still unclear why they work well, and this paper explains the phenomenon through the river-valley loss landscape.

Really nice use of the central flow framework!

Even with full-batch gradients, DL optimizers defy classical optimization theory, as they operate at the *edge of stability.* With Alex Damian, we introduce "central flows": a theoretical tool to analyze these dynamics that makes accurate quantitative predictions on real NNs.