Amazing. Incredibly based. Tessa Barton

Links (89) nintil.com/links-89

It warms my heart to see the underappreciated, compassionate work FarmKind is doing for farm animals. I grew up on a farm and I have a soft spot for the animals who feed us.

Deterministic inference, here you are. True on-policy RL is on the way. Although we are mostly using off-policy, having a deterministic mode will make many things easier!

I can't believe I am saying this, but Rich Sutton is looking dripped out

Today, I am launching Axiom At Axiom, we are building a self-improving superintelligent reasoner, starting with an AI mathematician.

“Health issues from 100% Huel diet discovered by 2035?” Is still only at 59% as of 10:30 this morning.

In all seriousness its really cool to see the gauntlet become a standard in evaluating base models. Tessa Barton Jeremy Dohmann Mansheej Paul Abhi Venigalla and I worked really hard thinking carefully about how to design aggregations that gave meaningful signal across model scales

Excited to share our latest work on untangling language models by training them with extremely sparse weights! We can isolate tiny circuits inside the model responsible for various simple behaviors and understand them unprecedentedly well. openai.com/index/understa…

Waymom

If you want a state of the art system, first you must fight state of the art bugs

Every time we train a great new model I need to frantically try to write a new take home that the model can’t defeat so we can still hire post-release. This one was tough, many drafts based on real problems fell before Claude Code’s “ultrathink” and needed to be scrapped.

Looking at the model’s internal feature activations, we noticed two things. (1) The model appeared to be internally aware that it was “holding back its true thoughts” and providing a fake summary. (2) The model seemed to interpret the results as a prompt injection attack. (3/7)

Astasia Myers IMO he was moreso talking about the marginal returns on increasing compute and how to allocate compute. (napkin math illustration bear with me) scaling pretraining and scaling RL is not over but the gains from scaling aren't addressing fundamental failures in models (he mainly

Today, we are releasing a new version of K2 (K2-V2), a 360-open LLM built from scratch as a superior base for reasoning adaptation, while still excelling at core LLM capabilities like conversation, knowledge retrieval, and long-context understanding. K2 fills a major gap: highly

I'm giving two talks at NeurIPS tomorrow! - iterative improvement of generative models, incl π0.6* (9:40 am, SPIGM workshop, with Yoonho Lee @NeurIPS) - long-horizon memory & autonomy (10:30 am, EWM workshop)

Good researchers obsess over evals The story of Olmo 3 (post-training), told through evals NeurIPS Talk tomorrow. Upper Level Room 2, 10:35AM.

incredible

Rich Sutton’s fits never miss. He wins both the Turing and Couture-ing Award!