My notes on nanochat, including links to the training data it uses simonwillison.net/2025/Oct/13/na…

RLMs are meant to address context rot, which is that weird effect when you have a long Claude Code or Cursor instance where it can’t properly handle your long history. OOLONG is a challenging new long context benchmark where models answer queries over an extremely dense context.

Nearly all journalists leave Pentagon after refusing to sign new press access rules

For now, the place to find news that most people don't know about is in books, because AI search hasn't reached there yet. On the other hand, Web research offers a lot of lore that older books don't know about. Current books, like my MAINTENANCE and Brian Potter's THE ORIGINS

18 months ago, Andrej Karpathy set a challenge: "Can you take my 2h13m tokenizer video and translate [into] a book chapter". We've done it! It includes prose, code & key images. It's a great way to learn this key piece of how LLMs work. fast.ai/posts/2025-10-…

Wow crazy result

Btw the model router is the amazing open model Arch-Router-1.5B by katanemo Salman Paracha (Intelligent Infrastructure ): huggingface.co/katanemo/Arch-…

Librarians, help us celebrate 1 trillion web pages preserved by the @InternetArchive! 🌐 Use our resource guide, complete with templates, visuals & event ideas, to connect your community to the web’s history. More ⤵️ blog.archive.org/2025/10/07/cal… #Wayback1T #libraries #librarians

You spend $1B training a model A. Someone on your team leaves and launches their own model API B. You're suspicious. Was B was derived (e.g., fine-tuned) from A? But you only have blackbox access to B... With our paper, you can still tell with strong statistical guarantees

too many models, not enough evals.

Our Huxley-Gödel Machine learns to rewrite its own code, estimating its own long-term self-improvement potential. It generalizes on new tasks (SWE-Bench Lite), matching the best officially checked human-engineered agents. Arxiv 2510.21614  With Wenyi Wang, Piotr Piękos,

Very nice blog post from Thinky (Kevin Lu et al) about on-policy distillation for LLMs -- we published this idea back in 2023 and it is *publicly* known to be successfully applied to Gemma 2 & 3, and Qwen3-Thinking (and probably many closed frontier models)! The idea behind

Even before MMitchell recently raised this discussion, I've had conversation after conversation with students & new grads struggling with this exact dilemma. I want to help! Here's a live thread of AI-related opportunities for those looking to do good & make (enough) money:

New paper: 🧭 Introducing VAL-Bench: Measuring Value Alignment in Language Models. A benchmark that measures the consistency in language model expression of human values when prompted to justify opposing positions on real-life issues. Work with Aman Gupta and Denny O'Shea!

After two years of work, we’ve made an AI Scientist that runs for days and makes genuine discoveries. Working with external collaborators, we report seven externally validated discoveries across multiple fields. It is available right now for anyone to use. 1/5

👀 tempting to reproduce this with [model to come]

Seconded! :)

From Terry Tao on Mathstodon: “ A new paper with Bogdan Georgiev, Javier Gomez-Serrano, and Adam Zsolt Wagner: "Mathematical exploration and discovery at scale" arxiv.org/abs/2511.02864 , in which we record our experiments using the LLM-powered optimization tool #AlphaEvolve to

👹😁

One of the few podcasts where I talk about longevity. I share some of my findings, including an experiment I did where I purposefully gained and lost 70 lbs to prove a point 😅