1/5. Excited to share a spicy paper, "Rethinking conventional wisdom in machine learning: from generalization to scaling", arxiv.org/pdf/2409.15156. You might love it or dislike it! NotebookLM: notebooklm.google.com/notebook/43f11… While double-descent (generalization-centric,

Good post-training data is precious and scarce; compute is less so. We should focus on methods which squeeze more out of existing data by spending additional compute per datapoint, rather than optimizing for cheaper post-training methods

We finally have an official `nvidia-smi` for TPU 🎉 Simply install it with `pip install tpu-info`

People learning JAX, feel free to reach out if the learning feels too steep, hopefully we can flatten it out. Also, checkout the JAX LLM for help from the community: discord.gg/m9NDrmENe2

Can we predict emergent capabilities in GPT-N+1🌌 using only GPT-N model checkpoints, which have random performance on the task? We propose a method for doing exactly this in our paper “Predicting Emergent Capabilities by Finetuning”🧵

People completely misunderstand the data wall. It's the data slop wall. Most of the data is so bad it's a waste of a good gpu to backprop it.

Very excited to share the thinking model! It has been a lot of fun working on this.

Appreciate Aidan McLaughlin looking into the thinking model results. Originally scores looked weak as the response was plucked from the thought content versus output. We are looking into ways of making thinking output less confusing for people running evals. This is why we 🚢, to

Simplify. Scale. Resolve bottlenecks. Repeat.

It’s done because it’s much easier to 1) collect, 2) evaluate, and 3) beat and make progress on. We’re going to see every task that is served neatly packaged on a platter like this improved (including those that need PhD-grade expertise). But jobs (even intern-level) that need

Whether you like it or not, the future of AI will not be canned genies controlled by a "safety panel". The future of AI is democratization. Every internet rando will run not just o1, but o8, o9 on their toaster laptop. It's the tide of history that we should surf on, not swim

For friends of open source: imo the highest leverage thing you can do is help construct a high diversity of RL environments that help elicit LLM cognitive strategies. To build a gym of sorts. This is a highly parallelizable task, which favors a large community of collaborators.

Despite many complaints about Jax being hard to use, it has a crucial advantage over PyTorch: for distributed jobs, XLA is sufficiently good at auto-scheduling parallelism strategies, e.g., sharding, overlapping compute and comms. If PyTorch becomes good at that, it's checkmate.

Making LLMs run efficiently can feel scary, but scaling isn’t magic, it’s math! We wanted to demystify the “systems view” of LLMs and wrote a little textbook called “How To Scale Your Model” which we’re releasing today. 1/n

Deepseek R1 inference in pure JAX! Currently on TPU, with GPU and distilled models in-progress. Features MLA-style attention, expert/tensor parallelism & int8 quantization. Contributions welcome!

Today we are launching 2.5 Pro! I think it's the best model in the world. State-of-the-art reasoning and great vibes (+39 ELO gap on lmsys!) 2.5 Pro improves in coding, stem, multimodal, instruction following, and lots more. Available in AI Studio & the Gemini App!

The team has built such a smart model. The gap becomes bigger on harder problems!

Llama 4 inference in pure JAX! Expert/tensor parallelism with int8 quantization. Contributions welcome!

2.5 Flash is out! You can now specify thinking budgets, or disable thinking entirely for lower latency. Strong code & reasoning capabilities, cost effective, fast. It's a great workhorse model for enterprise and developers, excited to hear your feedback.

Today we're putting out an update to the JAX TPU book, this time on GPUs. How do GPUs work, especially compared to TPUs? How are they networked? And how does this affect LLM training? 1/n