awards.acm.org/about/2024-tur… Machines that learn from experience were explored by Alan Turing almost eighty years ago, which makes it particularly gratifying and humbling to receive an award in his name for reviving this essential but still nascent idea.

lol half of the replies to this are like "use dspy" or "use this company/service/library downstream of dspy" lol, but what Tom is describing is IMO not the right problem it's not that "prompts are hard", I trust you can write them, they're just the wrong canvas for AI systems

on my way back to NYC, i met wise Leon Bottou in the airport. we talked. then i told him "you should tweet that!" and, he delivered much more than a tweet: a blog post with thoughts and insights on AI research only he can deliver this clearly and succinctly.

Wrapped up Stanford CS336 (Language Models from Scratch), taught with an amazing team Tatsunori Hashimoto Marcel Rød Neil Band Rohith Kuditipudi. Researchers are becoming detached from the technical details of how LMs work. In CS336, we try to fix that by having students build everything:

2025: Is AI making us stupid? 2016: Are phones Making Us Stupid? 2008: Is Google Making Us Stupid? 1884: Are Books Making Us Stupid?

Tough life if you naturally use em dashes—like this—everyone thinks you are an AI copy paster. I now need to force myself not to use them.

Today I finished my PhD at Auburn University Ginn College of Engineering with Anh Totti Nguyen . What’s next? Off to Guide Labs to build foundation AI models that are interpretable-by-design with Julius Adebayo and a crew of amazing people 🌁

Diffusion Beats Autoregressive in Data-Constrained Settings Comparison of diffusion and autoregressive language models from 7M to 2.5B params and up to 80B training tokens. Key findings: 1. Diffusion models surpass autoregressive models given sufficient compute. Across a wide

Forbes Phosphor Capital Dasha Shunina Y Combinator Auctomatic Zeus Living Full story here: forbes.com/sites/dariashu…

“What matters is training on the task you will test on” the lesson we learn over and over again.

Real math = only Grothendieck. Real chess = Carlsen not Stockfish. Real Go = Lee Sedol (AlphaGo doesn't count). Real thinking = Einstein not plebs. Real driving = F1 drivers not Waymo. Lmao what a take.

COLM reviewer on OLMO-2 paper: 'novelty is limited' 😅. The paper is one of the best references for open-source models.

youtu.be/hguIUmMsvA4

(1/n) Check out our new paper: "Fantastic Pretraining Optimizers and Where to Find Them"! >4000 models to find the fastest optimizer! 2× speedups over AdamW? Unlikely. Beware under-tuned baseline or limited scale! E.g. Muon: ~40% speedups <0.5B & only 10% at 1.2B (8× Chinchilla)!

'Today', but so were GANs pre-diffusion, so was deep RL pre-transformers, and RNNs pre-attention.

It is 2016-2019 again. RL papers in nature .

Nice evaluation paper. When you ask an LLM to explain the activations of another LLM. Turns out the explainer-LLM just explains itself instead!