Thanks for acknowledging Dynamic Token Pooling as a predecessor to H-Net, Albert Gu! We had some decent ideas in that paper (e2e and entropy-based tokenisation), but it surprises me that it took 2 years (an eternity in NLP) to find the right recipe and scale better than BPE

If you are at ICML Conference make sure to attend Adrian Lancucki’s invited talk on our inference-time *hyper*-scaling paper (and more!) at the tokenization workshop this Friday tokenization-workshop.github.io/schedule/

We blend imitation (SFT) and exploration (RLVR) in post-training with a simple idea: Sample a prefix of an SFT demonstration, let your policy model complete it, and mix it with other RLVR rollouts Intuitively, the model relies more on hints for problems currently out of reach

*The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs* by Piotr Nawrot Edoardo Ponti Kelly Marchisio (St. Denis) Sebastian Ruder They study sparse attention techniques at scale, comparing to small dense models at the same compute budget. arxiv.org/abs/2504.17768

The amazing folks at EdinburghNLP will be presenting a few papers at ACL 2025 (ACL 2025); if you're in Vienna, touch base with them! Here are the papers in the main track 🧵

Reach out to Yifu Qiu if you’re looking for a research scientist starting next year! He is extremely talented and he’s been doing fantastic research on world models inside general-purpose LLMs/VLMs

Represent! ✌️

Massive congrats Weixuan Wang Minghao Wu Barry Haddow alexandra birch!!! 🚀🚀🚀🚀

A feast of SAC Highlights awards at ACL 2025 for my students! Lead authors: - Nina Gregorio & Matteo Gay for "The Cross-linguistic Role of Animacy in Grammar Structures" - Giwon Hong for "Mixtures of In-Context Learners" Massive congrats! 2025.aclweb.org/program/awards/

My Lab at the University of Edinburgh🇬🇧 has funded PhD positions for this cycle! We study the computational principles of how people learn, reason, and communicate. It's a new lab, and you will be playing a big role in shaping its culture and foundations. Spread the words!

Apply to ELLIS if you’d like to do a PhD in NLP/ML spending time in two different European universities!

With SEMI🌓, you can integrate entirely new modalities (satellite images, galaxies, inertia measurements, molecules, ...) into LLMs with as few as 32 samples!

Were you or someone you know the first generation in the family to go to university? The John Crook Scholarship at St John's offers the chance to study Cambridge University for two further years. Apply by 6pm BST on 15 October 2025 via UCAS Info👇 joh.cam.ac.uk/apply/undergra…

🌍 11 ELLIS Members and Scholars from five countries have received ERC Starting Grants! Congratulations to all awardees! 👏 Last week European Research Council (ERC) awarded 478 grants totaling €761M to support early-career researchers across Europe. 🔗 Learn more: ellis.eu/news/erc-award…

Scaling laws for test-time compute should not depend on the token budget, but on actual runtime! Generation can be sped up (KV cache compression, quantisation, etc.) while mostly maintaining quality, defining better Pareto frontiers: arxiv.org/pdf/2506.05345