#NLProc New paper on "evaluation-time scaling", a new dimension to leverage test-time compute! We replicate the test-time scaling behaviors observed in generators (e.g., o1, r1, s1) with evaluators by enforcing to generate additional reasoning tokens. arxiv.org/abs/2503.19877

Check out Ian's thread on our paper on scaling evaluation-time compute with reasoning models as process evaluators. Two main findings: (1) reasoning models are good process evaluators; (2) we can use reasoning evaluators for imrpoving LLM generation. See details in the thread 👇

Exciting SOTA results on SWE-Bench! But I'm also equally excited by the possibilities of being able to have coding agents "abstain" from sending a patch when the result is low-quality. This'll save lots of code review time!

I am pleased to announce that I have updated the online versions of my 2 textbooks (see probml.github.io/pml-book/): I fixed all issues listed on github, added some new references (esp on LLMs), and made a few other small tweaks.

We have to remember that what we observe is not nature herself, but nature exposed to our method of questioning. - Werner Heisenberg

Glad to share that our AgoraBench paper has been accepted at ACL 2025 2025 (main)! Special thanks to our coauthors JuYoung Suk Xiang Yue Vijay V. Seongyun Lee Yizhong Wang Kiril Gashteovski Carolin Sean Welleck Graham Neubig! A belief I hold more firmly now than when I started this project

Tomorrow, I will deliver a talk at the University of Edinburgh at 11:00, titled "Grounded Intelligence: Towards Reliable and Explainable LLM Systems through Synthetic Data, Evaluation, and Modular Design". If you are in the area, drop by and let's chat afterwards

Thrilled to introduce AlphaGenome, our new DNA sequence model now available via our AlphaGenome API. Really excited to see how the scientific community uses AlphaGenome’s predictions to understand genome function, drive biological discoveries, develop new treatments, and more...

Besides the citation count, here's a new metric for a popularity of a research paper: having its own Wikipedia article!

Checkout our paper on medical AI agent -MEDDxAgent: A Unified Modular Agent Framework for Explainable Automatic Differential Diagnosis

It was pleasure to host this great talk by Joshua Joshua Ong on Autoformalisation and Symbolic Reasoning for Mathematical Reasoning. Check it out on our YouTube channel 👇

Beautiful pattern in mathematics ✍️

'Theorem Prover as a Judge for Sythetic Data Generation' has been accepted to ACL (Main) 🚀. Do check us out at July 30th (Wednesday) 11:00- 12:30pm at Hall 4/5! A huge thank you to my amazing collaborators: Shay Giwon Hong Wenda Li 📝: aclanthology.org/2025.acl-long.…