First #CallForPapers for #INLG2025! Submit work on any aspect of #NaturalLanguagGeneration, incl. but not limited to: rule-based data-to-text systems, summarisation and simplification with the latest LLMs, or new evaluation methods :) Deadline: 14 July 2025.inlgmeeting.org

Me after defending my dissertation:

Excited to fly to Albuquerque to present my latest piece at NAACL! If you want to learn how to design human-centered #NLProc evaluation UIs, visit my poster on May 1, Hall 3, 14:00-15:30. We can also chat about NLG, hallucinations, lexical change, and anything in between! :)

"LLM on way to replace doctors" gets published in Nature. meanwhile "LLM judgement not as good as human MDs" gets a spot in "Physical Therapy and Rehabilitation Journal".

New blog: Key messages from my NLG book Its been 6 months since my NLG book was released. I summarise what I think are its key messages, for rule-based NLG, ML and neural NLG, requirements, evaluation, safety/testing/maintainability, and applications. ehudreiter.com/2025/05/14/key…

Claude 4 just refactored my entire codebase in one call. 25 tool invocations. 3,000+ new lines. 12 brand new files. It modularized everything. Broke up monoliths. Cleaned up spaghetti. None of it worked. But boy was it beautiful.

Best writing advice

Opinion: Authors should be allowed to add hidden prompts for LLMs in papers. If you lazily paste the paper you're supposed to judge on ChatGPT etc., you don't belong in peer reviewing.

The Second #CallForPapers just went out and announces two of our keynote speakers: Verena Rieser (Google DeepMind) & Minlie Huang (Tsinghua University)! Submit your work on NLG, whether LLM or rule-based :D Deadline: 14 July 2025.inlgmeeting.org (first posted elsewhere)

We just received some reviews for EMNLP and I'm filled with an immense amount of rage.

Reviewing Getting reviewed

Remember my tweet from the other day? Well, this is not what I meant.

Microsoft claims their new AI framework diagnoses 4x better than doctors. I'm a medical doctor and I actually read the paper. Here's my perspective on why this is both impressive AND misleading ... 🧵

as a parent, i will never push a career path onto my kids. i would give them full freedom to decide which AI lab they want to join for $100 mil

I love this analysis of the limitations of the experimental setting/design. This is the kind of expert insight and methodological rigor we need when evaluating LLMs!

Writing a rebuttal is 30% technical and 70% reviewers' psychology.

did people get greedy and sloppy and ruin it like with almost everything ever? you tell me

You have a budget to human-evaluate 100 inputs to your models, but your dataset is 10,000 inputs. Do not just pick 100 randomly!🙅 We can do better. "How to Select Datapoints for Efficient Human Evaluation of NLG Models?" shows how.🕵️ (random is still a devilishly good baseline)

Motivated by recent discussion with my group: Ignore subjective statements such as "I find LLMs to be incredibly useful for XX", especially when made by people (such as AI companies or gurus) who have strong biases/incentives/COI .