all theorems are correct, but some are useful

Except Rust. Rust is a goddamn miracle x.com/TaliaRinger/st…

Q: Does my LM leak probability onto infinite strings? A: For RNNs and PFSAs you need to test, but Transformers always generate EOS in finite time (prob=1). 🤔First we need to formalize the question… cs.jhu.edu/~jason/papers/… #ACL2023 poster Tue 11am w/Leo Du @ryandcotterell et al

We summarized the #acl2023nlp Toronto conference for you with some poster recordings and author interviews! 👇 🎬 youtu.be/-Agcr0nawuk Featuring Szymon Tworkowski Jasivan Sivakumar Kundan Krishna (at ACL 2O25) Emanuele Bugliarello Leo Du Florian Mai Franz Nowak Paul Drm Moritz Plenz and Jay Alammar 👏

Following up a weekend effort by another weekend effort: llama2. rs 🦀 github.com/leo-du/llama2.… In a single Rust file w/ * zero dependencies (i.e. custom rng w/ PCG) * zero lines of `unsafe` code (very 🦀!) * support user prompts * (almost) same performance

One of the fundamental problems with probability notation in machine learning is due to the fact that few people really have a firm grasp on conditioning from a measure theoretical perspective. Another issue: random variables versus indexed collections of probability spaces.

Congratulations to Rylan Schaeffer, Brando Miranda, Sanmi Koyejo for winning a best paper award at NeurIPS for this insightful paper. Are Emergent Abilities of Large Language Models a Mirage? arxiv.org/abs/2304.15004

If you are interested in knowing how you can do energy-based sampling from language models, make sure to check our #NeurIPS23 paper titled “Structured Voronoi Sampling”...🧵 arxiv.org/pdf/2306.03061…

"Are Emergent Abilities of Large Language Models a Mirage?" is a NeurIPS outstanding paper!🙌🏿 Congrats especially to the students Rylan Schaeffer Brando Miranda & other awardees. If you want to learn more, check out the oral & poster 👇🏿this afternoon (Dec 14) 1/2

A case for chasing the "low hanging fruit" in science...

New paper out w/ Shriram Krishnamurthi (primary: Bluesky) accepted to OOPSLA'24: a psychometric analysis of programming language learning. We added ~200 quiz questions to a popular book on Rust and collected ~1,000,000 answers from ~60,000 people over 1 year. arxiv.org/abs/2401.01257

wrote a short note on using parallel scans for backprop: justintchiu.com/blog/pscan_dif… turns out there was already a paper on this too! arxiv.org/abs/1907.10134

You'd think that classes in the Borel hierarchy would be named in the same language...

"Let no one ignorant of geometry enter." Finally, a compute can enter Plato's academy.

So creative. arxiv.org/abs/2401.16657

I knew a lot of mathematicians were getting into formalization but I had no idea it was this much of a problem

wrote a short blog post on the latent variable model behind STaR: justintchiu.com/blog/star/

Dominic Verity, Mario Carneiro and I just announced a new project to formalize some aspects of ∞-category theory in #Lean via the notion of an ∞-cosmos.

- What do you call someone who trains LLMs on Apple Silicon in Metal instead of CUDA? - FullMetal Alchemist

Love this distinction between “compiler optimization” and “programming model”. Coming back to a common example, part of Rust’s success is its programming model. Also, great talk!