I think about this type of thing _all_ the time, how did we do all this with our hunter gatherer brains...

1. We often observe power laws between loss and compute: loss = a * flops ^ b + c 2. Models are rapidly becoming more efficient, i.e. use less compute to reach the same loss But: which innovations actually change the exponent in the power law (b) vs change only the constant (a)?

Without its international students, Harvard is not Harvard. hrvd.me/IntStudents25t

How is mathematics at Harvard, now that it is free from the antisemitic influence?

Katie’s really doing a service to the community here

There is no AI research program in the US without Chinese and Indian students. If you think otherwise, it’s because you’re not a researcher

Leo Gao jack morris This is demonstrably false. Neural nets in the kernel regime achieve generalization, overfit benignly, undergo double descent, and exhibit neural scaling laws (with e.g. chinchilla optimal compute predictions). Yes, feature learning is lacking, which means these miss transfer

You can differentiate 90% of the ML yappers on Twitter by asking them to prove the Marchenko–Pastur Law or recite Guerra’s and Talagrand’s rigorous derivations of the Parisi formula. If they flinch at either, it’s all vibes, no fundamentals.

Preach

Sensational work by the gang

Check out the #KempnerInstitute's presentations at #ICML2025 today! #AI #NeuroAI #LLMs 🧵Here's a thread of abstracts. (1/20)

Losing my mind rn

as a linguist, the speaker should know better: it’s not about “speaking more efficiently” (using mathematical terms for prosaic things isn’t more efficient!) but rather to signal ingroup membership in an emerging elite technocrat class defined by it’s relation to math and code

arxiv.org/abs/2507.18472 Another nice IKKT related paper this year. If spacetime can truly and fully emerge from random matrix models, either holographically or through some other duality, it will be a paradigm shift, also impacting how we think about quantum mechanics.

This is absolutely the best possible education a self-motivated kid can get

Coming March 17, 2026! Just got my advance copy of Emergence — a memoir about growing up in group homes and somehow ending up in neuroscience and AI. It’s personal, it’s scientific, and it’s been a wild thing to write. Grateful and excited to share it soon.

Honestly the thing that motivated me to do this episode was learning that there's less than $200M/year of smart philanthropy on factory farming - GLOBALLY. Just to explain how fucking crazy that is: 1. It's insane how cheap the interventions that will spare BILLIONS of animals