Congrats to John Hopfield and Geoffrey Hinton! well deserved recognition that some important foundations of AI rest on physics! Physics departments take note: understanding and improving AI systems is a new frontier topic for physics, just as biophysics was earlier. Time to hire!

Given today’s great news from the #NobelPrize2024, I want to share a couple of personal thoughts on Hopfield Networks. This idea had an enormous impact on at least three large disciplines: Statistical Physics, Computer Science and AI, and Neuroscience.

John Hopfield has a nice article in the annual reviews of condensed matter physics. It starts off with a discussion of what physics is, which I think is totally on point.

What an exciting week it's been ...! Huge congrats for work done at DeepMind.

A wonderfully organized school in an inspiring and didactic setting! Lecture notes from the Les Houches summer school are now published. Boris Hanin & I lectured on the theory of neural networks at large width.

Very excited to share our interview with Jascha Sohl-Dickstein on the history of diffusion models — from his original 2015 paper inventing them, to the GAN "ice age", to the resurgence in diffusion starting with DDPM. Enjoy!

So nice to be surrounded by >12000 physicists at American Physical Society summit working hard to understand the nature of our universe and build better tech like solar cells and novel quantum materials. Important for the public to remember that our living standards depend on this research.

I was in the audience. It was a great talk. It is great to see word2vec get formally analyzed!

For humans, mathematical symbols (and formal systems like lean) are *tools* we learn how to use, not a structure that wraps around us. I think that's the right role for formal still manipulation: a tool that can be employed by an intelligent system if/when it supports a goal.

In neuroscience, we often try to understand systems by analyzing their representations — using tools like regression or RSA. But are these analyses biased towards discovering a subset of what a system represents? If you're interested, check out our new commentary! Thread:

Very excited to lead this new Simons Foundation collaboration on the physics of learning and neural computation to develop powerful tools from physics, math, CS, stats, neuro and more to elucidate the scientific principles underlying AI. See our website for more: physicsoflearning.org

Very excited to be part of this Simons collaboration on the Physics of Learning & Neural Computation!

I am excited to announce what William Fedus and I have been working on: Periodic Labs, a world class team of experimentalists, theorists, and LLM experts. Scientific discovery is inherently an out-of-domain task. Experimental iteration is required for significant advances,