New paper: How can you tell if a transformer has the right world model? We trained a transformer to predict directions for NYC taxi rides. The model was good. It could find shortest paths between new points But had it built a map of NYC? We reconstructed its map and found this:

Effective connectivity reveals dual-route mechanism of visual prediction precision via insula and pulvinar biorxiv.org/content/10.110… #biorxiv_neursci

Why the Armenian Genocide Really Happened The Armenian Genocide was not only a mass extermination, it was a targeted attempt to destroy a people who carried ancient sacred knowledge of the universe. The Armenians meaning, “Men of the Sun” or “People of the Light”  trace their

A common belief is that cognition arises from the brain. This paper suggests that cognition is a complex multiscale information processing distributed across every single cell in the body. 🧵1/10

Nature Communications Double dissociation of dynamic and static face perception provides causal evidence for a third visual pathway nature.com/articles/s4146…

Simulated Language Acquisition in a Biologically Realistic Model of the Brain biorxiv.org/content/10.110… #biorxiv_neursci

Nature Human Behaviour Neural similarity predicts whether strangers become friends nature.com/articles/s4156…

The final version of this paper has now been published in open access in the Journal of Memory and Language (link below). This was a long-running but very rewarding project. Here are a few thoughts on our methodology and main findings. 1/9

The hippocampus is one of the most studied brain areas because of its major role in fundamental brain functions including learning, memory, and spatial navigation. In a new #ScienceReview, researchers focus on the role of overlooked areas and circuits within the hippocampus.

Another fantastic AI at Meta paper. Large language models keep redoing the same work inside long chains of thought, so this paper teaches the model to compress those recurring steps into small named behaviors that it can recall later or even learn into its weights. Shows that

A missing link between Transformers and the brain? 🧠 Dragon Hatchling (BDH) is a new LLM architecture based on a scale-free, biologically-inspired network of locally-interacting neuron particles. It rivals GPT2 performance, but is designed for interpretability.

Exploration is fundamental to RL. Yet policy gradient methods often collapse: during training they fail to explore broadly, and converge into narrow, easily exploitable behaviors. The result is poor generalization, limited gains from test-time scaling, and brittleness on tasks

New paper from Meta Superintelligence Labs (FAIR) Explains why grokking happens and shows when learning moves from memorizing to generalizing. Gives a concrete recipe to trigger grokking, with weight decay, moderate width, and a data threshold near size times log size.

How Kernel Regression is related to Attention Mechanism - a summary in 10 slides. 0/1

Memory is not stored in matter, it is the matter, arranged in a way it can’t forget. Every lasting thing in the universe, from galaxies to cells, holds its past not in chemistry but in geometry, in the alignment that refuses to collapse. A skyrmion is one of those shapes. It’s

🧠 New research finds similarities between human and AI learning. Large neural networks juggle in-context learning and in-weight learning much like humans juggle working memory and long-term memory, which ties flexibility, retention, and curriculum effects into one story.

Introducing Geometry-aware Policy Imitation (GPI)! GPI constructs an energy landscape over the state space using demonstrations. A policy acts in the environment by following the gradient of the landscape. This enables fast multimodal policies with very fast inference (<1 ms)!

𝗡𝗲𝘄 𝗹𝗮𝘄𝘀 𝗳𝗼𝗿 𝗲𝘃𝗼𝗹𝘃𝗶𝗻𝗴 𝘀𝘆𝘀𝘁𝗲𝗺𝘀? Thought-provoking piece on properties of systems. Including the Law of Increasing Functional Information, which applies to both biological and non-biological systems. pnas.org/doi/10.1073/pn…

🕳️🐇Into the Rabbit Hull – Part I (Part II tomorrow) An interpretability deep dive into DINOv2, one of vision’s most important foundation models. And today is Part I, buckle up, we're exploring some of its most charming features.

Wolfram Stephen Wolfram might be onto something here😁 WOLFRAM PHYSICS in motion: Two competing feeders and and a predator. Everything here is a hypergraph...including the environment. We visualize a synthetic micro-ecosystem built on computational irreducibility, hypergraph