[LG] How to guess a gradient U Singhal, B Cheung, K Chandra, J Ragan-Kelley, J B. Tenenbaum, T A. Poggio, S X. Yu [UC Berkeley & MIT] (2023) arxiv.org/abs/2312.04709 - Backpropagation has limitations in memory usage and hardware efficiency when training large models. This

*Startup founder has entered the chat.*

Distributed training using parallel LoRAs, infrequently synced. My fav part is the analogy to git, where lots of coders can work together on a project, coordinated by simple operators like pull, commit, merge. Potential implications toward community training of big models.

Interesting set of evaluations for Claude 3 in the model card: --Autonomous Replication and Adaption --Biological Evaluations --Cyber Evaluations www-cdn.anthropic.com/de8ba9b01c9ab7…

Possibly my favorite reaction to our work. Utkarsh Singhal Kartik Chandra (also on Mastodon and Bsky)

New paper: The Platonic Representation Hypothesis In which we posit that _different_ foundation models are converging to the _same_ representation of reality. paper: arxiv.org/abs/2405.07987 website: phillipi.github.io/prh/ code: github.com/minyoungg/plat… 1/8

Apple's market cap increases by 2 OpenAI's after they announce that they're using OpenAI 😅

New paper💡! Certain networks can't perform certain tasks due to lacking the right prior 😢. Can we make these untrainable networks trainable 🤔? We can, by introducing the prior through representational alignment with a trainable network! This approach is called guidance. (1/8)

Combination of chain-of-thought reasoning and overfitting is weirder than I could have imagined.

Expectation vs reality:

Learn about aligning minds and machines. Call for papers is live now—join us at ICLR 2025 #ICLR2025. representational-alignment.github.io/2025/

Simpler times...

There's no such thing as bad data, only bad learning. Cool result!

🔬✨ Experience the First Signs of Vision at MIT Museum After Dark! Step into evolution & relive the 540-million-year journey from light-sensitive cells to modern sight. eyes.mit.edu/exhibitions/mi… 📅 April 10, 2025 | ⏰ 6-9 PM | 📍 MIT Museum | Tickets

🔄✨ Come join us at the Second edition of the Re-Align workshop ICLR 2025! 🚀🧠 The workshop explores the fascinating question of how artificial and biological systems align in their representations of the world. #ReAlign #ICLR2025

ICLR 2026 Here's a map to the venue.