🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Joshua Kazdan Sanmi Koyejo Azalia Mirhoseini John Hughes Jordan Juravsky Sara Price Aengus Lynch Bradley Brown Ryan Ehrlich Ronnie Clark Quoc Le hazyresearch And thank you to our funders and hosting institutions Stanford Trustworthy AI Research (STAIR) Lab Stanford AI Lab & many more!

Joshua Kazdan Sanmi Koyejo Azalia Mirhoseini John Hughes Jordan Juravsky Sara Price Aengus Lynch Bradley Brown Ryan Ehrlich Ronnie Clark Quoc Le hazyresearch Stanford Trustworthy AI Research (STAIR) Lab Stanford AI Lab I also embarrassingly forgot to give credit to Erik Jones in my first tweet! He was also an integral part of the effort, helping with the scaling analyses

Today, we're releasing a new paper – One-Minute Video Generation with Test-Time Training. We add TTT layers to a pre-trained Transformer and fine-tune it to generate one-minute Tom and Jerry cartoons with strong temporal consistency. Every video below is produced directly by

Test-time training (TTT) layers are RNN layers where the hidden state is a machine learning model and the update rule is a step of gradient descent. See this thread for previous work. x.com/karansdalal/st…

Our approach simply adds TTT layers to a pre-trained Diffusion Transformer and fine-tunes it on long videos with text annotations. To keep costs manageable, we limit self-attention to local segments and let TTT (linear complexity) operate globally.

We create an “On-Chip Tensor Parallel” algorithm to implement an efficient TTT-MLP kernel. Specifically, we shard the weights of the “hidden state model” across Streaming Multiprocessors, and use the DSMEM feature Hopper GPUs implement AllReduce among SMs. This avoids costly

Grateful for wonderful collaborators. This work will be presented at CVPR 2025. Daniel Koceja Gashon Hussein Jiarui Xu Yue Zhao Jan Kautz Carlos Guestrin Tatsunori Hashimoto Sanmi Koyejo Yejin Choi Xiaolong Wang

Daniel Koceja Gashon Hussein Jiarui Xu Yue Zhao Jan Kautz Carlos Guestrin Tatsunori Hashimoto Sanmi Koyejo Yejin Choi Xiaolong Wang + our wonderful collaborators without Twitter – Shihao Han, Ka Chun Cheung, Youjin Song, and Yu Sun.

📢 Over the moon that Open Problems in Technical AI Governance has now been published at TMLR! See the updated version here: shorturl.at/joQJS

We're also organising a workshop on technical AI governance at #ICML2025! This is a great opportunity to present work on any of the problems outlined in the paper. Submissions are due May 7th and we're also looking for PC members. More info 👉🏻Technical AI Governance @ ICML 2025

Huge thanks again to Anka Reuel | @ankareuel.bsky.social for co-leading this work, and to Lisa Soder for doing incredible work making the workshop happen. And of course to all our wonderful co-authors: Cas (Stephen Casper), Tim Fist, Onni Aarne, Lewis Hammond, Lujain Ibrahim لجين إبراهيم, Alan Chan, Peter Wills,

Markus Anderljung, Ben Garfinkel, Lennart Heim, ⿻ Andrew Trask, Gabriel Mukobi, Rylan Schaeffer, Mauricio Baker, Sara Hooker, Irene Solaiman, Sasha Luccioni, PhD 🦋🌎✨🤗, Nitarshan Rajkumar, Nick Moës, Jeffrey Ladish, David Bau, Paul Bricman, Neel Guha, Jessica Newman, Yoshua Bengio, Tobin South, Alex Pentland,

Sanmi Koyejo, Mykel Kochenderfer, Robert Trager, Oxford Martin AI Governance Initiative, SISL, Stanford Trustworthy AI Research (STAIR) Lab

📢 New white paper: Scholars from Stanford HAI, The Asia Foundation, and University of Pretoria map the current landscape of technical approaches to developing LLMs that better perform for and represent low-resource languages. (1/4) ↘️ hai.stanford.edu/policy/mind-th…

LLM development suffers from a digital divide: Most major LLMs underperform for low-resource languages; are not attuned to relevant cultural contexts; and are not accessible in parts of the Global South. (2/4)

The paper discusses the trade-offs of three approaches (massively multilingual models, regional multilingual models, and monolingual or monocultural models) and highlights ongoing initiatives to address underlying data scarcity and quality issues. (3/4)

Here, the authors present three high-level recommendations for AI researchers, funders, policymakers, and civil society organizations: (4/4) hai.stanford.edu/policy/mind-th…

Excited to present this work at ICLR's SynthData Workshop on Sunday April 27! Come through from 11:30-12:30 @ Peridot 202-203 to talk anything synthetic data for post-training, benchmarking, and AI for healthcare in general.

will present this work as spotlight at icml this year :)

Most major LLMs are trained using English data, making it ineffective for the approximately 5 billion people who don't speak English. Here, Stanford HAI Faculty Affiliate Sanmi Koyejo discusses the risks of this digital divide and how to close it. stanford.io/3SfGmRk