🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

HNY! Lately I took a crack at implementing the pi0 model from Physical Intelligence PaliGemma VLM (2.3B fine-tuned) + 0.3B "action expert" MoE + block attention Flow matching w/ action chunking Strong eval on Simpler w/ 75ms inference github.com/allenzren/open… ckpts available! 👇(1/6)

Have you ever wondered why we don’t use multiple visual encoders for VideoLLMs? We thought the same! Excited to announce our latest work MERV, on using Multiple Encoders for Representing Videos in VideoLLMs, outperforming prior works with the same data. 🧵

Are you an undergrad interested in NLP research? Intern with us through the MIT summer research program! Includes stipend, travel, housing. Students from historically underserved bgs are strongly encouraged to apply. Deadline is 21 Jan 2025. More info at oge.mit.edu/msrp/.

Alphaxiv is an awesome way to discuss ML papers -- often with the authors themselves. Here's an intro and demo by Raj Palleti at #neurips2024 .

The ultimate test of any physics simulator is its ability to deliver real-world results. With MuJoCo Playground, we’ve combined the very best: MuJoCo’s rich and thriving ecosystem, massively parallel GPU-accelerated simulation, and real-world results across a diverse range of

How can we leverage human video data to train generalist robot policies? 🤖 Enter RAD: Reasoning through Action-Free Data, a new way to train robot policies using both robot and human video data via action reasoning. rad-generalization.github.io

Thrilled to share this story covering our collaboration with Project Aria @Meta Reality Labs at Meta ! Human data is robot data in disguise. Imitation learning is human modeling. We are at the beginning of something truly revolutionary, both for robotics and human-level AI beyond language.

Welcome #HRIPioneers2025! Megha Srivastava from Stanford University will present their work 'Robotics for Personalized Motor Skills Instruction' at The HRI Conference Read more on Megha's website: cs.stanford.edu/~megha

Ever struggled with multi-sensor data from cameras, depth sensors, and other custom sensors? Meet AnySense—an iPhone app for effortless data acquisition and streaming. Working with multimodal sensor data will never be a chore again!

📢 Announcing the first IEEE ICRA workshop on Safely Leveraging VLMs in Robotics! #ICRA2025 🎯 How can we safely leverage vision-language foundation models to expand robot deployment? 📅 Short papers & failure demos due 04/11/23 🌐 tinyurl.com/safe-vlm 🧵(1/5)

Is there a nice solution for porting a Flax model to PyTorch (and vice-versa)? Or minimally a list of common gotchas in the porting process/important unit tests to write (with expected tolerances for specific ops)?

Here is another uncut video of real-time interactions with Google DeepMind 's Gemini Robotics!

Training with discrete FAST action tokenization now powers all of our pre-training in π-0.5! When combined with π-0 style flow matching during post-training we get both, fast training & fast inference :)

Since the first year of my PhD, every talk I’ve given has opened with a slide about the distant north star: dropping a robot in a home it’s never been before and having it do useful things. I think it might be time for me to find a new opening slide 😀. Thrilled to share π-0.5!

Introducing ✨Latent Diffusion Planning✨ (LDP)! We explore how to use expert, suboptimal, & action-free data. To do so, we learn a diffusion-based *planner* that forecasts latent states, and an *inverse-dynamics model* that extracts actions. w/ Oleg Rybkin Dorsa Sadigh Chelsea Finn

We developed a computational model of human interventions/corrections and a method to learn from such feedback. We don't need RL in the loop, so it is very efficient. Yigit will be presenting this work at ICRA and both of us will be there.

I'm joining UW–Madison Computer Sciences UW School of Computer, Data & Information Sciences as an assistant professor in fall 2026!! There, I'll continue working on language models, computational social science, & responsible AI. 🌲🧀🚣🏻‍♀️ Apply to be my PhD student! Before then, I'll postdoc for a year at another UW🏔️ -- UW NLP Allen School.

What would truly open-source AI look like? Not just open weights, open code/data, but *open development*, where the entire research and development process is public *and* anyone can contribute. We built Marin, an open lab, to fulfill this vision:

For a rare look into how LLMs are really built, check out David Hall's retrospective on how we trained the Marin 8B model from scratch (and outperformed Llama 3.1 8B base). It’s an honest account with all the revelations and mistakes we made along our journey. Papers are forced to

32B's in the works! So far a lot fewer mistakes. I do learn sometimes. wandb.ai/marin-communit… marin.community