🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Every CV guy I know has privately admitted at some point that current video datasets do not really seem to care about time. That the video tasks are "too short" & don't test much time understanding We introduce EgoSchema -- A litmus test for truly long-form video understanding

How far can we go with vision alone? Excited to reveal our Large Vision Model! Trained with 420B tokens, effective scalability, and enabling new avenues in vision tasks! (1/N) Kudos to Xinyang (Young) Geng Karttikeya Mangalam Amir Bar, Alan Yuille Trevor Darrell Jitendra MALIK Alyosha Efros!

Happy to present LVM (Large Vision Model). Scalable and tasks can be specified via prompts. Enjoy!

Together with the Ego4D consortium, today we're releasing Ego-Exo4D, the largest ever public dataset of its kind to support research on video learning & multimodal perception — including 1,400+ hours of videos of skilled human activities. Download ➡️ bit.ly/3teP49w

hello san francisco

Want to make your photorealistic 3D avatar dance like your favorite actor? Check this out!

We cast real-world humanoid control as a next token prediction problem, akin to predicting the next word in language. Check out our robot walking in San Francisco (Ilija Radosavovic et al) …anoid-next-token-prediction.github.io

Another success of sim-to-real for training robot policies! This task, using two multi-fingered hands, requires considerable dexterity, and is hopefully representative of other household tasks that we wish to solve in the future.

Imitation learning works™ – but you need good data 🥹 How to get high-quality visuotactile demos from a bimanual robot with multifingered hands, and learn smooth policies? Check our new work “Learning Visuotactile Skills with Two Multifingered Hands”! 🙌 toruowo.github.io/hato/

It was great to work with Karttikeya Mangalam, Andrea Bajcsy and Jitendra MALIK! Project: neerja.me/atp_latent_cor… Arxiv: arxiv.org/abs/2312.06653

10 years after DQN, what are deep RL’s impacts on robotics? Which robotic problems have seen the most thrilling real-world successes thanks to DRL? Where do we still need to push the boundaries, and how? Our latest survey explores these questions! Read on for more details. 👇

Fun collaboration w/ Antonio Loquercio, Carlo Sferrazza, Haozhi Qi, @jane_h_wu, Pieter Abbeel, Jitendra MALIK Checkout our paper at arxiv.org/pdf/2409.08273. We release code at github.com/hgaurav2k/hop.

Please see the website for more details. synNsync🪩is joint work with my awesome ✨co-authors✨: Lea Müller Jathushan Rajasegaran Georgios Pavlakos Shiry Ginosar Angjoo Kanazawa Jitendra MALIK Website🖥️: von31.github.io/synNsync/ Data💾: github.com/Von31/swing_da… Arxiv📜: arxiv.org/abs/2409.04440 🧵6/6

Autoregressive modeling is not just for language, it can equally be used to model human behavior. This paper shows how..

Touche', Sergey!

Happy to share these exciting new results on video synthesis of humans in movement. Arguably, these establish the power of having explicit 3D representations. Popular video generation models like Sora don't do that, making it hard for the resulting video to be 4D consistent.

I'm happy to post course materials for my class at UC Berkeley "Robots that Learn", taught with the outstanding assistance of Toru. Lecture videos at youtube.com/playlist?list=… Lecture notes & other course materials at robots-that-learn.github.io

Enjoy watching a humanoid walking around UC Berkeley. It only looks inebriated :-)

Angjoo Kanazawa Angjoo Kanazawa and I taught CS 280, graduate computer vision, this semester at UC Berkeley. We found a combination of classical and modern CV material that worked well, and are happy to share our lecture material from the class. cs280-berkeley.github.io Enjoy!