🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Come join us in our CoRL workshop on generalists agents! How can we build robot generalists using learning from video, scaling up simulators, real robot exploration, generative AI and more? #FirstTweet

Gabe's work on memory-augmented prompting of LLMs supports open-ended semantic parsing and task planning without forgetting. The model can store, retrieve and adapt instructed plans on-the-fly at deployment time!

🤖🌐How to scale up data across diverse tasks and environments for robot skill learning? Harness the power of language, vision generative models in simulation! Excited to share Gen2Sim, a step towards autonomous robotic skill acquisition in simulation. gen2sim.github.io

Check out our latest work on 3D feature field transformers for learning manipulation policies from demonstrations, with SOTA performance in RLBench.

“Find us today at 10:45 am at poster number 305 #NeurIPS”

With ODIN, 3D perception most benefits from 2D feature pre-training, and is used in the real world, outside dataset-given 3D meshes. Congrats Ayush and ODIN team!

Very excited to speak tomorrow on unifying 2D/3D models of images, language and actions at multimodalitiesfor3dscenes.github.io. On Tuesday, I will talk about generative video perception at generative-vision.github.io/workshop-CVPR-… and memory-prompted 3D parsing at 3dcompat-dataset.org/workshop/C3DV2…. See you #CVPR2025 !

For anyone excited in unifying 2D / 3D perception, come chat with us this morning!

Come chat with us today about Diffusion-ES, that combines evolutionary search with diffusion models for efficient planning! Code is also now publicly available.

We show an LLM/VLM agent can transform plans into plan abstractions by adding language comments on preconditions, state changes and subgoals, using VLM's knowledge and human feedback. In-context planning with retrieved abstractions sets a new SOTA across domains.

The Genesis physics engine is just released, incredible effort from Xian and his team. It is a most general, fast and easy to use physics engine, which we hope it will accelerate robotics research and beyond.

A thorough apples-to-apples speed comparison between IsaacGym, MujocoMJX and GENESIS in static and dynamic scenes, with and without collisions. We hope to have a paper ready within the next few months.

Yes, we can.

We train UniDisc - a diffusion model that can generate image and text jointly! Using diffusion instead of AR permits high controllability including infilling in multimodal space, not possible with any other multimodal generative model currently. Congrats to Alex and Mihir!

Self-training to maximize self-confidence improves LLM reasoning, without any extrinsic reward.

Humans shift their gaze when they think. VLMs should too. Grounding their reasoning in the image leads to more accurate answers.

Check out Adam's new work for denser, faster and more accurate 2D point tracks with AllTracker.

Beautiful demo of AllTracker

When data is limited and training spans multiple epochs, discrete diffusion beats autoregressive models for text generation. Check out Mihir and Mengning’s thorough analysis: