🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Our #CoRL2023 paper shows that by composing the energies of diffusion models trained to sample for individual constraint types such as collision-free, spatial relations, and physical stability, it can solve novel combinations of known constraints. diffusion-ccsp.github.io (🧵 1/N)

Introducing a way to convert synthesized robot videos to robot execution without using any action labels! flow-diffusion.github.io We also release a codebase (with pretrained models) for text-to-video generation. Train your own models for robot control in only 1 day with 4 GPUs!

What’s left w/ foundation models? We found that they still can't ground modular concepts across domains. We present Logic-Enhanced FMs:🤝FMs & neuro-symbolic concept learners. We learn abstractions of concepts like “left” across domains & do domain-independent reasoning w/ LLMs.

Ever heard about "Bilevel Planning" or "Task and Motion Planning", but been unsure what those words mean? Ever wanted a gentle intro to these methods so you can just understand what's going on? Our new blog post might help! lis.csail.mit.edu/bilevel-planni…

Definitely one of my top 3 favourite papers :) It marries deep learning with a minimal set of universal grammar rules for grounded language learning. It draws inspiration from lexicalist linguistics and cognitive science (bootstrapping from core knowledge).

Come chat with us on Tuesday morning at #203 at NeurIPS Conference !

Check out our new framework that automatically generates planning domain knowledge with LLMs, learns to ground it, and verifies it through interaction. We believe that learning such verifiable and compositional planning representations from language is important for embodied AI!

Can we leverage VLMs for robot manipulation in the open world? Checkout our new work MOKA, a simple and effective visual prompting method!

The key idea is to query GPT-4V to perform multiple choice from a set of keypoints and waypoints. Here is an example: suppose the current task is to sweep the trash bag off the table. We mark the involved objects with a set of points, and overlay the image with grids.

Can we use wearable devices to collect robot data without actual robots? Yes! With a pair of gloves🧤! Introducing DexCap, a portable hand motion capture system that collects 3D data (point cloud + finger motion) for training robots with dexterous hands Everything open-sourced

Introducing our @icml_conf paper: Learning Iterative Reasoning through Energy Diffusion! We formulate reasoning as optimizing a sequence of energy landscapes. This enables us to solve harder problems at test time with more complex optimization. Website: energy-based-model.github.io/ired/

What makes a maze look like a maze? Humans can reason about infinitely many instantiations of mazes—made of candy canes, sticks, icing, yarn, etc. But VLMs often struggle to make sense of such visual abstractions. We improve VLMs' ability to interpret these abstract concepts.

Excited to announce our upcoming workshop: “Planning in the Era of LLMs”! 🌐🤖 Join us as we explore the intersection of planning representations, algorithms, and the power of LLMs. Let’s discuss how combining these insights and solutions can unlock new capabilities in AI.

Excited to bring back the 2nd Workshop on Visual Concepts at #CVPR2025 2025, this time with a call for papers! We welcome submissions on the following topics. See our website for more info: sites.google.com/stanford.edu/w… Join us & a fantastic lineup of speakers in Tennessee!

Excited to organize Workshop on Learning Meets Model-Based Methods for Contact-Rich Manipulation @ ICRA 2025! We welcome submissions on a range of topics—check out our website for details: contact-rich.github.io Join us for an incredible lineup of speakers! #ICRA2025

Tutorial on "Foundation Models Meet Embodied Agents", with Yunzhu Li Jiayuan Mao Wenlong Huang . We categorize LLMs, VLMs, VLAs and their usages under a MDP formulation. Come and talk with us! Also on Zoom! …models-meet-embodied-agents.github.io Feb 25 8:30-12:30 Location: 118A room,

We will organize an #ICML2025 Workshop on Programmatic Representations for Agent Learning, bringing together experts in decision-making and code generation to explore how structured representations can make agent learning more interpretable, generalizable, efficient, and safe!

Join us for the session tomorrow. It will be held in person at NAACL and also available via VC.

Excited to see more work at the intersection of programmatic representations, program synthesis, and learning! If you’re exploring programmatic representations for agents, consider submitting to our ICML & RLC workshops!