Excited to share our #CVPR2024 work (oral & award candidate) on integrating map uncertainty into trajectory prediction led by Xunjiang Gu. Our key insight is simple: Scene uncertainties matter for agent trajectories. 🧵1/n

DistillNeRF: Perceiving 3D Scenes from Single-Glance Images by Distilling Neural Fields and Foundation Model Features Letian Wang, Seung Wook Kim, Jiawei Yang, Cunjun Yu, Boris Ivanovic Steven L. Waslander, Yue Wang, Sanja Fidler, Marco Pavone, Peter Karkus

Excited to share our ECCV 2024 work: Accelerating Online Mapping and Behavior Prediction via Direct BEV Feature Attention 🚗🚗🚗 Online map estimation shows great potential in guiding downstream tasks, as seen in our recent best paper finalist at CVPR 2024. But can we speed this

🚀 Introducing 𝐖𝐨𝐥𝐟 🐺: a mixture-of-experts video captioning framework that outperforms GPT-4V and Gemini-Pro-1.5 in general scenes 🖼️, autonomous driving 🚗, and robotics videos 🤖. 👑: wolfv0.github.io/leaderboard.ht…

🚀 Just dropped the code: #InstantSplat! Reconstruct your 3D scenes in seconds. Get it now at github.com/NVlabs/Instant… and start building! ✨ #3DVision #AI #ComputerVision #NVlabs #OpenSource

New NVIDIA #DRIVELabs on advancements that use #generativeAI (such as text-to-simulation) to create realistic environments, generate natural driving behaviors, and edit resulting scenarios to enable rigorous #AV evaluation and training. nvda.ws/3Xak6LQ

This week we will be presenting 3 papers European Conference on Computer Vision #ECCV2026, on online mapping (Boris Ivanovic), traffic scenario generation, and VLM-based AV stacks (Chaowei Xiao): Paper 1: arxiv.org/pdf/2407.06683 Paper 2: arxiv.org/abs/2312.13303 Paper 3: vlm-driver.github.io More on online mapping 👇

The Autonomous Vehicle (AV) Research group NVIDIA is now hiring. From AV foundation models to AI safety and generative simulation, we are pushing the state of the art in AV and embodied AI! To apply: Junior RS: nvidia.wd5.myworkdayjobs.com/en-US/NVIDIAEx… Senior RS: nvidia.wd5.myworkdayjobs.com/en-US/NVIDIAEx…

We also have several open positions for interns: nvidia.wd5.myworkdayjobs.com/NVIDIAExternal… To learn more about our work, check out our website: research.nvidia.com/labs/avg/

Despite of what the posting says, these can be international (e.g. Europe)! Same for intern positions!

❓ How can we significantly boost local-optimizers performance under strict runtime constraints in robotics and autonomous driving? 🌟 I'm excited to share our latest research: Learning Multiple Initial Solutions to Optimization Problems (𝐌𝐈𝐒𝐎) (1/13)

Introducing DreamDrive, which combines the complementary strengths of generative AI (video diffusion) and neural reconstruction (Gaussian splatting) to transform any street-view image into a dynamic 4D driving scene! Web: pointscoder.github.io/DreamDrive/ Paper: arxiv.org/abs/2501.00601

Complementing DreamDrive, I am thrilled to introduce STORM, which enables fast scene reconstruction with a single feed-forward model. STORM transforms camera logs into dynamic 3D models - in real time! Web: jiawei-yang.github.io/STORM/ Paper: arxiv.org/abs/2501.00602

For the first time ever, NVIDIA is hosting an AV Safety Day at GTC - a multi-session workshop on AV safety. We will share our latest work on safe AV platforms, run-time monitoring, safety data flywheels, and more! #AutonomousVehicles #AI at #GTC25 ➡️ nvda.ws/3Xc3xPo

Don’t miss this deep dive into the future of autonomous vehicles! Excited to present about how foundation models are transforming AV technology with Jose M. Alvarez at #GTC25! Check out all the session details below 👇

At #GTC2025, Jensen unveiled Halos, a comprehensive safety system for AVs and Physical AI. Halos integrates numerous technologies developed by my team NVIDIA, and I was thrilled to help coordinate its launch alongside Riccardo Mariani and many amazing colleagues NVIDIA DRIVE.

Announcing the 2025 NAVSIM Challenge! What's new? We're testing not only on real recordings—but also imaginary futures generated from the real ones! Two rounds: #CVPR2025 and #ICCV2025. Submit in May for Round 1! opendrivelab.com/challenge2025 🧵(1/3)

Our #CVPR2025 paper on training traffic models in closed loop is an Oral at CVPR!! The work was done by Zhejun Zhang (zhejz.github.io) in collaboration with Peter Karkus, me, Wenhao Ding, Yuxiao Chen, Boris Ivanovic and Marco Pavone. Page: zhejz.github.io/catk/ 🧵...

🛣️ New NVIDIA DRIVE Labs video on the future of mapless driving! High-definition (HD) maps have been essential for autonomous driving, but their cost and maintenance challenges limit deployment. 🚗 In this episode of #DRIVELabs, we explore how NVIDIA is reducing reliance on HD

📢 The first X-Sense Workshop: Ego-Exo Sensing for Smart Mobility at #ICCV2025!  🎤 We’re honored to host an outstanding speaker lineup, featuring Manmohan Chandraker, Bharath Hariharan, Cathy Wu (Hiring PhD/postdocs), Holger Caesar, Bolei Zhou, Boyi Li, Katie Luo x-sense-ego-exo.github.io