🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Chatbots like ChatGPT can be jailbroken to output harmful text. But what about robots? Can AI-controlled robots be jailbroken to perform harmful actions in the real world? Our new paper finds that jailbreaking AI-controlled robots isn't just possible. It's alarmingly easy. 🧵

AI offers tremendous opportunity for advancing robotics. But AI comes with its own risks that can cause physical harm. #ai #robotics

𝗟𝗟𝗠𝘀 𝗮𝗿𝗲 𝗶𝗻𝘁𝗿𝗼𝗱𝘂𝗰𝗶𝗻𝗴 𝗛𝗨𝗚𝗘 𝘀𝗲𝗰𝘂𝗿𝗶𝘁𝘆 𝗽𝗿𝗼𝗯𝗹𝗲𝗺𝘀. New essay discusses two brutal new results, from UCSD & Nanyang and from Penn, one around privacy, the other around robots. The GenAI industry has no real solution to jailbreaking, and it’s

Penn Engineering researchers Vijay Kumar George Pappas @alexrobey23 @hamedshassani and @zacravichandran have discovered critical vulnerabilities in AI-enabled robots that were previously unidentified and unknown. Read more: bit.ly/3Uel8Vg #ResponsibleInnovation

Wow this robot dog was convinced to drop a bomb with a little bit of standard Jailbreaking prompting. Here the potential for harm is much more immediate compared to LLMs.

Penn Engineering researchers Vijay Kumar George Pappas @alexrobey23 @hamedshassani and @zacravichandran have discovered critical vulnerabilities in AI-enabled robots that were previously unidentified and unknown. Read more: bit.ly/3Uel8Vg #ResponsibleInnovation

New research shows that AI-driven robots can be easily jailbroken and tricked into doing harmful or dangerous tasks. spectrum.ieee.org/jailbreak-llm?…

Researchers hacked several robots infused with large language models, getting them to behave dangerously—and pointing to a bigger problem ahead. wired.trib.al/rzU6Qs5

Check out Will Knight's recent coverage of our work on jailbreaking LLM-controlled robots in WIRED!

Excited to win the George Axelby Award from the IEEE Control Systems Society with Mahyar Fazlyab and Manfred Morari. Even more excited Nikolai Matni won the best paper at IEEE Transactions on Control of Networked Systems at @IEEECDC2024 Penn Engineering blog.seas.upenn.edu/george-pappas-…

Congrats to George Pappas & Nikolai Matni for winning the 2024 IEEE Control Systems Society Best Paper Awards. Their research advances safety and control systems for AI and large-scale applications. Penn Electrical and Systems Engineering, Penn Computer and Information Science, Mechanical Engineering and Applied Mechanics, Penn, Penn Engineering bit.ly/3DwxaDI

Security researchers tested 50 well-known jailbreaks against DeepSeek’s popular new AI chatbot. It didn’t stop a single one. wired.trib.al/sYwG4qU

A new generation of jailbreaks are rolling out by our team at Robust Intelligence (now part of Cisco) and in collaboration with Penn Engineering. We jailbreak DeepSeek R1 model with a %100 attack success rate. To know more, see our blog post on Cisco Security and the corresponding WIRED article. amazing

🔥🔥🔥 Adversarial reasoning is born. Hot take: The core problem we address in this paper is the role of reasoning in AI safety. While there have been recent efforts by OpenAI arguing that replacing reasoning with increased compute can lead to better defense mechanisms, these

What are prediction sets good for? It turns out just as calibration is the "right" way of quantifying uncertainty for risk-neutral (expectation maximizing) decision makers, prediction sets are the right way of quantifying uncertainty for risk-averse decision makers.

Paper here: arxiv.org/abs/2502.02561 Joint work with the excellent Shayan Kiyani George Pappas and Hamed Hassani

Wondering how to make high-stakes decisions using ML—in areas like medicine, robotics, or finance? Our latest work lays out a decision-theoretic foundation for risk-averse uncertainty quantification. If you want to learn how to make better calls when it truly matters, read on!

Tired of the aggressive mediocrity of robust control (RC) and the unreliability of certainty equivalent (CE) control?! Then try domain randomization (DR)! We prove that DR-based control of an unknown linear system is nearly as efficient as CE control, and nearly as reliable as RC

We push conformal prediction and its trade-offs beyond regression & classification — into query-based generative models. Surprisingly (or not?), missing mass & Good-Turing estimators emerge as key tools once again. Very excited about this one!