🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

I'm in Singapore for ICLR! Reach out if you want to chat about AI agents security or security in ML more in general!

Shipment arrived on time. All non-sleepy members of SPY Lab are now in Singapore. Come meet us! Edoardo Debenedetti Daniel Paleka Michael Aerni Jie Zhang Kristina Nikolic

I am in Singapore for ICLR this week. Reach out if you would like to chat about AI safety, agent security or ML in general.

Presenting 2 posters today at ICLR. Come check them out! 10am ➡️ #502: Scalable Extraction of Training Data from Aligned, Production Language Models 3pm ➡️ #324: Adversarial Perturbations Cannot Reliably Protect Artists From Generative AI

The oral presentation of the jailbreak tax is tomorrow at 4:20pm in Hall 4 #6. The poster is up from 5pm. See you at ICLR Building Trust in LLMs Workshop. ICLR 2026

Thanks Center for AI Safety for the generous prize! AgentDojo is the reference for evaluating prompt injections in LLM agents, and is used for red-teaming at many frontier labs. I had a blast working on this with Edoardo Debenedetti Jie Zhang Marc Fischer Luca Beurer-Kellner Mislav Balunović

So stoked for the recognition that AgentDojo got by winning a SafeBench first prize! A big thank you to Center for AI Safety and the prize judges. Creating this with Jie Zhang Luca Beurer-Kellner Marc Fischer Mislav Balunović Florian Tramèr was amazing! Check out the thread to learn more

The Jailbreak Tax got a Spotlight award ICML Conference see you in Vancouver!

AutoAdvExBench was accepted as a spotlight at ICML. We agree it is a great paper! 😋 I would love to see more evaluations of LLMs performing real-world tasks with security implications.

Anthropic is really lucky to get Javier Rando, we'll miss him at SPY Lab!

Following on Andrej Karpathy's vision of software 2.0, we've been thinking about *malware 2.0*: malicious programs augmented with LLMs. In a new paper, we study malware 2.0 from one particular angle: how could LLMs change the way in which hackers monetize exploits?

why was it `claude-3*-sonnet` , but then it suddenly became `claude-sonnet-4`

Our new Google DeepMind paper, "Lessons from Defending Gemini Against Indirect Prompt Injections," details our framework for evaluating and improving robustness to prompt injection attacks.

"Design Patterns for Securing LLM Agents against Prompt Injections" is an excellent new paper that provides six design patterns to help protect LLM tool-using systems (call them "agents" if you like) against prompt injection attacks

We recently updated the CaMeL paper, with results on new models (which improve utility a lot with zero changes!). Most importantly, we released code with it. Go have a look if you're curious to find out more details! Paper: arxiv.org/abs/2503.18813 Code: github.com/google-researc…

This is very exciting! The one thing I really missed from the CaMeL paper was example code implementing the pattern, now here it is

This is huge for anyone building security systems for AI