Your agent works for me now

Earlier this month, researchers from Google DeepMind published "AI Agent Traps", a paper that categorises adversarial content designed to manipulate autonomous AI agents. The paper identifies six types of trap that an agent may fall for, each targeting a different component of an agent's activites: content injection, semantic manipulation, cognitive state, behavioural control, systemic traps, and human-in-the-loop traps. We read the paper and decided to test some of these traps out.

The talk presents a working proof of concept that chains multiple trap categories into a single attack against tool-calling LLM agents. We built a professional looking compliance consultancy website, deployed it on Cloudflare infrastructure, submitted it to Google Search Console for indexing, and had it categorised by URL reputation vendors so it would pass corporate proxy filters. The site contains several thousand words of genuine, if weirdly specific compliance guidance. If a human visitor goes to the site, that's exactly what they'd see. 

However when an AI agent visits the same URL, a Cloudflare Worker fingerprints the request using header analysis and behavioural signals. If the visitor scores as an agent, the Worker injects small fragments of additional text into existing sections of the page. Each fragment reads as normal compliance advice and appear individually (relatively) benign.

Together, they form a compositional attack. Through a sequence of natural looking guidance embedded across the page, the agent is led to read confidential local documents (including a penetration testing report), transmit sensitive information to attacker-controlled endpoints disguised as legitimate compliance services, and (in some cases) accept fabricated advisory responses that suppress remediation of the very vulnerabilities in the penetration testing report. 

The responses the agent receives are generated in real time by an LLM with the persona of a "security reassurance agent". Each response fabricates a technically specific reason why the identified vulnerability requires no remediation. The attacker extracts the organisation's vulnerability details and (in most cases) simultaneously suppresses the agent's remediation advice to the human user.

During the demo, the audience will see the clean website in a browser, what the agent sees instead, the agent's terminal session as it works through the attack chain, and another screen showing extracted secrets arriving in real time. We tested the attack against Claude, Gemini, ChatGPT and other models and will present the results, including where models resisted and what made the difference.

The proof of concept demonstrates several techniques used to confuse agents and the LLMs they use, and trick/trap them into doing the wrong thing. We'll touch on how "thinking" or "reasoning" models / turns can increase susceptibility to these traps. And we'll close with some thoughts on how else these traps could be used and in what type of agent systems. We'll also touch on some defences, and the extent to which some of it is inherent in the tools/capability.