Indirect Prompt Injection Defenses: What Actually Holds

Eight published defenses against indirect prompt injection were evaluated against adaptive attackers. All eight broke, at attack success rates above 50%. A larger study from researchers at OpenAI, Anthropic and Google DeepMind ran adaptive attacks against twelve published defenses spanning injection and jailbreak and bypassed most above 90%, where the majority had originally published near-zero rates. Its injection set covered the detector models teams deploy off the shelf, along with spotlighting and prompt sandwiching.

You probably have some of those defenses running. A fine-tuned detector on retrieved content. Delimiters wrapping untrusted text. Sandwiching that repeats the user instruction after tool output. Spotlighting on RAG documents. Every one of them is a reasonable engineering decision and every one of them was measured against an attacker who was not adapting.

The defenses lost together, for the same reason, and the reason is where they were installed.

Defenses Sort by Where They Sit Relative to the Model’s Decision

An indirect prompt injection defense occupies one of three positions.

Upstream of the decision. Everything that inspects, transforms, or reasons about text before the model acts on it: detector models, perplexity filtering, delimiting, datamarking, instructional prevention, paraphrasing, adversarial fine-tuning. The control is asking a question about content. Is this string data or is it an instruction.

Around the capability. Everything that limits what the agent is permitted to do regardless of what it decides: tool allowlists, session-scoped credential removal, architectural separation between the planner and the executor. The control never reads the text. It reads the action request against a fixed policy.

Downstream at the action. Everything that observes what the agent actually did and compares it against what that agent normally does: runtime behavioral baselines, per-agent enforcement at the syscall and tool boundary. The control never reads the text either. It reads execution.

One clarification before the evidence. Whether a control is deterministic is a separate question from whether it survives an adaptive attacker. Capability control is deterministic: a policy permits a data flow or refuses it. Action-boundary control splits in half. Detecting that behavior deviates from a baseline is statistical and carries a false positive rate you will pay for in triage. Enforcing a per-agent policy derived from that baseline is deterministic once the policy exists. Collapsing the two is how runtime products get oversold.

Position predicts robustness better than technique does. The eight defenses in the Zhan study were a detector model, an LLM-based detector, perplexity filtering, instructional prevention, delimiter isolation, sandwich prevention, paraphrasing, and adversarial fine-tuning. Different mechanisms, three different implementation layers, one position. All of them sit upstream of the decision, and all of them broke.

Upstream Controls Are Text Classification Against an Adaptive Adversary

Upstream defense reduces to a classification problem: given a span of text, decide whether it is content the agent should process or an instruction it should refuse. The classifier can be a fine-tuned model, a perplexity threshold, a delimiter convention the system prompt tells the model to respect, or weights adjusted through adversarial training. The shape of the problem does not change.

Classification against adversarial input has a known failure mode, and language models did not fix it. A deployed defense holds still. Its behavior can be probed, its concentration points found, its gaps mapped, and an attacker with unlimited attempts and a copy of the paper will find the seam and route around it, which is the entire distance between a self-reported number and an adaptive one. Moving second is usually a disadvantage. Here it is the advantage.

The gap is the number to bring to your next vendor conversation. A defense publishing 2% attack success against a static benchmark and above 90% against an adaptive one is not carrying 2% of risk. The published figure describes a threat model nobody faces.

The prompting defenses fare worst of all, which matters because they are the cheap first line most teams reach for. That same adaptive evaluation moved spotlighting and prompt sandwiching from originally reported attack success in the low-to-mid twenties into the mid-to-high nineties once the attacker optimized against the design. A convention that tells the model to treat marked text as data holds exactly as long as nobody is trying.

None of this makes upstream defense worthless. It filters the opportunistic volume, the copy-pasted “ignore previous instructions” payloads that make up most of what actually hits a public-facing agent. What it cannot do is carry the load against a targeted attacker. Treat it as a rate reducer, and assume every payload written specifically for your stack passes it.

Capability Controls Hold Inside a Defined Scope

Capability restriction is the honest response to an unsolved classification problem, and the serious research went there.

Meta’s Agents Rule of Two states the constraint plainly: within a single session an agent should satisfy no more than two of three properties. Processing untrustworthy input. Accessing sensitive systems or private data. Changing state or communicating externally. The framing is explicitly provisional, holding until robustness research allows reliable detection and refusal of injection.

The UK’s NCSC arrives at the same position from the standards side. Its assessment reads prompt injection as a confused deputy problem, with the complication that a classical confused deputy can be patched while an LLM is inherently confusable. The work is reducing risk and impact through design, and where a system cannot tolerate the residual risk, the guidance says plainly it may not be a good use case for an LLM.

The strongest published implementation of that position is CaMeL, which extracts control flow and data flow from the trusted user query and executes the plan through an interpreter, so untrusted retrieved content can never alter the program. Capabilities attached to each value govern where it is allowed to travel. On AgentDojo it solves 77% of tasks with provable security against 84% for an undefended agent.

Seven points of task completion for a structural guarantee is a good trade, and the guarantee holds.

The limit is scope, and CaMeL’s authors state it themselves. The guarantee covers control flow and data flow. It does not cover an attack whose consequence never touches a data flow: an injection that makes the agent summarize a document dishonestly, or draft phishing content inside an otherwise authorized reply, satisfies every policy because no protected value moved anywhere it should not. The threat model also assumes the agent’s memory has not already been compromised, which is an assumption and not a control.

So a data-flow guarantee is precise about where information can go and silent about what the agent does with the authority you left it.

Downstream Controls Measure Behavior Instead of Text

Inside those permitted flows, the agent is not compromised in the way an exploited process is compromised. It is redirected. It holds its own credentials, calls its own registered tools, hits endpoints it is authorized to hit, and every individual action passes every authorization check you have. This is coercion: a trusted input redirecting an agent’s authorized capability toward the attacker’s goal.

Coercion is why the sequence is the attack and no single step is. Your IAM policy fires on unauthorized access and nothing here is unauthorized. Your egress control fires on unknown destinations and the destination is on the allowlist. Your container runtime fires on unexpected processes and the process is the one that always runs. Every control in the path was built to catch an actor who should not be there, and the actor is one you provisioned. We have previously mapped the full progression from poisoned retrieval through to exfiltration, stage by stage, across a production cluster.

What changes is the pattern. A support-triage agent that has queried the ticket store and the knowledge base for three months does not normally reach the customer records table, and it does not normally reach it immediately after retrieving a document from a source it has never pulled from before. The individual action is authorized. The sequence is out of character.

Measuring that requires a definition of normal belonging to one specific agent. ARMO builds it as Application Profile DNA (APD™), a per-agent behavioral baseline assembled from kernel-level observation of what the workload actually does: which tools it invokes, which APIs it reaches, which destinations it contacts, which system calls are routine for it. The baseline is derived from execution rather than declared in configuration.

Enforcement then runs from the same signal. The progressive path from observation to enforcement starts in audit, where deviations are recorded and the policy is proven safe against real traffic, and moves to enforce, where deviations are stopped. Credential handling closes the same loop from the other side: the agent holds only substituted keys, so an action that clears every other check still fails at the point where it tries to use something real.

The obvious attack on a behavioral control is to stay inside the envelope. An attacker who knows a baseline exists can spread the coercion across sessions and widen the agent’s tool usage gradually, so no single deviation clears a threshold. That attack is real and it is the hardest case in this article. What makes it expensive is where the attacker has to work. A baseline is built from one agent’s execution history in one cluster, so it is unpublished, unqueryable, and different in every deployment. The attacker optimizes blind and cannot carry the result to the next target. An upstream detector offers the opposite conditions: download the weights, iterate offline, ship a payload that scores clean everywhere. Behavioral evasion is slow, per-target, and noisy for as long as it runs.

The good news is that this control does not touch reasoning quality. It never enters the agent’s context, never sits between the model and its tools as a classifier, and never narrows the space of things the agent may consider. It narrows what the agent may execute, measured against that agent’s own history. The agent stays as useful as you built it.

Blast Radius Sets the Control Budget

The injection technique does not vary by agent. Blast radius does, and blast radius is what the control budget should follow.

Split your agent inventory across two axes: exposure to untrusted input, and authority to change state. That produces three classes worth treating differently.

Read-only agents with untrusted exposure. Public chat surfaces, document summarizers, research agents. They process attacker-reachable content and cannot write. Upstream filtering is proportionate here, because the consequence of a successful injection is a wrong answer rather than a changed system. Spend on output inspection and on narrowing what each agent is allowed to read, since the remaining exposure is disclosure through the response itself.

Write-scoped agents with untrusted exposure. Support triage, ticket remediation, inbox automation, anything reading customer-supplied text and writing to a system of record. This is the dangerous middle and it is where most production agents live. Run both surviving controls, because their failure modes are disjoint. A data-flow policy stops the customer record leaving for an attacker-supplied destination. It permits the agent to write a wrong resolution into the ticket, close the case, and notify the customer, every step inside an authorized flow. Behavioral enforcement covers that second failure, and the signal lives in the tool-invocation sequence rather than in any single call.

Infrastructure agents. Coding agents in remote development environments, cluster remediation agents, anything holding cloud credentials or Kubernetes API access. Apply Rule of Two properly and remove a property, because the blast radius justifies the utility loss. Then add behavioral enforcement underneath, since permission scoping does not catch tool misuse inside the permitted scope.

Four questions separate action-boundary enforcement from upstream controls wearing a runtime label.

Ask whether the behavioral baseline is derived from observed execution or declared in configuration. Ask whether enforcement is per-agent or per-namespace, since a namespace-wide policy cannot distinguish two agents with different normal behavior. Ask whether the platform runs in audit before it enforces, because a vendor who cannot show you the deviations first is asking you to gamble production on an untested policy. Ask what sits between the model’s decision and the tool call, since a control that only reads logs afterward is telemetry. That single distinction sorts most of the categories on an AI workload security shortlist.

The Injection Is Not the Event Worth Instrumenting

The injection is a string in a document, and strings are cheap to write, cheap to vary, and cheap to retry until one lands. Building your program around catching that string means competing on a surface where the attacker moves last and moves for free.

The agent’s behavior is not cheap to fake. To complete a coercion the attacker has to make the agent do something, and doing something leaves execution evidence against a baseline the attacker never saw and cannot query.

Instrument the thing the attacker cannot avoid producing.

ARMO’s runtime security for AI workloads builds per-agent behavioral baselines from kernel-level observation and enforces against them without code changes. Watch a demo to see what a coerced agent looks like against its own baseline.

Frequently Asked Questions

Can indirect prompt injection be prevented entirely?

Not at the input layer. Language models process instructions and data through the same mechanism, so there is no parameterized-query equivalent that separates them structurally, and OWASP’s guidance on prompt injection holds that no foolproof prevention exists. Consequences can be bounded structurally, though. Information-flow control designs give provable guarantees over where data travels at a cost of roughly seven points of task completion on AgentDojo. What they do not cover is what the agent does inside the flows you permitted, so bounding real blast radius takes a data-flow guarantee and a control over the actions the agent keeps authority to perform.

How do I know whether my current defenses were adaptively tested?

Check whether the published attack success rate came from a static benchmark suite or from an adaptive evaluation where the attacker knew the defense design. Static-benchmark numbers routinely sit near zero and rise above 90% under adaptive testing, so the gap between the two evaluations is more informative than either figure alone. If the vendor cannot tell you which evaluation produced the number, assume static and discount accordingly.

What telemetry do I need before behavioral detection will work?

Per-agent tool invocation records, the API destinations each agent reaches, process and syscall activity attributable to the workload, and identity attribution tying all of it to a specific agent rather than a pod or namespace. Kernel-level collection through eBPF gets most of this without code changes. Framework-level SDK instrumentation adds the prompt and tool-call context the kernel cannot see.

How long does a per-agent baseline take to become useful?

It depends on how repetitive the agent’s work is. A narrow task loop, such as ticket triage against a fixed set of tools, converges within days because the tool and destination set is small. Broad tool access and varied tasks take longer and produce a wider envelope. Run in audit mode throughout, and use the falling deviation rate as your signal that the baseline has settled.

Does behavioral detection replace input filtering?

No. Input filtering removes the opportunistic volume before it reaches the agent, which cuts the number of events behavioral detection has to adjudicate. Behavioral detection catches what gets through, including payloads written specifically to defeat the filter you deployed. They cover different portions of the same attack surface.