Design Patterns for Securing LLM Agents Against Prompt Injection: What Every Pattern Leaves Open

Count the concessions in the paper that introduced these patterns. It works through ten case studies, and in eight the authors’ own assessment of their most constrained design concedes something that gets through anyway. Malicious files can still be moved somewhere they do not belong. Resumes can still boost their own ranking. Retrieval can still be persuaded to omit a contraindication. Recipients and bodies can still be shaped by an attacker who never got to choose the action.

Seven of those eight concessions use the same word to do it.

That count is the most useful thing in the paper, and it is the part every summary of it skips. If you write the agent, the six patterns tell you what to build. If you run a cluster full of agents somebody else wrote, they tell you something more immediately useful: exactly what the architecture you inherited leaves open, and therefore what you are expected to cover underneath it.

The Patterns Buy Control-Flow Integrity, and That Is Worth Buying

Start with what works, because it genuinely does.

Design Patterns for Securing LLM Agents against Prompt Injections, written by fourteen authors across Google, Microsoft, IBM, ETH Zurich and EPFL, proposes six ways to constrain an agent so injected instructions cannot redirect it. The action-selector picks from a fixed list and never sees the result. Plan-then-execute commits to its tool calls before touching untrusted data. Map-reduce shards untrusted documents across isolated sub-agents. The dual LLM pattern, credited in the paper to Simon Willison, pairs a privileged planner that never reads untrusted text with a quarantined worker that reads it and holds no tools. Code-then-execute replaces the plan with a program, instantiated as a working system in CaMeL. Context-minimization strips the user’s prompt from the context once it has done its job.

One principle runs under all six. Once an agent has ingested untrusted input, it must be constrained so that input cannot trigger any consequential action.

Worth noting what the paper declines to do. It states in a footnote that it draws no distinction between direct and indirect prompt injection and treats both as a single class, which is the same conclusion you reach from the defender’s side once you accept that the split stops mattering at the context window. Where the text entered is an ownership question. What the agent does next is not.

This is a real advance and worth saying plainly. Each pattern is a structural property you can reason about on a whiteboard, which is more than any detection model offers. Classifiers give you a probability. An action-selector agent that never sees tool output gives you an argument. That is why the paper holds application-specific agents to be securable today and general-purpose ones not: constraints are checkable where heuristics are not.

So the honest verdict is that the patterns work at what they claim. They provide control-flow integrity: untrusted data cannot change which tool the agent calls, or in what order. The paper says so directly about plan-then-execute, borrowing the term from systems security.

Control flow is one half of a tool call.

In Eight of Ten Case Studies, the Authors Concede Something Gets Through

The paper applies its patterns to ten worked examples, grading each candidate design on utility and security. Take the most constrained design the authors grade in each case, read their own security bullet, and the pattern is hard to miss.

§	Case study	Most constrained design graded	What the authors concede gets through anyway
4.1	OS assistant with fuzzy search	Dual LLM with map-reduce	Malicious files can still be moved or copied somewhere they do not belong
4.2	SQL agent	Action-sandboxing for the interpreter	Sandbox reconnaissance and data extraction through analysis output stay possible
4.3	Email and calendar assistant	Plan-then-execute, then dual LLM	Recipients and bodies remain influenceable; the quarantined model’s response can be tampered with
4.4	Customer service chatbot	Action-selector over a request allowlist	Embeddings can still be manipulated toward the wrong allowlisted request
4.5	Booking assistant	Least-privilege user access	Injections in third-party calendar descriptions and provider content still reach the agent
4.6	Product recommender	Map-reduce over reviews	A single review can still make itself fit every scoring category
4.7	Resume screening assistant	Map-reduce retrieval with dual LLM summary	Resumes can still boost their own ranking; unprivileged model responses stay manipulable
4.8	Medication leaflet chatbot	Context-minimization	Users can still persuade the model to omit relevant sections, including contraindications
4.9	Medical diagnosis intermediary	Structured formatting	Nothing. The schema leaves no room
4.10	Software engineering agent	Dual LLM with strict data formatting	Injection must survive conversion to an API description, which the authors call unlikely

Source: Beurer-Kellner et al., case studies, §4. Quoted assessments are the authors’ own.

One row deserves an asterisk. The booking assistant’s last two designs, restricted calendar access and dual LLM, carry no security assessment at all, so the concession above comes from the last design the paper actually grades. An ungraded design is not a validated one.

Read the right-hand column on its own and the residuals sort themselves. Placements. Recipients. Bodies. Embeddings. Rankings. Omissions. Analysis output. Every one is a value carried by an action the agent was authorized to take.

This is not a weakness in the paper, and these are not unrelated gaps that better engineering would close one at a time. It is one property showing up eight times. The patterns prevent untrusted input from selecting an action, and they do that. Nothing in any of the six was designed to govern what the selected action carries.

Nor is the column hypothetical. The same shape appears across the named incidents already on the record, where the vendor fix repeatedly closed the exfiltration channel and left the injection itself untouched.

The Residual Closes Exactly Where the Output Space Closes

Two rows in that table break the pattern, and they are the most instructive rows in it.

In the medical diagnosis case, the authors constrain the model’s summary of symptoms into a structured object with no open-ended text fields. Their verdict is that the design leaves no room for prompt injection in the summary. In the software engineering agent, untrusted documentation is converted by a quarantined model into a strictly formatted API description, with method names capped at thirty characters. Injection would have to survive that conversion, which the authors call unlikely.

Both exceptions work the same way. They shrink the output space until there is nothing left to inject into.

That gives you a rule you can apply to your own fleet without reading the paper again: the residual closes where the output space closes. Where an agent’s job can be reduced to filling fields in a schema, a pattern can eliminate injection influence entirely. Where the agent has to write a sentence, choose a recipient, order a list of candidates or decide where a file lands, the residual survives by construction.

Price it before you reach for it. The authors are candid about the cost. Structured formatting works for symptoms only if symptoms fit a rigid format. The agent that sees API descriptions can no longer read natural-language documentation or example code, and the paper marks its utility as reduced. Constrain a coding assistant’s output space far enough and you have built an autocomplete, which is the trust tax that removes the reason the agent was deployed.

So the split is by agent class rather than by pattern quality. Triage bots, routers and scoring services can be closed. Copilots, research assistants and coding agents have open output spaces because open output is the product. Those are also the agents your organization actually deployed, which is why prompt injection in production agent workloads rarely ends at the architecture review.

The Pattern Fixes the Verb; the Object Stays Open

Every tool call has two parts. The verb is the tool invoked: email.write, file.move, db.query. The objects are the arguments it carries: the recipient, the destination path, the query string.

The six patterns lock the verb. All of them. An action-selector agent has a fixed verb list. A plan-then-execute agent has a fixed verb sequence. A map-reduce agent has one verb per shard. Not one of the six constrains the object.

So the residual in that table has a single shape. An agent performs an action from its approved list, using its own credentials, at a moment its architecture permits, carrying arguments an attacker shaped. No policy fires, because no policy was broken. OWASP catalogs the condition as excessive agency and tool misuse, which names the risk without telling you how to see it.

ARMO calls this coercion, and the naming matters less than the recognition that the paper’s residual column and this threat model are the same object described twice. A coerced agent is not a hijacked one. It is an authorized one, misusing its own permissions in a sequence nobody wrote down as forbidden.

Here is the part that decides who deals with it. Choosing a pattern is a build-time commitment made by whoever writes the agent. The residual is a runtime inheritance handed to whoever runs the cluster, and those are almost never the same team. The vendor who built your copilot picked its pattern. The engineer who shipped the internal LangChain service picked one, or picked none. The coding agent in your dev environment came with whatever its publisher decided. You own the residual from all three and can re-architect none of them.

Three Residual Shapes, Three Different Things to Watch

The obvious objection lands first, so take it first: this is input validation with extra steps. Validate the arguments and the residual closes.

It does not, and the reason is worth being precise about. Validation is stateless and per-call. It confirms that a recipient is a well-formed address, that a path sits inside an allowed prefix, that a query matches a grammar. It cannot confirm the recipient is the right one. Every residual in that table describes a call that passes its own schema: the resume was a valid resume, the file move a valid move, the email went to a real address. What marks them is that this agent has not sent to that address, reached that prefix, or ordered candidates that way in any previous run. That is a property of the call measured against the agent’s own history.

Sort the surviving residuals by what an attacker gets to influence and you get three shapes, each leaving different evidence.

Parameter residual. The argument values on an approved call: the email recipient, the destination path, the query filter. It is the most consequential of the three because it is the one that reaches outside the workload, and it has a named protocol-level instance. The highest-scoring MCP tool poisoning paradigm never executes a poisoned tool at all, completing instead through an already-approved one with a single changed argument.

Selection residual. Which item gets retrieved, ranked or surfaced: the boosted resume, the review that scores well against every category, the leaflet section quietly left out. Nothing leaves the system, and the agent’s decision has been made for it.

Output residual. What the agent emits downstream, including into another agent’s context. The tampered summary from a quarantined model is this shape, and it is the one that propagates across an agent fleet rather than stopping at the workload where it landed.

None of the three is visible at the prompt layer, because none involves an instruction the agent refused or accepted. They are visible in what the workload does: which endpoints it reaches, which files it opens, which processes it spawns, how its calls sequence against every previous run.

That is a comparison problem, and the good news is that comparison needs only a reference. ARMO builds one per agent from kernel-level observation, called Application Profile DNA (APD™): a runtime-derived record of what normal looks like for that specific workload rather than for workloads in general, at 1 to 2.5% CPU and 1% memory with no sidecars and no code changes. An approved verb carrying an unusual object is invisible to a policy engine and legible against a baseline, because the baseline knows which objects that verb has carried before. The same comparison surfaces intent drift when an agent’s authorized actions stop resembling its own.

Detection is where this starts. Enforcement is a separate decision with a separate cost, which is why ARMO’s runtime security for AI workloads runs in audit before enforce: observe the agent, generate the constraint from what it did, then promote from alerting to blocking once the evidence says the constraint is safe. A policy written from observed behavior is one you can defend to the team whose production it governs.

One bound on all of this. Kernel-level observation covers workloads on infrastructure you control. A vendor-hosted copilot processing your data inside someone else’s tenancy sits outside that reach, and the honest answer there is contractual rather than behavioral.

Pick the Pattern for the Agent You Build; Instrument the One You Inherit

Two audiences read this keyword and they need different things.

If you are writing the agent, use the patterns, and use more than one. That is the paper’s own second recommendation and it holds. Close the output space wherever the job tolerates a schema, because a residual you designed away needs nothing watching it. Then write down which residual your pattern leaves, because that note is the handover document your platform team needs.

If you inherit the agent, the architecture question is already settled and you were not in the room. What remains is the object column: which values move through approved actions, and whether this run resembles every previous one. That is behavioral ground, and it is the same ground whether the agent in front of you implements plan-then-execute, context-minimization or nothing at all. Size the response to it accordingly, because the interval between a scored signal and an executed containment is what decides whether the object had already left.

The verb was decided at build time. The object shows up at runtime, and so does the evidence.

Book a demo to see how ARMO baselines agent behavior and turns a deviation into a single attack story.

Frequently Asked Questions

Which design pattern should I use for an agent that reads email and calendar data? Plan-then-execute is the paper’s recommendation here, because the choice and sequence of actions for an email assistant almost never legitimately depend on the content it reads. Have the agent commit to its tool calls before retrieving anything, then execute. The residual you accept is that recipients and message bodies stay influenceable, including through invisible-character encoding, so pair it with egress monitoring on the outbound path.

Can I apply these design patterns to an agent I did not build? No. All six are architectural commitments made inside the agent’s code, available to you only if you control the source. For inherited agents, including vendor copilots and third-party coding assistants, your options sit outside the agent: scoping its credentials, constraining its network reach, and baselining what it does at runtime so unusual arguments on approved actions become visible.

Does plan-then-execute stop indirect prompt injection? It stops indirect injection from changing which tools the agent calls, which the paper describes as a form of control-flow integrity. It does not stop injected content from shaping the arguments those calls carry. Treat it as narrowing the attack from arbitrary action to influenced action, then size your remaining controls to what an influenced action can reach.

How do I tell a legitimate change in agent behavior from a residual being exploited? Compare against the agent’s own history rather than a generic policy, because agents change behavior legitimately whenever prompts, models or tools are updated. A per-agent baseline gives you the reference: a new endpoint, an unusual argument on a familiar call, or a call sequence that has never occurred are all deviations from that specific agent. Correlating several into one chain separates a model update from an incident.

Do I still need guardrails if I have implemented a design pattern? Yes, and they sit in different places. Guardrails screen the content entering and leaving the conversation, design patterns constrain what the agent’s control flow can do with that content, and runtime behavioral detection watches the actions that result. Sorting published defenses by where they sit relative to the agent’s decision is the fastest way to see which of the three you are missing, and OWASP LLM01 defines the risk class all three are answering.