How Does Prompt Injection Work? 5 Steps From a Sentence to a Tool Call

Prompt injection works in five steps, and four of them happen inside the model.

The comparison everyone reaches for is SQL injection, and it is the right comparison for exactly one step. Both attacks put an instruction where the system expected data. The difference is what the system does next. A database engine hands the string to a parser that knows, structurally, which bytes are the query and which are the parameter, and the fix for SQL injection was to make the application tell the parser. A language model has no parser. It has a sequence of tokens and a trained habit of doing what the tokens say.

That habit is the mechanism. Here is the chain, one step at a time, ending where it ends: in a tool call executed with your service account.

Step One: Everything the Model Reads Becomes One Sequence

The developer’s system prompt, the user’s message, the document the agent retrieved and the result the tool returned all arrive at the model as one stream of tokens.

That is the whole of Willison’s original definition: the attack works by concatenating untrusted input with a trusted prompt. Concatenation is the literal operation. The application takes strings from several sources, joins them, and sends the result. Whatever structure the developer believed existed between “instructions” and “content” existed in the application’s code, and the join flattened it.

Chat-formatted models add role tags to the stream, and those tags look like a boundary. They are text. A model reads a system tag, a user tag and an assistant tag as tokens like any others, and research on abusing chat templates for injection shows that content which imitates the template’s own markup can persuade the model that a new turn, with a new role, has begun. The tag is a convention the model learned to respect. It is not a wall the model cannot see past.

Delivery path ends at this step. Whether the text was typed by the user or retrieved from a page decides who could attack you, and once the join happens the two are the same tokens.

Step Two: No Token Carries a Privilege Bit

Nothing in the sequence records which tokens are allowed to issue instructions.

Every other injection class was fixed by adding that record. Parameterized SQL queries send the query and its data through separate channels, and the parser never treats a parameter byte as syntax. A CPU runs kernel code in a privilege ring that user code cannot enter, and the hardware enforces the ring. In both cases the boundary is a property of the system, checked deterministically, independent of what the untrusted bytes say. Input validation is the first thing every security team reaches for on a new injection class because it has worked on every previous one.

A token sequence has no such property. The model’s next-token prediction runs over the entire sequence with one set of weights, and the weights have no register that says “this span is trusted.” Researchers have tried to add one. StruQ trained models on a structured query format with reserved delimiters between instruction and data and filtered those delimiters from user content, and reported a large reduction in attack success against the attacks it was designed for. The residual is the finding: even with a structured format enforced at the application and trained into the model, the model still reads the data span as text it can act on, because reading text as text is what it does. A separate line of work, Zverev and colleagues at ICLR 2025, asked directly whether language models can separate instructions from data and concluded that current models do not have that property in any principled sense.

This is why the mitigation prescribed for the formal weakness entry does not complete. CWE-1427 is filed as a neutralization weakness, and neutralization means removing dangerous characters. There is no character to remove from “confirm the customer’s eligibility against the accounts table.” There is no privilege bit to check, because there is no bit.

Step Three: Instruction Following Is Trained Into Every Position

A model follows an instruction inside a retrieved document for the same reason it follows an instruction in the user’s message: it was trained to.

Pretraining teaches a model to predict text. Instruction tuning then teaches it that when text has the shape of a request, the most likely continuation is compliance. That training does not come with a position restriction. Nothing in it says “only follow imperatives that appear in the second message.” Perez and Ribeiro showed in 2022 that the model would abandon its assigned goal for one placed in the input, and Greshake and colleagues showed the next year that the input could be a web page the model fetched on its own. Same mechanism, further away.

The consequence for attackers is that the most effective payload is the plainest one. An imperative-shaped sentence that reads like operational text triggers the trained behavior without triggering anything else, which is why the techniques that reach production data are the ones that perform no evasion. Override phrases and encodings exist to defeat filters and refusals. Against a model whose job is to read a ticket and act on it, there is nothing to defeat.

The consequence for defenders is a coupling. The property being exploited is the property that makes the model useful. A model that reliably ignored instructions in retrieved content would also ignore the instructions in the document a user asked it to summarize, and instruction following that generalizes better is, in this one respect, injection that generalizes better.

Step Four: The Instruction Hierarchy Is a Preference the Model Learned

Vendors answer step three by training the model to prefer some instructions over others, and the answer works at a rate.

The instruction hierarchy, as OpenAI’s researchers describe it, defines how a model should behave when instructions of different privilege conflict and trains the model on generated examples of that behavior. Spotlighting marks retrieved content so the model can learn to treat it as lower priority. Both raise resistance to injection, and the papers measure the raise. Both also report residual attack success, and the 2026 release of a training dataset built to improve instruction hierarchy on frontier models is evidence that the frontier still needs the training.

A trained preference has a failure rate because it is a learned generalization rather than an enforced check. It holds on the distribution it was trained on and degrades off it. That is the mechanism underneath a pattern the site has documented from the other side: every published input-side defense was evaluated and every one broke against an adaptive attacker, a detector’s headline number averages over technique classes it scores very differently, and any control that acts before the model’s decision is a rate rather than a property. None of those are tuning failures. They are what a preference looks like when measured.

The preference is still worth training and worth deploying. Attackers in the wild reuse templates, and a model that has learned to discount the common ones removes most of what is actually sent. Keep it. The point is what it is: the last probabilistic step in the chain.

The next step is different in kind. It is the first one with a boundary in it.

Step Five: The Application Turns Output Into an Action With Its Own Credentials

The model produces text, and the application decides the text is a tool call.

With function calling, the model emits a structured block naming a tool and its arguments. The application parses it, looks up the tool, and executes it, using whatever identity the application holds: a service account, an API key, a database role, a network route. The model never touched any of those. It produced tokens that the application chose to treat as a command, and borrowed authority is the third part of the definition, the one that turns an odd paragraph into an incident. From here the attack runs through eight more stages in production, and the text is at none of them.

This is also the first step where a deterministic control can exist. The tool either is or is not registered. The credential either does or does not reach the table. The network policy either does or does not route to the destination. A policy on the call either is or is not armed. None of those depend on what the tokens say, which is what the first four steps could never offer. The confused deputy NCSC describes cannot be taught whom to obey, so the fix is to constrain what it can do, and step five is where “what it can do” is decided.

The record of the attack lives here too. A tool call with arguments, a credential exercised against a table it has never touched, a connection opened to a destination outside the agent’s history: those are observable from the kernel regardless of how clean the prompt was. ARMO records that layer with an eBPF sensor and builds a per-agent behavioral profile, Application Profile DNA (APD™), so that a step-five action is evaluated against what this agent normally does rather than against a fleet-wide rule. The case the profile exists for is coercion, where every step is authorized and the sequence is the attack, which is step five seen in full.

Turning the profile into a boundary follows the Audit and Enforce sequence: observe the agent’s real behavior first, review what a policy would have blocked, arm it per agent once the record supports it. Credential isolation adds a bound of a different kind. When the agent holds placeholder credentials and the real ones are substituted outside its reach, a step-five call that needs the key completes with a key that opens nothing. The first four steps ran exactly as before. The fifth reached less.

ARMO’s runtime security for AI workloads sits at step five. Bring one agent’s tool registry and IAM policy to a working session and the chain gets traced from its real prompts to its real calls.

Four Steps Belong to the Model Vendor and the Fifth Belongs to You

The sequence, the missing privilege bit, the instruction following and the trained hierarchy are properties of the model, and the vendor’s roadmap is the only thing that changes them.

Spend there and you are tuning a rate you do not own. The fifth step is a boundary you do: which tools, which credentials, which destinations, which policy. One plain request per consequence that boundary allows tells you where it actually holds, and the list is short enough to finish.

The model reads everything as text. What it can do with the text is yours.

FAQ

Why can’t input validation stop prompt injection? Validation needs something to validate against, and a plain-language instruction contains nothing that distinguishes it from a plain-language request. There is no dangerous character, no forbidden syntax and no privilege marker on any token. Filters catch the payloads that carry override phrasing or encoding, and those are the payloads that were performing evasion; the ones that reach production data perform none.

Does prompt injection work on every LLM? On every instruction-tuned model, because instruction following is the property being exploited and it is trained into every position of the input. Rates differ: models trained with an instruction hierarchy resist common override templates far better than untrained ones. No published model has removed the mechanism, and the vendors who train the hierarchy report residual attack success in their own evaluations.

Why does prompt injection get worse as models get more capable? Because capability at following instructions in context and vulnerability to instructions in context are the same generalization. A model that better understands what a document is asking for is a model that better understands what an injected sentence is asking for. The hierarchy training pushes back, and it pushes back at a rate rather than as a hard limit.

What is the instruction hierarchy? A training approach, described by OpenAI researchers in 2024, that teaches a model to prioritize instructions from higher-privilege sources such as the system prompt over lower-privilege ones such as user input or retrieved content. It raises resistance to common override attacks measurably. It is a learned preference, so it holds best on the attacks it was trained on and degrades off that distribution, which is why it reduces prompt injection without eliminating it.