Runtime Incident Classification: Turning a Noisy Alert List Into a Triage Decision

Here is a scene every security team knows. A reverse shell opens a connection to an external address, pulls a service-account token, and starts moving against your cloud identity. Two rows below it on the same dashboard sits a payload that hit a front-end container and never executed. Both are tagged high severity. Both are competing for the same analyst’s attention at the same moment.

We have spent years arguing that the core problem with runtime security is noise: too many findings of equal weight, which trains people to fix nothing because everything looks the same. Severity scoring was supposed to help, and it does tell you one thing: how bad an incident could be in the abstract. What it does not tell you is the thing an analyst actually needs in the middle of the night – what should I do about this, and how fast.

That is the gap runtime incident classification closes. It adds the axis severity was always missing: not how bad, but how urgent, and what response that urgency calls for.

Four labels, each tied to a different response

Classification sorts every runtime incident into one of four categories. The point is not the label on its own – it is that each label maps to a decision your team has already made about how it responds.

• Active Threat is activity that strongly indicates an ongoing compromise. The response is to investigate and contain now.
• Attempted Attack is malicious activity that was blocked, failed, or never reached compromise – the context shows it did not land. The response is to confirm it was stopped, then make sure that attacker will not have a second chance.
• Review Required is activity that is suspicious or specific to your environment and needs a person to decide whether it is expected. The response is to validate it against context first.
• Informational is low-risk activity that is almost certainly benign but worth keeping for visibility and audit. The response is to file it, not to page anyone.

This is a different axis from severity. A blocked attempt and an active compromise can carry the same severity and still demand opposite responses, and the classification is what separates them. Each label also arrives next to the MITRE tactic the incident maps to, so the category and the technique sit in front of you together. While an incident is still being evaluated it shows as Analyzing, and when there is not enough context to classify it reliably, it is marked Not available and stays open for you to handle by hand.

Active Threat vs Attempted Attack: same alert, opposite response

The split that earns its keep is the one between Active Threat and Attempted Attack, because that is the line between an incident that ends your evening and one that does not.

Take an Active Threat. A Python process opens an interactive outbound shell to an external address. On its own, that might be one alert among hundreds. What makes it an Active Threat is what follows and how it connects: the shell is followed by service-account token access and cloud identity activity, and the timeline ties those events into a single chain instead of three disconnected alerts on three dashboards. 

The reasoning you see reads in plain terms – outbound interactive shell behavior, a suspicious process-and-network combination, behavior that deviates from the application’s baseline. You are not handed a score to interpret. You are handed the short version of the analysis a SOC engineer would otherwise assemble by hand, and from there you move straight to the response options in the same view.

Now take an Attempted Attack that looks similar at a glance. A command injection payload arrives at a front-end container over HTTP and tries to download and run a shell script from an external server. The activity is real and hostile. The difference is the outcome: there is no evidence the script ran. Same family of behavior, opposite situation. Instead of paging anyone, you review the context and dismiss it.

That distinction is the whole value. Both incidents can show up as critical; only the classification tells you that one is a live compromise to contain and the other is a probe that already failed, so your team spends its attention where the attack is live, not where it was stopped. This is the same cross-layer correlation that builds ARMO’s attack story, pointed at the one question triage exists to answer. We’ve laid out the tiers that decision breaks into; classification is that decision layer, now native to the platform.”

When the system isn’t sure, it asks you – and logs what you decide

Not every incident is a clean call, and the classification does not pretend otherwise. Review Required is the category for activity that is suspicious or specific to your environment and needs a person to weigh in.

Here is a typical one. An IAM user creates an EC2 instance with a custom user-data script through Terraform. In isolation that pattern can read like a persistence technique, so it lands as Review Required rather than being waved through or escalated. Opening the incident story, the Terraform workflow and the staging context make clear this is known infrastructure activity, not an intrusion.

At that point you decide. An analyst with the right role can reclassify the incident, choosing a reason from a fixed set – known operational behavior, false positive, confirmed attack, testing or simulation activity, benign anomaly, or other – and adding an optional note. In this case the activity is routine operational behavior, so it moves to Informational. That is what Informational is for: the incidents you want recorded for audit and visibility, filed rather than fired at anyone.

What matters as much as the change is that the change is remembered. Every reclassification keeps a full history – the original label, the new one, the reason, the explanation, who made the change, and when – visible both on the dashboard row and on the incident’s Story tab alongside the attack timeline it relates to. The classification is a starting point the platform commits to, and the human override is tracked, not a silent edit.

Wire the label to the response, so the page fires only for what’s live

Classification is most useful when it stops being something you read and becomes something that acts. In threat detection policies, the classification is a trigger: you choose which categories a policy applies to, and route the response from there.

Most teams set this up the same way. A policy scoped to Active Threat fires an urgent response – report, stop, kill, or pause the affected container, apply a network policy or seccomp profile to quarantine it, and push the incident to your SIEM or ticketing workflow. A separate policy can take Review Required incidents with a lighter touch, a lower-priority notification for someone to look at during the day. Everything classified Informational stays in the record without interrupting anyone.

This is where detection, classification, and response finally sit in one workflow instead of three tools. And because the classification is AI-driven, it carries into the place this matters most right now: the AI workloads your platform teams are shipping, where the volume and unpredictability of runtime behavior make undifferentiated alerting unworkable. The full threat detection policy configuration covers the options, but the model is simple – the label decides who gets paged.

Triage is a decision, not a score

For a long time we treated triage as a sorting problem: rank everything by severity and work down the list. But severity only measures how bad an incident could be. It cannot tell you whether an attack is happening now, was already stopped, or is just your own team running Terraform. That judgment is the real work of triage, and it is the work classification does. It puts a decision next to every incident, grounds that decision in the attack story, keeps you in control of it, and lets you wire it to a response. The outcome is the one thing alert fatigue takes away: attention spent on what is live and ongoing, and not on the noise around it. Book a demo to see runtime incident classification in your own environment.

FREQUENTLY ASKED QUESTIONS

How does ARMO decide an incident’s classification?

When a runtime incident is detected, the platform evaluates it against the definitions of the four categories using the incident’s full context and its place in the correlated attack story. It returns the most appropriate label along with short, plain-language reasoning for the decision. The reasoning is there so you can see why a label was assigned, and the incident classification documentation describes how each category is defined.

Can I change a classification when I have more context?

Yes. Users with Admin or Manager roles can reclassify any incident, selecting a reason from a fixed list and adding an optional explanation. The original label, the new one, the reason, who changed it, and when are all preserved in the incident’s history. That means an override is always traceable, on both the dashboard and the incident’s Story tab.

What happens to an incident while it is still being classified?

While the analysis runs, the incident shows as Analyzing and stays fully accessible for investigation. Once complete, it updates with its label and reasoning. If there is not enough context to classify it reliably, it is marked Not available and remains open for you to review or classify by hand.

How do I make sure my team is only paged for the urgent incidents?

Use classification as a trigger in your threat detection policies. Scope an urgent policy to Active Threat so only those incidents fire immediate response actions and high-priority notifications, and handle the other categories with separate, lighter policies. When a policy is scoped to a classification, it waits for the classification to finish before acting, which usually happens shortly after detection.

Does classification replace severity?

No. Severity still tells you how serious an incident could be; classification tells you how urgent it is and what response it calls for. The two work together, alongside your existing filters for cluster, namespace, and status. That lets you build focused views, like active threats in your production namespace.