Measuring triage

About 16 min read

On this page

Speed metrics measure how fast the queue moves. They say nothing about whether the verdicts were right, and a team that is measured only on speed learns to be confident, fast and wrong. This page defines the numbers a triage team should report instead: each one with its formula, its denominator, the record field it comes from, how often to read it, and the way it gets gamed. The definitions are written so they can be pasted into a SIEM search or a ticketing report, and so two teams using them get numbers that compare.

It is written for the SOC lead who runs the metrics review, and for the analyst who wants to know what their records are counted for. The adoption guide says when to start measuring (a baseline before anything changes) and how often to review; this page says what to measure.

Principles

  • Diagnostic, not target. A measure made into a target stops measuring (Goodhart’s law, in its popular form). Every number here is a question to ask about the queue, never a quota. The moment closes-per-shift becomes the implicit scoreboard, every other triage metric starts lying.
  • Never rank individual analysts on speed. Flow and quality numbers belong to the team. Per-analyst speed rankings teach people to take the easy cases and skip the hard checks; close-note quality turned into a leaderboard kills the honesty the rubric depends on.
  • Every number has a pair. Speed is read beside a quality check, and each quality check beside the failure it hides. The pairs are in the tables below; a metric reported without its pair is not reported.
  • The case is the unit of work. Queue and quality metrics count cases (alerts grouped as the fast path describes). Detection metrics count alerts, per rule, because detection engineering tunes rules, not cases.
  • Medians and the 90th percentile, not means. Triage times have long tails. One case stuck for three days moves a mean more than a hundred quick closes; report the median (p50) and the 90th percentile (p90).
  • Write the formula down before the dashboard. Numerator, denominator, window, segment and source field, recorded next to the baseline, or the number will be quietly redefined the first time it looks bad.
  • The receiver judges the escalation. Escalation acceptance and handoff rework come from whoever receives the case, never from its author.
  • Counts beside rates when the denominator is small. Two missed incidents out of four is a rate of 50 percent and a sample of four. Below about twenty in the denominator, report the count and review each case.

Translate the IR-coupled metrics to your team model

Several metrics below are written as if a separate incident response (IR) function receives the escalations. On a small or single-person team, translate the counterparty without dropping the measurement: “IR accepts” means whoever your escalation path engages (an on-call engineer, a managed provider, an IR retainer) agrees the case warranted it, and “rework” means the packet’s next reader had to ask for something it should have contained, even when that reader is you a week later. When a managed provider runs first-line triage, your team measures its escalations the same way. What must survive the translation is the independence.

Which clock: the timestamps triage owns

Every flow metric is a difference between two timestamps, so the timestamps come first. The event report’s decision header records the key times; the handoff packet adds the page and the receipt.

TimestampWhat it marksWhere it is recorded
DetectedThe detection’s own time: when the alert firedAlert report, Level 0 record, event report key times
AcknowledgedThe first recorded analyst action on the case: a query, a note, a step. Opening or claiming the ticket does not countEvent report key times, or the case tool’s activity log
VerdictWhen both verdict axes were first recordedEvent report key times
Escalated or closedThe Escalation decision, including early exits (a Level 0 close, a tripwire escalation)Event report key times; Level 0 record “Closed at”
First criterion confirmedWhen the first escalation criterion reached High confidenceHandoff packet timeline
Paged, page acknowledgedThe P1 page and its acknowledgment, or the 15-minute fallbackHandoff packet P1 page block
AcceptedWhen the receiver took ownershipHandoff packet receipt
Contained, reopenedContainment taken effect; the case reopened on new evidenceEvent report key times

MTTD and MTTR are not triage metrics

Mean Time To Detect (MTTD) Detection time minus the time of first malicious activity. An incident metric, not a triage one: only IR's compromise timeline establishes the time of first activity, and for a benign close it does not exist. FIRST's Security Incident Timing Metrics define it. See Measuring triage. needs the time of the first malicious activity, which only the compromise timeline that IR reconstructs can establish; for a benign close it does not exist. FIRST’s Security Incident Timing Metrics (SITM) FIRST's reference for incident timelines (version 1.0, May 2023): eleven timeline records and four key metrics (time to detect, respond, contain and remediate), each measured from the time of first activity. These are incident clocks; triage's own clocks start at detection. See Measuring triage. measure time to detect, respond, contain and remediate all from that time of first activity, which makes them incident metrics. Report them from IR’s timelines, joined to triage’s key times, never from the queue alone. Mean Time To Respond/Resolve (MTTR) A widely quoted speed metric whose R is ambiguous: respond, resolve, remediate or recover, depending on who reports it. Properly it is an incident metric: FIRST's Security Incident Timing Metrics measure time to respond, contain and remediate from the time of first malicious activity, which only IR's compromise timeline establishes. Never report it without its formula. Triage's own clocks (time to acknowledge, time to verdict, time to decision) start at detection, and a mean hides the long-tail cases, so report medians and the 90th percentile and pair speed with quality measures such as the reopen rate and handoff rework. See Measuring triage. is the worse of the two, because the R is read as respond, resolve, remediate or recover: never report it without its formula. The triage clocks are time to acknowledge, time to verdict and time to escalate.

The metric set

Five groups. Windows are defaults; pick yours, write it down, and keep it. “Cases” means cases closed or escalated in the window unless a row says otherwise.

Flow: how the queue moves

MetricFormulaSourceCadenceGamed by, and its pair
Time to acknowledge ( Mean Time To Acknowledge (MTTA) A triage flow metric: acknowledged minus detected, reported as the median and the 90th percentile per priority band rather than as a mean. Acknowledged means the first recorded analyst action on the case; opening or claiming a ticket does not count. See Measuring triage. )p50 and p90 of acknowledged − detected, per priority bandKey timesWeeklyAuto-acknowledge rules and bulk claiming. Count only a recorded action. Pair: SLA-breach rate
Time to verdict by levelp50 and p90 of verdict − acknowledged, per depth levelKey times, depth levelWeeklyClosing before the checks are done. Pair: reopen rate for the same level
Time to escalatep50 and p90 of escalated − detected, P1 and P2 onlyKey times, priorityMonthlySending P2 packets before they are complete to beat the clock (a P1 pages first by design). Pair: handoff rework
Criterion-to-page intervalpaged − first criterion confirmed, every P1, listedPacket timeline, P1 page blockEvery P1Back-dating the confirmation. Each one is reviewed, not averaged; a long one is a late P1
SLA-breach rateCases whose acknowledge or decision clock was exceeded ÷ cases in that band, per bandPriority, key times, the default clocksWeeklyDowngrading the priority to escape the clock. Pair: de-escalation rate and reopens
Uninvestigated shareAlerts closed with no disposition (aged out, bulk-closed, auto-closed outside a machine-close entry) ÷ alerts receivedCase tool status, verdict fieldsWeeklyBulk closes stamped with a boilerplate verdict, which moves them into “investigated”. Pair: a QA sample drawn from bulk closes

Time-in-phase belongs here too, as a diagnostic only. Where the case tool records phase transitions, it shows where cases spend their clock: long Uncover tails point at retrieval and coverage gaps (the three visibility questions), long Alert phases at parsing and enrichment debt. Time stalled before the Escalation decision on cases that end P1 points at criteria treated as an end-of-arc checklist instead of a tripwire. Never set per-phase time targets: they produce phase-skipping.

Quality: were the decisions right

MetricFormulaSourceCadence
Escalation acceptanceEscalations whose receipt says “Escalation warranted? yes” ÷ escalations with a filled receipt. Report “partially” on its own lineHandoff packet receiptMonthly
Receipt fill rateEscalations with a filled receipt ÷ escalationsHandoff packet receiptMonthly
Handoff reworkEscalations whose receipt names rework (anything but “none”) ÷ escalations with a filled receipt, broken down by packet sectionHandoff packet receiptMonthly
Handback A receiving owner returning an escalated case to the sending tier with a named reason: the evidence chain is missing, the criteria or the cell are not supported, or the problem is detection tuning. Recorded on the packet's receipt and counted by reason. See Case D. and de-escalationCases handed back ÷ escalations, by reason; P1 and P2 cases lowered ÷ P1 and P2 cases (Case D is one handback)Packet, protocolsMonthly
Reopen Rate Cases closed in a month and reopened on new evidence within 30 days, divided by the cases closed that month. A rising rate points at closes made before the deciding evidence was in. See Measuring triage. Cases closed in the month and reopened on new evidence within 30 days of their close ÷ cases closed in the month, per level and per pattern entryKey times “reopened”, reopen conditionsMonthly, one month in arrears
Missed-incident rateConfirmed incidents with at least one related alert or case that triage had closed as benign ÷ confirmed incidents. Always list themIR post-incident reviews, threat hunts, red-team findingsQuarterly
Close-note qualitySampled notes passing all five rubric checks ÷ notes sampled; plus the failure rate per check; report the sample sizeWeekly QA sampleWeekly, read as a trend
Verdict Agreement The share of blindly re-triaged cases that reach the same activity verdict, priority band and decision as the original. Report it with Cohen's kappa, which corrects for agreement by chance. See Inter-Rater Agreement and Measuring triage. Blind re-triage cases where the second analyst records the same activity verdict ÷ cases re-triaged; the same for priority band and for the decision. Add Cohen’s kappa, because most cases are benign and raw agreement flattersMonthly blind re-triageMonthly

The quality metrics are where gaming does the most damage, so each carries its trap:

Escalation acceptance quality of the E decision

Low acceptance means noisy escalation, criteria applied loosely, or packets too thin to judge. The trap: driving it to 100 percent, which a team achieves by under-escalating. A healthy number sits below perfect, because some defensible escalations turn out benign. Read it only beside missed incidents.

Reopen rate honesty of the closes

The most practical routine check on close quality, and only as complete as the paths that bring new evidence back to closed cases (re-fired alerts, retro-hunts, purple-team results, the missed-incident reviews below): reopens concentrate where discriminators were skipped, negatives went unrecorded, or the pattern reflex closed something the pattern did not cover. Count a new case that cites an earlier close on the same entity as a reopen, or the number can be dodged by opening fresh tickets. The trap: punishing reopens, which teaches analysts to argue against reopening instead of writing better closes. A reopen against a rubric-passing note is a detection lesson; against a thin note, a QA lesson.

Missed incidents the heaviest signal

Rare, and each one outweighs a quarter of good numbers. Every miss gets a blameless review against the original record, asking which phase’s deliverable would have caught it. The trap: treating zero as the target and shading post-incident timelines to protect it. The record discipline in the Documentation example exists so these reviews have honest material.

Handoff rework quality of the packet

Cheap to collect and directly actionable: recurring gaps point at one packet section going unfilled. The trap: measuring it without the receiver, which turns it into triage grading its own homework. A low receipt fill rate makes acceptance and rework meaningless, so report it first.

Close-note quality sampled, not census

The weekly sample is stratified so Level 0 records and AI-assisted notes are always in it. Trend matters more than level, and the score belongs to the team: a falling trend is a training or workload signal. The trap: a per-analyst leaderboard, which converts an editing tool into a surveillance tool. Check 3 asks analysts to admit uncertainty, and nobody does that for a leaderboard.

Verdict agreement is the method shared?

A second analyst re-works a random sample from the same raw evidence without seeing the first verdict. Disagreement is not an error count: it is where the team’s reading of the method diverges, and the teaching guide treats exactly that as the thing to train. The facilitator kit has the kappa worksheet and the agreement thresholds. The trap: letting the second analyst see the first verdict, or picking the cases; both inflate agreement.

Detection health: per rule, shared with detection engineering

These count alerts, per rule and version, over a rolling 30 days. They feed the conversation the benign-close feedback loop starts, and they use the detection-axis verdict on every member alert of a grouped case.

MetricFormulaWhat it separates
Precision (Detection) Detection-axis true positives divided by alerts with a detection verdict, per rule and rule version. Benign true positives count as correct: a rule that fires accurately on authorized activity is precise and needs context, not a logic change. See Per-rule numbers. Detection-axis true positives (which include Benign True Positive An alert that fired on exactly the activity its detection logic describes, where that activity turns out to be authorized or expected in the local context: a red-team exercise, an admin's sanctioned use of a dual-use tool, a scanner doing its job. In the verdict record it is a true positive on the detection axis and authorized on the activity axis. The detection worked; the context clears it. Distinct from a false positive, where the logic matched something it should not have, so never record it as one. The close does not tune the rule, which is healthy. It feeds context instead: a pattern-entry proposal, or a scoped, expiring exception keyed to the tool version, extension or plugin ID and destination set, with an owner and re-fire triggers. Platform labels differ: Microsoft Sentinel calls this verdict Benign Positive, and Defender XDR calls it Informational, expected activity. Full entry: ASSURED terms. ) ÷ alerts with a detection verdictBroken logic from noisy but correct rules
Actionability Alerts whose activity verdict is malicious or policy violation, divided by alerts with a verdict. Beside precision, it separates rules that need exceptions or enrichment from rules that need logic fixes. Detection engineering's per-rule variant counts fires that needed any action beyond the close. See Measuring triage. Alerts whose activity verdict is malicious or policy violation ÷ alerts with a verdictRules that need Exception (Detection) A narrow carve-out in which the rule still runs and only a defined, context-keyed tuple is excluded from alerting. A governed exception is scoped, owned, approved by a second person, expiring, stored as code and monitored for hits. Distinct from suppression, which hides the alert, and tuning, which changes the logic. See Exception and suppression governance. or enrichment from rules that need logic fixes
Benign-true-positive shareTrue positives with activity “authorized” ÷ alerts with a verdictWork for an exception or enrichment, owned and expiring
False-positive share, logic and data False Positive A detection-axis verdict: the alert's logic matched something it should not have, because the behavior the rule claims did not occur. ASSURED records two kinds: false positive (logic), a logic flaw or an over-broad pattern, and false positive (data), a parsing error, a missing field or bad telemetry. The fix is detection-side: tune, refine, or repair the data feed. Distinct from a benign true positive, where the behavior really happened but was authorized; that is never a false positive. Platform labels differ: Microsoft Sentinel's False Positive and Defender XDR's False positive mean this, while their Benign Positive and Informational, expected activity mean a benign true positive. (logic) ÷ alerts; false positive (data) ÷ alerts, reported separatelyA rule fix versus a parsing or pipeline fix
Malicious yieldAlerts ÷ malicious true positives (alerts per real finding)The base rate the analyst should calibrate trust against
Tuning cycle timep50 of change merged − request openedWhether the feedback loop moves at all
Exception inventoryActive exceptions per rule; share past expiryStale exceptions, where attackers hide

Gamed by: tuning a rule until its precision looks good, which can silence the malicious variant along with the noise; and relabeling benign true positives as false positives, which sends a correct rule for a logic change it does not need. Pair: missed incidents, and a known-malicious test sample re-run after every tuning change.

Fast-path health: is Level 0 earning its speed

MetricFormulaCadenceGamed by, and its pair
Level mixCases closed at Level 0, Level 1 and Level 2 ÷ cases closed, plus machine closes as their own lineWeekly, as a trendCelebrating a rising Level 0 share. Pair: reopen rate per level
Level 0 Reversal Rate Level 0 closes overturned on re-verification, divided by closes re-verified, per pattern entry. One confirmed false close reaches the default reversal threshold and suspends the entry. See Measuring triage. Level 0 closes overturned on re-verification ÷ Level 0 closes re-verified, per Pattern LibraryASSURED term In ASSURED, the SOC's written catalog of known-benign alert patterns, each entry with named discriminators, an owner, a version and a review date, approved by a second person. It is governed like code: version history, invalidation triggers (a rule change, observed abuse), an optional machine-close flag, and a reversal threshold that suspends the entry automatically. A five-minute Level 0 close is earned by a current entry, not granted by the timer. See the fast path. Full entry: ASSURED terms. Weekly sample, read monthlySampling only the easy closes. The sample is random, per pattern
Re-verification coverageActive entries with at least one close re-verified this week ÷ active entries with closes this weekWeeklySkipping low-volume entries. Should be 100 percent
Entries past review dateCount, and ÷ active entriesMonthlyExtending the date without a review. Should be zero, and a review leaves a version-history line
Close-rate anomaliesEntries whose closes or host spread this week fall outside their normal range (count)WeeklyWidening the normal range. Each one is a Level 2 trigger under the ladder rules
Benign-close feedback Tuning RequestASSURED term In ASSURED, the single artifact triage sends detection engineering after a verdict: one rule per request, evidence cited by ID, the proposed change and its risk, and a benign and a malicious test sample. See The tuning request. filed from benign-close findings, and closedMonthlyFiling tickets nobody reads. Pair: tuning cycle time

The tell of a fast path going wrong is a rising Level 0 share with rising reopens: the reflex is winning, not the library.

AI use: is the assistant earning its place

Split every quality metric three ways, using the Material AI use field and the closer named on the record: unassisted (Material AI use “none”), AI-assisted (the field names an assistant), and machine-executed (closed by automation on an entry with the Machine-Close FlagASSURED term In ASSURED, a per-entry, per-version switch on a pattern entry, off by default, that permits a machine close once every Level 0 parity condition holds. Turning it on needs a second person's approval, and any change to the entry turns it off again. Full entry: ASSURED terms. ). The gap between the segments is the only local evidence of whether the assistant helps. Reopen rate, missed incidents, close-note quality and reversal rate matter most, and machine-executed closes carry the reversal and miss tracking the governed tier requires.

Three more numbers apply only to AI-assisted work, measured on the QA sample:

  • Draft correction rate: AI-drafted sections the analyst changed materially ÷ AI-drafted sections reviewed. Near zero is not a sign of a perfect assistant; it is one of the rubber-stamp signals, alongside approval times shorter than the checks take.
  • Citation validity: claims in AI drafts whose evidence link resolves to the cited tool output ÷ claims checked.
  • False-benign rate per alert class: on the fixed regression set of closed cases, re-run on every model, prompt or tool change. The governance page covers Golden Set A held-out set of your own closed cases with settled verdicts, used to evaluate an AI assistant or automation before and after every change. It covers each alert class with the hard cases: confirmed malicious cases, benign true positives that look malicious, false positives, past misses, injection-laced samples and cases whose right answer is undetermined. See Governing automated triage. sizing and Shadow Mode Running an AI assistant on the live queue with its output recorded but hidden from the working analyst, so its verdicts can be compared with the settled ones without anchoring the person it is checked against. It comes before any write access. See Governing automated triage. .

The trap: accepting the vendor’s aggregate accuracy figure, measured on someone else’s alerts, in place of these.

Capacity math

Whether a queue is staffable is arithmetic, and it is worth running before arguing about tooling. The fast path gives the short form; here it is in full.

Analyst-hours a day = cases reaching a person × Σ (level share × minutes at that level) ÷ 60

Analyst-shifts a day = analyst-hours ÷ productive hours per shift

“Cases reaching a person” is what is left after grouping, tuning and governed machine closes. Productive hours are what is left of a shift after handover, meetings, the QA and re-verification samples, pattern maintenance and training; the table assumes 6 of 8. The worked example uses the intro’s volume of 3,000 alerts a day and minutes per level of 5 (Level 0), 30 (Level 1, the middle of its range) and 90 (Level 2, an experienced analyst’s full arc).

ScenarioLevel 0Level 1Level 2Analyst-hoursShifts (6 h)
3,000 cases, 70/25/52,100 → 175 h750 → 375 h150 → 225 h775about 129
3,000 cases, 80/17/32,400 → 200 h510 → 255 h90 → 135 h590about 98
600 cases (80% removed upstream), 70/25/5420 → 35 h150 → 75 h30 → 45 h155about 26
600 cases, 80/17/3480 → 40 h102 → 51 h18 → 27 h118about 20

Three things the table says:

  • An 80 percent Level 0 rate does not rescue an unreduced queue. Raising the Level 0 share from 70 to 80 percent saves about a quarter of the hours, and Level 0 alone still costs 200 analyst-hours a day at 3,000 cases. The lever is the volume that reaches a person: grouping, tuning and governed machine closes, in that order of safety.
  • The middle rung is where the hours go. At 70/25/5, Level 1 is a quarter of the cases and almost half the hours. Cheaper context (enrichment that fetches the eligibility-gate facts, a gate made cheap) moves more time than a faster Level 0.
  • Run it backwards to set the intake you can honor. Weighted minutes per case are 15.5 at 70/25/5 and 11.8 at 80/17/3. Twelve analyst-shifts a day at 6 productive hours is 4,320 minutes, enough for about 280 cases at the first mix and about 365 at the second. Demand above that does not disappear: it shows up as uninvestigated share and SLA breaches.

Plan below full load. As utilization approaches 100 percent, queue waiting time grows without limit, and the P1 that arrives at a full queue waits behind everything else. Leave headroom for the cases that climb the ladder mid-shift, and treat capacity as a legitimate escalation reason when it runs out (the tier model at 3 a.m. shows how).

Fields the records must carry

None of these numbers can be computed from free text. Each one depends on a field that the templates already define; wire them into the case tool as structured fields, with the verdict and priority values spelled exactly as the templates spell them.

FieldWhere it livesFeeds
Case ID and every member alert IDAll templatesCase counts, grouping ratio
Detection rule name, ID and versionAlert report, Level 0 recordEvery detection-health metric
Depth level, and any upward movesAlert report, event reportLevel mix, time to verdict by level
Both verdict axes and the status (closed, escalated, merged into case)All templatesPrecision, shares, actionability, uninvestigated share
Priority and matrix cellAlert report, event reportSLA-breach rate, time to escalate
Key times (detected, acknowledged, verdict, escalated or closed, contained, reopened)Event report decision header; at Levels 0 and 1, the case tool’s own timestamps or an added lineEvery flow metric, reopen rate
Pattern entry ID and version; closed by an analyst or by automationLevel 0 recordFast-path health, the machine-executed segment
Material AI useAll templatesThe AI split
Receipt: accepted by and at, escalation warranted, rework requestedHandoff packetAcceptance, rework, receipt fill rate
Packet timeline and P1 page blockHandoff packetCriterion-to-page interval
Reopen conditions, and a link from a reopened case to its earlier closeEvent report, case toolReopen rate

The monthly triage report

One page, the same every month, built from the fields above: monthly-triage-report.md. It opens with volume and capacity (alerts in, cases reaching a person, demand in analyst-hours against hours available), then shows each quality metric beside its pair and its baseline, the detection-health table for the noisiest rules, fast-path health, the AI split, and a short list of every missed incident and reopen. It ends with the work items the numbers produced, each with an owner. A report that produces no work item is a meeting, not a steering input.

The monthly report feeds the quarterly metrics review the adoption guide describes, run against the Stage 2 baseline. It never carries a per-analyst ranking.

How this relates to published metric sets (as of October 2026)
  • FIRST Security Incident Timing Metrics (SITM) v1.0 (released 29 May 2023; SITM is owned by FIRST): eleven timeline records and four key metrics (time to detect, respond, contain and remediate), each measured from the time of first activity. These are incident-level clocks; this page’s triage clocks start at detection. FIRST Metrics SIG.
  • FIRST Metrics for the FIRST CSIRT Services Framework FIRST's catalog of the services and functions an incident response team can offer (version 2.1, November 2019). The functions closest to triage are 5.2.1 Correlation, 5.2.2 Qualification (deciding whether an event needs handling) and 6.2.1 Incident triage (prioritization and categorization). See Standards mapping. v1.1 (May 2026): its event-management and incident-triage sections include time to acknowledge alerts and incident reports, false-positive ratios per detection use case, time from incident receipt to triage completion, and the error rate of incident triage. Time to acknowledge, per-rule precision, time to verdict and verdict agreement here are close relatives. Version 1.1.
  • MITRE, 11 Strategies of a World-Class Cybersecurity Operations Center (second edition, 2022): Strategy 10, “Measure performance to improve performance”, including a section on measures that do not lead to positive outcomes. MITRE.
  • SOC-CMM The SOC Capability Maturity Model, a self-assessment of a security operations center across five domains (business, people, process, technology, services); version 2.4 appeared in late 2025. It asks whether a SOC has a defined triage process, not what that process should be. See the adoption maturity model. assesses the whole SOC; the adoption guide’s maturity model shows where a triage-practice score sits inside it.

Key takeaway

Measure decision quality first and speed only as its pair: escalation acceptance against missed incidents, Level 0 share against reopens, time to verdict against the reopen rate for the same level. Write each formula down with its denominator and source field before anyone builds a dashboard, let the receiver judge the escalation, split every quality metric by AI use, and never rank analysts on speed. Then run the capacity math: the depth ladder only works on the volume that grouping, tuning and governed machine closes leave for people.

Technical details verified . Product names, defaults and retention change; check your own tenant.

Edit this page on GitHub Report a correction