Scoreboard
Release 1 has one fault per incident. Release 2 adds 11 harder ones: faults in both sources at once, a real fault behind a louder harmless change, decoys, lost logs, legitimate changes that look like corruption, and alerts identical to a release 1 incident with a different answer.
A confident error is a wrong root cause asserted as a diagnosis, in any split. Abstaining on an incident whose evidence cannot support a cause counts as correct. The coverage guardrail never changes a root cause, so a flagged error still counts as a confident error; the flag rows show what it caught and what it flagged needlessly. It only checks diagnoses on runs whose output reached consumers, a few incidents per release, so most diagnoses could never be flagged. Runs recorded before the guardrail existed get its verdict by replay.
Incident console
Select an incident. Highlighted text in a tool output is a passage the investigator quoted as evidence; the grounding check verified it is there verbatim.
What is being checked
The same harness scores both investigators against labels they never see.
Read-only tools
Ten tools over the recorded evidence: run ledger, checks, contracts, profiles, lineage, logs, runbook. No tool can change data, re-run a pipeline, or approve anything.
Verified citations
Each claim cites a tool call and a quote. A quote that is not in that call's output is discarded, and a diagnosis with no verified quote is withheld.
Calibrated abstention
Novel incidents have no supportable root cause. Saying so, and what is missing, scores as correct; inventing a cause is a confident error.
Investigation coverage
When a run's output reached consumers, a diagnosis is marked for review unless the investigator examined the delivery's column profile, including its largest change. Real citations cannot show evidence that was never gathered.
Human approval
Fixes are proposals from a closed catalog. A policy that reads the evidence rejects unsafe ones, and a named person with the right role approves the rest.