Field test
Play release gatekeeper.
Five decisions from a real AI release pipeline, with the evidence that was on the table at the time. The naive call is available at every step. So are the consequences.
- Demo
- Replay
- Threshold
- Refusals
- Verdict
Step 1 · The demo looks great
A release candidate beats the offline eval.
The demo is convincing and the curated benchmark agrees. Your release gate exists for exactly this moment. What is your call?
Current system~91.2
Candidate~93.5
Numbers marked ~ are illustrative interpolations. The anchors are real: the replayed regression, the 99% filed-field threshold recovering 31 points of automation, and the 9% refusal budget cutting unsupported claims from 14% to 1.8% are documented in the decision docs linked from the verdict.