Replay of a real run — captured 2 Jul 2026
Calibrate
Scenario-Tuning Agent
Every step below is a real trace event from the signed run ledger.
sc-calibrate
Transcript
I'll work through this methodically — history first, then hypothesis, backtest, gate, and finally commit. Let me start by pulling the labeled alert history.
Excellent — the history tells a clear story. Let me break it down before forming a hypothesis:
Current cashDepositThreshold: 8,000 | Total alerts: 8
| Alert | firedValue | Label |
|---|---|---|
| H-STR-04 | 8,200 | fail FP |
| H-STR-06 | 8,400 | fail FP |
| H-STR-05 | 8,600 | fail FP |
| H-STR-07 | 8,800 | fail FP |
| H-STR-03 | 9,100 | pass TP |
| H-STR-02 | 9,400 | pass TP |
| H-STR-01 | 9,500 | pass TP |
| A1 | 9,700 | pass TP |
Current precision: 4/8 = 50% | Current recall: 4/4 = 100%
Key observation: There is a clean gap between the highest false positive (8,800) and the lowest true positive (9,100). All 4 false positives cluster between 8,200–8,800; all 4 true positives sit at 9,100 and above. Raising the threshold to 9,000 should eliminate all FPs while preserving every TP — a precision improvement with zero recall cost.
Let me record this hypothesis and run the backtest simultaneously:
Backtest confirms the hypothesis exactly. Now I'll run the binding eval gate:
Backtest results (proposed = 9,000):
- Precision: 1.00 (+0.50 vs current 0.50) ✅ Major improvement
- Recall: 1.00 (±0.00 vs current 1.00) ✅ No coverage loss
- Alert volume: 4 (−4 vs current 8) — volume halved, driven entirely by FP removal
The gate passed cleanly — not marginal. All three conditions satisfied:
| Gate condition | Result |
|---|---|
| Precision strictly improves | pass 0.50 → 1.00 (+0.50) |
| Recall ≥ floor (0.90) | pass 1.00 ≥ 0.90 (held perfectly) |
| Volume not catastrophically collapsed | pass 4 alerts (halved, not zeroed) |
gatePassed: true, marginal: false, impliedRecommendation: adopt — the gate certifies this change as safe. Committing now:
The decision brief
Tuning Decision — structuring scenario
Recommendation: ADOPT
| Current (8,000) | Proposed (9,000) | Δ | |
|---|---|---|---|
| Precision | 0.50 | 1.00 | +0.50 |
| Recall | 1.00 | 1.00 | 0.00 |
| Alert volume | 8 | 4 | −4 |
Reasoning
The alert history showed a structurally clean false-positive band: every FP fired between 8,200 and 8,800, and every TP fired between 9,100 and 9,700 — no overlap, no ambiguity. The 300-unit gap between the highest FP (8,800) and the lowest TP (9,100) allowed a threshold of 9,000 to sit squarely between them.
- Volume reduction (−4 alerts, −50%) is entirely attributable to FP suppression — no true positives were lost at this threshold.
- Recall held at 1.00, comfortably above the 0.90 recall floor. No financial crime coverage was sacrificed.
- Precision doubled (0.50 → 1.00), cutting analyst workload in half.
- The eval gate returned
gatePassed: true,marginal: false, andimpliedRecommendation: adopt— a clean, non-borderline certification. Change is safe to promote to production.