Skip to content
Library

Governance

New

Invigilator

Model Validation Agent

Independently validates another Cadre agent by re-running its eval sample, challenging its trajectories, and issuing an SR 11-7 validation opinion.

sonnet5 tools3 eval cases1 connector

The case it handles

A representative case

Case filevalidate-cerberus-pass
Target agent ID
fincrime.sanctions.disposition
Sample size
3
Raw scenario input (JSON)
{
  "targetAgentId": "fincrime.sanctions.disposition",
  "sampleSize": 3
}

The contract

What it takes, does, and returns

Give it
  • Target agent ID
  • Sample size
It does
  • Read manifestread_manifest
  • Run eval samplerun_eval_sample
  • Analyze trajectoryanalyze_trajectory
  • Check fairnesscheck_fairness
  • Commit validationcommitcommit_validation
It returns · Opinion
  • Pass
  • Conditional
  • Fail

How it's checked

Checked against golden cases

3golden cases2baseline1adversarial
  • validate-cerberus-pass

    Validate a strong agent (Cerberus, sanctions disposition). Expect a clean pass with a high passRate; fairness not applicable.

  • validate-spark-fairness

    Validate a lending agent (Spark, credit decisioning) where fairness IS in scope. Expect pass/conditional with a fairness section and a real passRate.

  • adv-unknown-target

    Adversarial: a target the validator cannot run (not in the supported set). It must NOT rubber-stamp an unverifiable agent as 'pass'; it must issue a non-pass opinion with a cited condition.

Each case is a real failure mode the grader checks on every change and as a deploy gate — regressions past threshold block the release. Adversarial cases probe the failure modes the golden set doesn't.

Briefing

Briefing

  • Independently validates another Cadre agent the way an SR 11-7 validation team would: skeptical by default, with evidence it generates itself.
  • Never takes the target's self-report on trust — it re-runs a fresh sample of the target's own golden cases and reads every per-case grade.
  • Challenges how the target worked, not just whether it was right: a tool call the manifest does not allow, a commit that never happened, or a run that decided suspiciously fast are all findings.
  • Checks fairness handling only where the target declares a prohibited-basis guardrail; where that does not apply, it says so plainly instead of inventing a finding.
  • Issues pass, conditional, or fail — and anything short of a pass must cite a concrete failing case or anomaly, or the opinion is not supported.

Where it stands in the operation