Lending
Backstop
Model Risk Auditor
Audits a credit decision for soundness, consistency, and fair-lending integrity (SR 11-7 / 12 CFR 1254 flavor).
sonnet5 tools3 eval cases2 connectors
The case it handles
A representative case
Case fileL1-clean
- Decision · Application
- L1
- Decision · Product
- auto
- Decision · Decision
- approve
- Decision · Terms
- 3 fields
- Decision · Score band
- prime
- Decision · Dti pct
- 11
- Decision · Ltv pct
- 103
- Decision · Policy basis
- score 768 in the prime band, DTI within the maximum, LTV within the maximum
- Decision · Factors
- strong credit score, low revolving utilization, clean payment history
- Decision · Confidence
- 0.96
Raw scenario input (JSON)
{
"decision": {
"applicationId": "L1",
"product": "auto",
"decision": "approve",
"terms": {
"apr": 6.49,
"amount": 31000,
"termMonths": 60
},
"scoreBand": "prime",
"dtiPct": 11,
"ltvPct": 103,
"policyBasis": [
"score 768 in the prime band",
"DTI within the maximum",
"LTV within the maximum"
],
"factors": [
"strong credit score",
"low revolving utilization",
"clean payment history"
],
"confidence": 0.96
}
}The contract
What it takes, does, and returns
Give it
- Decision · Application
- Decision · Product
- Decision · Decision
- Decision · Terms
- Decision · Score band
- Decision · Dti pct
- Decision · Ltv pct
- Decision · Policy basis
- Decision · Factors
- Decision · Confidence
It does
- Get applicationget_application
- Get credit reportget_credit_report
- Get policyget_policy
- Find similarfind_similar
- Record assessmentcommitrecord_assessment
It returns · Rating
- Pass
- Findings
- Fail
How it's checked
Checked against golden cases
3golden cases3baseline
- L1-clean
prime approve, within policy, factors support it → pass
- L2-supported
decline rests on hard policy fails (score below min, LTV over max) → sound; pass, minor findings acceptable
- L11-inconsistent
within-policy near-prime file declined with no hard basis; similarly-situated L12 was approved → findings/fail
Each case is a real failure mode the grader checks on every change and as a deploy gate — regressions past threshold block the release.
Briefing
Briefing
- Second-line audit of a committed credit decision in the spirit of SR 11-7: tests whether the decision made is defensible, without re-underwriting the file.
- Checks consistency against policy and against similarly-situated files decided before — on credit merit only, never protected attributes.
- Tests whether the stated factors genuinely support the outcome; a decline resting on factors that do not fail policy is a finding.
- Treats deviations from policy as overrides needing documented justification; a clean decision passes, because inventing issues is itself a failure of the audit.
- Rates pass, findings, or fail, naming the comparator or factor behind anything below pass.
Where it stands in the operation