# Model Validation Report: [Model name] [version]

> Independent validation report for a quantitative or machine learning
> model: model overview and use, materiality, conceptual soundness, data
> quality, outcomes analysis and back-testing, benchmarking, sensitivity
> and stress testing, implementation checks, ongoing monitoring,
> limitations, findings and the validation conclusion. Structure follows
> the components of model validation in the US banking agencies'
> *Supervisory Guidance on Model Risk Management* attached to Federal
> Reserve SR letter 26-2 (17 April 2026), which supersedes SR 11-7
> (2011). Replace every `[bracketed]` value and every row marked
> *Example*. Worked examples use a retail credit probability of default
> (PD) scorecard.

<!--
Guidance: How to use this template
- The validator writes this report; the developer supplies evidence.
  Record independence and any conflicts in section 2.
- Scale depth to model materiality (section 3.3). A low-materiality
  model may need only sections 1, 3, 6, 10, 13 and 14 in brief.
- Every finding must point to evidence and have a severity, an owner and
  a due date. A conclusion without findings is rarely credible.
- SR 26-2 states that generative and agentic AI models are outside its
  scope; for those, use this structure together with the LLM evaluation
  plan and AI red team templates.
-->

## Document control

<!-- Guidance: Identify the model version validated, the validation type and the people accountable for the conclusion. -->

| Field | Value |
|---|---|
| Report ID | [MV-2026-014] |
| Model ID in inventory | [MDL-0231] |
| Model name and version | [Retail PD scorecard v4.0] |
| Validation type | [Initial / Periodic / Change / Targeted] |
| Model tier (materiality) | [Tier 1 high / Tier 2 medium / Tier 3 low] |
| Version | [0.1] |
| Status | [Draft / In review / Final] |
| Lead validator | [@validator] |
| Model owner | [@model-owner] |
| Model developer | [@developer] |
| Classification | [Confidential] |
| Last updated | YYYY-MM-DD |
| Related documents | [Model development document], [Model card], [ML system design], [Monitoring plan] |

### Approvals

| Role | Name | Decision | Date |
|---|---|---|---|
| Lead validator | [@name] | [Final] | YYYY-MM-DD |
| Head of model validation | [@name] | [Approved / Changes requested] | YYYY-MM-DD |
| Model owner (acknowledges findings) | [@name] | [Acknowledged] | YYYY-MM-DD |
| Model risk committee | [Committee] | [Approved for use / Conditional / Not approved] | YYYY-MM-DD |

## Revision history

<!-- Guidance: Record report drafts and the developer responses that changed findings. -->

| Version | Date | Author | Change summary |
|---|---|---|---|
| 0.1 | YYYY-MM-DD | [@name] | Draft findings |
| 0.2 | YYYY-MM-DD | [@name] | *Example:* Developer responses added; MV-F-03 downgraded after new evidence |
| 1.0 | YYYY-MM-DD | [@name] | Final report |

## Contents

<!-- Guidance: Keep aligned with the headings below. -->

1. [Executive summary](#1-executive-summary)
2. [Validation scope and approach](#2-validation-scope-and-approach)
3. [Model overview and use](#3-model-overview-and-use)
4. [Conceptual soundness](#4-conceptual-soundness)
5. [Data quality review](#5-data-quality-review)
6. [Outcomes analysis and back-testing](#6-outcomes-analysis-and-back-testing)
7. [Benchmarking](#7-benchmarking)
8. [Sensitivity and stress testing](#8-sensitivity-and-stress-testing)
9. [Implementation and process verification](#9-implementation-and-process-verification)
10. [Ongoing monitoring plan](#10-ongoing-monitoring-plan)
11. [Limitations and compensating controls](#11-limitations-and-compensating-controls)
12. [Third-party components](#12-third-party-components)
13. [Findings](#13-findings)
14. [Validation conclusion and conditions](#14-validation-conclusion-and-conditions)
15. [Sign-off](#15-sign-off)
16. [Glossary and references](#16-glossary-and-references)

## 1. Executive summary

<!-- Guidance: One page for the committee: what the model does, the conclusion, the most important findings and the conditions of use. -->

| Item | Summary |
|---|---|
| Model purpose | [PD estimation for unsecured personal loans, used in approval and pricing] |
| Conclusion | [Fit for purpose / Fit for purpose with conditions / Not fit for purpose] |
| Findings | [0] critical, [1] high, [3] medium, [2] low |
| Key conditions | [Conditions of use and deadlines] |
| Next validation | YYYY-MM-DD |

*Example:* "The scorecard is conceptually sound and discriminates well
(out-of-time Gini 0.58). Calibration is weak for applicants with thin
credit files (MV-F-01, high). The model is approved for use with a
conservative overlay for thin-file applicants until recalibration by
31 March 2027."

## 2. Validation scope and approach

<!-- Guidance: State what was and was not validated, the evidence used and how independence was ensured. Effective challenge needs expertise, independence and the standing to effect change. -->

### 2.1 Scope

In scope: [model methodology, development data, code, implementation in
the decision engine, monitoring reports]. Out of scope: [upstream
bureau data quality controls, covered by DQ review DQ-2026-07].

### 2.2 Evidence reviewed

| Evidence | Version | Provided by |
|---|---|---|
| Model development document | v4.0, YYYY-MM-DD | [@developer] |
| Development and test datasets | [snapshot IDs] | [@data-owner] |
| Model code repository | [commit] | [@developer] |
| Production scoring logs | [period] | [@ops] |

### 2.3 Independence and approach

The validation team reports to [Chief Risk Officer] and had no role in
development. Work comprised document review, replication of key results
from code and data, independent tests and interviews with developers and
users over [6] weeks.

### 2.4 Limitations of this validation

State anything that limited the work, so readers can weigh the
conclusion. *Example:* "Default outcomes for the 2025 cohort are not yet
mature, so out-of-time testing uses 2024 applications only. The vendor
did not provide the bureau score methodology; we relied on outcomes
analysis on our own portfolio instead. Developers were available for
interviews throughout, and no requested evidence was refused."

## 3. Model overview and use

<!-- Guidance: Describe purpose, users, decisions supported and materiality. Misuse beyond the intended purpose is a common source of model risk. -->

### 3.1 Purpose and intended use

[Model name] estimates the 12-month probability of default for
[applicants for unsecured loans up to 25,000 EUR]. Outputs are used for
[approval cut-offs, risk-based pricing and expected credit loss input].

### 3.2 Model description

| Attribute | Value |
|---|---|
| Method | [Logistic regression on WoE-binned variables / gradient boosted trees] |
| Inputs | [14 variables: bureau score, utilisation, months on book] |
| Output | [PD in [0, 1], mapped to 10 risk grades] |
| Development sample | [210,000 applications, 2019-01 to 2023-12] |
| Default definition | [90+ days past due within 12 months] |

### 3.3 Materiality

Materiality reflects inherent risk (complexity, assumptions, data
limits) and model exposure and purpose.

| Factor | Assessment | Rating |
|---|---|---|
| Exposure | *Example:* 1.8 bn EUR portfolio, 9,000 decisions per week | High |
| Purpose | *Example:* Credit decisions and financial reporting input | High |
| Inherent risk | *Example:* Moderate complexity, stable data | Medium |
| Overall tier | | Tier 1 |

## 4. Conceptual soundness

<!-- Guidance: Assess design choices, assumptions, qualitative judgement, data selection and developmental testing. Judge the quality and extent of the developer's evidence, not only the outcome. -->

| Area | Validator assessment | Rating |
|---|---|---|
| Choice of method | *Example:* Logistic regression appropriate for interpretable PD; alternatives considered | Satisfactory |
| Variable selection | *Example:* Economic rationale given for 13 of 14 variables; "device type" lacks rationale | Needs improvement |
| Key assumptions | *Example:* Stable relationship between bureau score and default through the cycle; partly tested | Needs improvement |
| Developmental testing | *Example:* Out-of-sample and out-of-time tests performed | Satisfactory |
| Interpretability | *Example:* Monotonic WoE bins; reason codes available | Satisfactory |
| Expert judgement and overlays | *Example:* Two manual bin merges documented | Satisfactory |

## 5. Data quality review

<!-- Guidance: Check relevance, completeness, accuracy and representativeness of data for the intended use, including any proxies. -->

| Check | Result | Comment |
|---|---|---|
| Completeness | *Example:* 2.1% missing utilisation, imputed as separate bin | Acceptable |
| Accuracy | *Example:* 500-record sample reconciled to source systems, 0 breaks | Acceptable |
| Representativeness | *Example:* Thin-file applicants 6% of sample vs 14% of current flow | Weakness, see MV-F-01 |
| Default definition consistency | *Example:* Same definition in development and production | Acceptable |
| Data lineage | *Example:* Documented in data pipeline spec v2 | Acceptable |

## 6. Outcomes analysis and back-testing

<!-- Guidance: Compare model outputs with actual outcomes on data not used for development. Report discrimination, calibration and stability, with thresholds agreed in advance. -->

### 6.1 Metrics and thresholds

Discrimination is measured with the Gini coefficient derived from the
area under the ROC curve:

$$
\text{Gini} = 2 \cdot \text{AUC} - 1
$$

Calibration per risk grade $g$ is tested with a binomial test of the
observed defaults $d_g$ among $n_g$ accounts against the predicted mean
PD $\bar{p}_g$:

$$
z_g = \frac{d_g - n_g \bar{p}_g}{\sqrt{n_g \bar{p}_g (1 - \bar{p}_g)}}
$$

Overall calibration error is summarised by the Brier score over $N$
accounts with predictions $p_i$ and outcomes $y_i \in \{0, 1\}$:

$$
\text{BS} = \frac{1}{N} \sum_{i=1}^{N} (p_i - y_i)^2
$$

| Metric | Green | Amber | Red |
|---|---|---|---|
| Gini (out-of-time) | ≥ 0.50 | 0.40 to 0.50 | < 0.40 |
| Grades failing binomial test at 95% | 0 to 1 | 2 | ≥ 3 |
| Population stability index | < 0.10 | 0.10 to 0.25 | > 0.25 |

### 6.2 Results

| Sample | N | Defaults | Gini | Brier | Grades failing | Status |
|---|---|---|---|---|---|---|
| *Example:* Development | 147,000 | 5,880 | 0.62 | 0.036 | 0 | Green |
| *Example:* Out-of-sample | 63,000 | 2,490 | 0.60 | 0.037 | 1 | Green |
| *Example:* Out-of-time 2024 | 58,400 | 2,610 | 0.58 | 0.041 | 2 | Amber |
| *Example:* Thin-file segment (OOT) | 8,100 | 610 | 0.41 | 0.071 | 3 | Red |

*Example:* For grade 7 in the out-of-time sample, $n_g = 4{,}200$,
$\bar{p}_g = 0.050$ and $d_g = 252$ give $z_g \approx 3.0$, so observed
defaults exceed prediction at the 95% level.

## 7. Benchmarking

<!-- Guidance: Compare against an alternative model, an earlier version, a vendor score or a simple challenger. Differences need explanation, not necessarily correction. -->

| Benchmark | Gini (OOT) | Comment |
|---|---|---|
| *Example:* Current production v3.2 | 0.54 | New model improves rank ordering |
| *Example:* Validator challenger (gradient boosting) | 0.61 | Small gain, lower interpretability |
| *Example:* Bureau generic score alone | 0.49 | Model adds information |

## 8. Sensitivity and stress testing

<!-- Guidance: Show how outputs change when inputs, assumptions or conditions change, and where the model breaks. -->

| Test | Change applied | Effect on portfolio mean PD | Assessment |
|---|---|---|---|
| *Example:* Input shock | Bureau score -20 points for all | +38% | Plausible, monotonic |
| *Example:* Missing data | Utilisation missing for 20% | +4% | Robust |
| *Example:* Macroeconomic stress | Unemployment +3 points via overlay | +55% | Overlay documented; within range |
| *Example:* Variable removal | Drop "device type" | Gini -0.004 | Variable not needed (see MV-F-02) |

## 9. Implementation and process verification

<!-- Guidance: Confirm that the production implementation matches the validated model and that inputs and outputs are controlled. -->

| Check | Result |
|---|---|
| *Example:* Replication of 10,000 production scores from logged inputs | 100% match to 6 decimals |
| *Example:* Version in decision engine equals validated version | Confirmed, artefact hash recorded |
| *Example:* Change control for cut-offs | Cut-off changes require committee approval; confirmed |
| *Example:* Input data feeds | Two feeds lack reconciliation alerts (MV-F-04) |

## 10. Ongoing monitoring plan

<!-- Guidance: Define what is monitored, how often, thresholds and the response when performance deviates. Link to the model monitoring plan. -->

| Metric | Frequency | Threshold | Action on breach | Owner |
|---|---|---|---|---|
| Gini on matured cohorts | Quarterly | < 0.50 amber, < 0.40 red | Investigate; red triggers revalidation | [@model-owner] |
| Calibration by grade | Quarterly | ≥ 2 grades failing | Recalibration review | [@model-owner] |
| PSI of score distribution | Monthly | > 0.10 amber, > 0.25 red | Root cause analysis | [@ml-ops] |
| Override rate | Monthly | > 8% | Review with credit policy | [@credit-policy] |
| Thin-file segment Gini | Quarterly | < 0.45 | Keep overlay; escalate | [@model-owner] |

## 11. Limitations and compensating controls

<!-- Guidance: List known limitations and how they are controlled in use. Users must be told about them. -->

| Limitation | Impact | Compensating control |
|---|---|---|
| *Example:* Under-predicts PD for thin-file applicants | Under-pricing, higher losses | +1 risk grade overlay; manual review above 10,000 EUR |
| *Example:* Development data mostly benign economy | Stress behaviour uncertain | Macroeconomic overlay; annual stress test |
| *Example:* Not validated for small business loans | Misuse risk | Inventory restriction; decision engine blocks other products |

## 12. Third-party components

<!-- Guidance: Vendor data, parameters or models still need validation evidence. Record what the vendor supplied and what you tested yourself. -->

| Component | Vendor | Evidence obtained | Validator testing | Gap |
|---|---|---|---|---|
| *Example:* Bureau generic score | [Bureau] | Vendor validation summary 2026 | Outcomes on own portfolio | Methodology proprietary |
| [Component] | [Vendor] | [Evidence] | [Tests] | [Gap] |

## 13. Findings

<!-- Guidance: Each finding has an ID, severity, evidence, required action, owner and due date. Severity reflects impact on fitness for purpose. -->

Severity scale: **Critical** (model not fit for purpose), **High**
(material weakness, condition of use required), **Medium** (weakness to
fix within [6] months), **Low** (improvement).

| ID | Area | Finding | Severity | Required action | Owner | Due | Status |
|---|---|---|---|---|---|---|---|
| *Example:* MV-F-01 | Outcomes | Poor calibration and discrimination for thin-file applicants | High | Overlay now; recalibrate with 2025 data | [@developer] | YYYY-MM-DD | Open |
| *Example:* MV-F-02 | Conceptual | "Device type" lacks economic rationale and adds no lift | Medium | Remove or justify | [@developer] | YYYY-MM-DD | Open |
| *Example:* MV-F-03 | Data | Under-representation of 2020 to 2021 period not discussed | Low | Document effect | [@developer] | YYYY-MM-DD | Open |
| *Example:* MV-F-04 | Implementation | Two input feeds lack reconciliation alerts | Medium | Add alerts | [@ml-ops] | YYYY-MM-DD | Open |

## 14. Validation conclusion and conditions

<!-- Guidance: State the conclusion clearly and the conditions under which use is acceptable. -->

Conclusion: **[Fit for purpose with conditions]**.

Conditions of use:

1. [Thin-file overlay applied from go-live, confirmed by @credit-policy.]
2. [MV-F-01 closed by YYYY-MM-DD; otherwise use for thin-file applicants
   stops.]
3. [Quarterly monitoring report submitted to the model risk committee.]

Next full validation: [YYYY-MM-DD, or earlier on material change].

## 15. Sign-off

<!-- Guidance: The model owner acknowledges findings; the committee decides on use. -->

| Role | Name | Statement | Date |
|---|---|---|---|
| Lead validator | [@name] | I performed this validation independently of development | YYYY-MM-DD |
| Head of model validation | [@name] | Conclusion endorsed | YYYY-MM-DD |
| Model owner | [@name] | Findings acknowledged; actions accepted | YYYY-MM-DD |
| Model risk committee chair | [@name] | [Decision] | YYYY-MM-DD |

## 16. Glossary and references

<!-- Guidance: Define terms and list documents used. -->

| Term | Meaning |
|---|---|
| Effective challenge | Critical analysis by objective, competent people with the standing to effect change |
| Outcomes analysis | Comparison of model outputs with actual outcomes |
| Back-testing | Outcomes analysis on historical periods not used for development |
| Benchmarking | Comparison with alternative models or estimates |
| Materiality | Importance of a model, from its exposure and purpose |
| PSI | Population stability index, a measure of distribution shift |

References: Board of Governors of the Federal Reserve System, SR 26-2
"Revised Guidance on Model Risk Management" (17 April 2026),
https://www.federalreserve.gov/supervisionreg/srletters/SR2602.pdf;
[model development document]; [model monitoring plan].

## Standard and licence note

Based on the *Supervisory Guidance on Model Risk Management* issued by
the Federal Reserve, FDIC and OCC with SR letter 26-2 (17 April 2026),
which supersedes SR letter 11-7 (4 April 2011); US government work in
the public domain. Headings follow its components of validation
(conceptual soundness, outcomes analysis, ongoing monitoring, vendor
products) in original wording; no text is reproduced. The guidance is
non-binding and aimed at banking organisations; check current
supervisory expectations for your jurisdiction. Template licence: MIT.