# Model Card: [Model name] [version]

> Complete model card for a trained machine learning model: what it is,
> what it is for and not for, how it was trained and evaluated, how it
> performs across groups of people and conditions, and what users must
> know before relying on it. The section structure follows the model card
> framework of Mitchell et al. (2019), *Model Cards for Model Reporting*.
> Replace every `[bracketed]` value and every row marked *Example*.

<!--
Guidance: How to use this template
- Write for two readers at once: an engineer deciding whether to integrate
  the model, and a non-specialist (product, legal, affected user) deciding
  whether it is appropriate. Plain words first, detail in tables.
- Fill sections 2 and 3 (intended use, factors) before the numbers. The
  factors you name decide how section 7 is disaggregated.
- Report results per group and per condition, with uncertainty. A single
  headline accuracy hides exactly the failures a model card exists to show.
- Publish one card per released model version. Never edit a released card
  silently; add a revision and bump the card version.
-->

## Document control

<!-- Guidance: Identify the card and the exact model artefact it describes. A card that cannot be tied to a model hash is not auditable. -->

| Field | Value |
|---|---|
| Document ID | [MC-MODEL-001] |
| Model name | [Model name] |
| Model version | [1.2.0] |
| Card version | [0.1] |
| Status | [Draft / In review / Approved / Published / Retired] |
| Owner | [@model-owner] |
| Classification | [Internal / Confidential / Public] |
| Last updated | YYYY-MM-DD |
| Related documents | [Datasheet for training data], [ML system design], [AI risk assessment], [Evaluation report] |

### Approvals

| Role | Name | Decision | Date |
|---|---|---|---|
| Model owner | [@name] | [Approved / Changes requested] | YYYY-MM-DD |
| Responsible AI / ethics reviewer | [@name] | [Decision] | YYYY-MM-DD |
| Product owner | [@name] | [Decision] | YYYY-MM-DD |
| Privacy or legal reviewer | [@name] | [Decision] | YYYY-MM-DD |

## Revision history

<!-- Guidance: Record changes to the model or to what the card claims about it. -->

| Version | Date | Author | Change summary |
|---|---|---|---|
| 0.1 | YYYY-MM-DD | [@name] | First draft for review |
| 0.2 | YYYY-MM-DD | [@name] | *Example:* Added results for speakers over 65 after fairness review |
| 1.0 | YYYY-MM-DD | [@name] | *Example:* Published with model 1.2.0 |

## Contents

<!-- Guidance: Keep aligned with the headings below. -->

1. [Model details](#1-model-details)
2. [Intended use](#2-intended-use)
3. [Factors](#3-factors)
4. [Metrics](#4-metrics)
5. [Evaluation data](#5-evaluation-data)
6. [Training data](#6-training-data)
7. [Quantitative analyses](#7-quantitative-analyses)
8. [Ethical considerations](#8-ethical-considerations)
9. [Caveats and recommendations](#9-caveats-and-recommendations)
10. [Versioning, maintenance and contact](#10-versioning-maintenance-and-contact)
11. [References and glossary](#11-references-and-glossary)

## 1. Model details

<!-- Guidance: Basic facts a reader needs to identify the model and judge its provenance. Answer: who built it, when, what kind of model, under what licence, and where to send questions. -->

### 1.1 Summary

[Two or three sentences in plain language. Example: "A gradient-boosted
classifier that estimates the probability that an incoming support ticket
is urgent, so that agents can sort their queue. It reads the ticket text
and metadata only; it does not read customer account data."]

### 1.2 Identification and provenance

| Attribute | Value |
|---|---|
| Developer / organisation | [Team, organisation] |
| Model date | YYYY-MM-DD |
| Model version | [1.2.0] |
| Artefact location and hash | [Registry URI], `sha256:[hash]` |
| Model type | *Example:* Gradient-boosted decision trees over TF-IDF and metadata features |
| Architecture details | *Example:* 600 trees, max depth 8, learning rate 0.05 |
| Base or foundation model | [None / Name and version of the pre-trained model it was fine-tuned from] |
| Input | *Example:* Ticket subject and body (English, up to 4 000 characters), channel, product area |
| Output | *Example:* Probability of "urgent" in [0, 1] and a binary label at threshold 0.62 |
| Training framework | [Framework and version] |
| Licence | [Internal use only / Open licence name] |
| Citation | [How to cite this model, if published] |

### 1.3 Training approach

[Describe the algorithm, objective, important hyperparameters, fairness
constraints or interventions, and how the decision threshold was chosen.
Example: "Trained with log loss on 18 months of labelled tickets; class
weights balance the 9% urgent rate; threshold chosen to give 90% recall on
the validation set."]

## 2. Intended use

<!-- Guidance: State the uses the model was designed and tested for, and name the uses it must not be put to. Out-of-scope uses are as important as in-scope ones; be specific. -->

### 2.1 Primary intended uses

| ID | Use | Decision supported | Human in the loop |
|---|---|---|---|
| IU-01 | *Example:* Rank the open ticket queue by urgency for support agents | Order in which agents read tickets | Yes, agent decides what to work on |
| IU-02 | *Example:* Trigger a page to the on-call lead when 5 or more urgent tickets arrive in 10 minutes | Whether to escalate | Yes, lead confirms incident |
| [IU-nn] | [Use] | [Decision] | [Yes / No, describe] |

### 2.2 Primary intended users

[Who operates the model output, with what training. Example: "Tier-1 and
tier-2 support agents trained on the triage runbook; support leads."]

### 2.3 Out-of-scope uses

| ID | Use that is out of scope | Why |
|---|---|---|
| OOS-01 | *Example:* Deciding refunds, credits or account closure | Not trained or evaluated for these outcomes |
| OOS-02 | *Example:* Tickets in languages other than English | Performance drops sharply (see 7.3) |
| OOS-03 | *Example:* Evaluating agent performance | Scores reflect ticket text, not agent behaviour |
| [OOS-nn] | [Use] | [Reason] |

## 3. Factors

<!-- Guidance: Factors are the characteristics that can change model performance: groups of people, instruments, and environments. List the relevant ones, then state which you actually evaluated and why the others were not. -->

### 3.1 Relevant factors

| Factor type | Factor | Why it may affect performance | Evaluated |
|---|---|---|---|
| Group | *Example:* Customer writing in a second language | Phrasing and vocabulary differ from training majority | Yes |
| Group | *Example:* Customer age band (self-declared, sample only) | Writing style; accessibility needs | Partly, sample of 2 000 |
| Instrumentation | *Example:* Channel: email, web form, chat transcript | Length and structure of text differ | Yes |
| Environment | *Example:* Incident days versus normal days | Ticket mix shifts, many near-duplicates | Yes |
| [Group / Instrumentation / Environment] | [Factor] | [Reason] | [Yes / No / Partly] |

### 3.2 Evaluation factors

[Which factors section 7 reports on and why these were chosen. Explain
any relevant factor that was not evaluated, for example because labels
for that group are not lawfully collectable, and what that means for
users.]

## 4. Metrics

<!-- Guidance: Name the measures, the decision threshold and how uncertainty is estimated. Explain why each metric matters for the intended use, including which error type is worse. -->

### 4.1 Performance measures

| Metric | Definition | Why it matters here |
|---|---|---|
| Recall (urgent) | $\frac{TP}{TP + FN}$ | A missed urgent ticket delays a customer in trouble |
| Precision (urgent) | $\frac{TP}{TP + FP}$ | False alarms waste agent attention and erode trust |
| False negative rate | $\frac{FN}{TP + FN}$ | Compared across groups to detect unequal harm |
| AUROC | Area under the ROC curve | Threshold-free ranking quality |
| Calibration error | Expected calibration error over 10 bins | Scores are shown to agents as probabilities |
| [Metric] | [Definition] | [Reason] |

### 4.2 Decision threshold

[Threshold value, how it was chosen, and the trade-off it represents.
Example: "0.62, chosen on validation data to reach recall of at least 0.90
while keeping precision above 0.55."]

### 4.3 Variation approaches

[How uncertainty is shown. Example: "95% bootstrap confidence intervals
from 1 000 resamples of the test set, stratified by channel. Group gaps
are reported with their own intervals."]

## 5. Evaluation data

<!-- Guidance: Describe the data used for the results in section 7: where it came from, why it was chosen, how it was preprocessed, and how it differs from production traffic. -->

| Attribute | Value |
|---|---|
| Dataset(s) | *Example:* Held-out tickets from YYYY-MM-DD to YYYY-MM-DD, never used in training |
| Size | *Example:* 24 000 tickets, 2 180 labelled urgent |
| Motivation | *Example:* Most recent period, closest to production mix |
| Labelling | *Example:* Two agents labelled independently; disagreements resolved by a lead; Cohen's kappa 0.81 |
| Preprocessing | *Example:* Same pipeline as training: lower-casing, signature removal, PII masking |
| Known gaps | *Example:* Only 3% of tickets from chat; no tickets from the new Japanese site |
| Datasheet | [Link to datasheet for this dataset] |

## 6. Training data

<!-- Guidance: Give as much detail about training data as you can share. If the data cannot be disclosed, describe its distribution over the factors in section 3 so readers can judge coverage. -->

| Attribute | Value |
|---|---|
| Source | *Example:* Support tickets from all public channels, YYYY-MM to YYYY-MM |
| Size | *Example:* 310 000 tickets, 9% urgent |
| Collection and consent basis | [Legal basis or consent mechanism; reference to privacy notice] |
| Personal data handling | *Example:* Names, emails and phone numbers masked before training |
| Distribution over factors | *Example:* Email 61%, web form 33%, chat 6%; second-language writers estimated 14% |
| Exclusions | *Example:* Tickets under 10 words; spam; internal test tickets |
| Datasheet | [Link to datasheet for this dataset] |

## 7. Quantitative analyses

<!-- Guidance: Show unitary results (one factor at a time) and intersectional results (combinations) where sample sizes allow. Always include n and confidence intervals. Highlight gaps larger than your agreed tolerance. -->

### 7.1 Overall results

| Metric | Value | 95% CI | Target | Met |
|---|---|---|---|---|
| Recall (urgent) | *Example:* 0.91 | 0.89 to 0.92 | at least 0.90 | Yes |
| Precision (urgent) | *Example:* 0.58 | 0.56 to 0.60 | at least 0.55 | Yes |
| AUROC | *Example:* 0.94 | 0.93 to 0.95 | at least 0.92 | Yes |
| Expected calibration error | *Example:* 0.031 | 0.025 to 0.037 | at most 0.05 | Yes |

### 7.2 Disaggregated results by single factor

| Factor | Group | n | Recall | Precision | FNR | Gap vs overall FNR |
|---|---|---|---|---|---|---|
| Channel | Email | *Example:* 14 600 | 0.92 | 0.60 | 0.08 | -0.01 |
| Channel | Web form | *Example:* 8 000 | 0.91 | 0.57 | 0.09 | 0.00 |
| Channel | Chat | *Example:* 1 400 | 0.82 | 0.49 | 0.18 | +0.09 (above tolerance) |
| Writer | Second language | *Example:* 3 300 | 0.86 | 0.55 | 0.14 | +0.05 |
| Day type | Incident day | *Example:* 2 100 | 0.95 | 0.71 | 0.05 | -0.04 |
| [Factor] | [Group] | [n] | [value] | [value] | [value] | [value] |

The fairness tolerance used in this card is: [Example: no group's false
negative rate may exceed the overall rate by more than 0.05 absolute].

### 7.3 Intersectional results

| Group combination | n | Recall | FNR | Note |
|---|---|---|---|---|
| *Example:* Chat and second language | 260 | 0.74 | 0.26 | Wide interval (0.66 to 0.81); treat with caution |
| *Example:* Non-English tickets (out of scope) | 410 | 0.52 | 0.48 | Confirms OOS-02 |
| [Combination] | [n] | [value] | [value] | [Note] |

### 7.4 Interpretation

[What the numbers mean for users. Example: "The model misses roughly one
in five urgent chat tickets. Chat tickets must not rely on the urgency
score alone until the chat-specific model in roadmap item RM-14 ships."]

## 8. Ethical considerations

<!-- Guidance: Record the ethical analysis honestly: sensitive data, effects on people, risks and mitigations, and contested uses. Link to the AI risk assessment for the full register. -->

| Topic | Consideration | Mitigation | Residual concern |
|---|---|---|---|
| Sensitive data | *Example:* Tickets may mention health or financial hardship | PII masking; no sensitive attributes as features | Free text can still reveal sensitive facts |
| Human life and safety | *Example:* Urgent tickets may involve service outages for hospitals | Hospital accounts always routed to priority queue regardless of score | None known |
| Unequal performance | *Example:* Higher miss rate for chat and second-language writers | Group-specific monitoring; agent guidance in 9 | Gap remains until RM-14 |
| Automation bias | *Example:* Agents may skip low-score tickets | Queue shows age as well as score; maximum wait rule | Monitored monthly |
| [Topic] | [Consideration] | [Mitigation] | [Residual] |

## 9. Caveats and recommendations

<!-- Guidance: Tell the user what was not tested, the conditions under which the model should not be trusted, and the practical steps they should take. -->

### 9.1 Caveats

- [Example: Not evaluated on tickets longer than 4 000 characters; text is truncated.]
- [Example: Performance after major product launches is unknown until labels arrive, usually 14 days.]
- [Example: Age band results come from a self-declared sample and may not represent all customers.]

### 9.2 Recommendations for users

- [Example: Read every chat ticket older than 15 minutes regardless of score.]
- [Example: Report misclassified tickets with the "wrong urgency" button; reports feed the next retraining.]
- [Example: Do not display the raw probability to customers.]

## 10. Versioning, maintenance and contact

<!-- Guidance: Readers need to know whether this card is current, how the model is kept healthy, and whom to contact about problems. -->

### 10.1 Version lineage

| Model version | Date | Change | Card version |
|---|---|---|---|
| *Example:* 1.0.0 | YYYY-MM-DD | First production model | 1.0 |
| *Example:* 1.2.0 | YYYY-MM-DD | Retrained on 6 more months; chat features added | 2.0 |
| [Version] | YYYY-MM-DD | [Change] | [Card version] |

### 10.2 Monitoring and retraining

[What is monitored in production, thresholds that trigger review, and the
retraining cadence. Example: "Weekly recall per channel from agent
feedback; retrain quarterly or when recall drops below 0.87 for two
consecutive weeks."]

### 10.3 Contact

| Purpose | Contact |
|---|---|
| Questions about the model | [@model-owner], [team channel] |
| Report a harmful or incorrect output | [Issue tracker or form] |
| Responsible AI review | [@reviewer] |

## 11. References and glossary

<!-- Guidance: Cite the method and any documents the card relies on. -->

### 11.1 References

| Reference | Use |
|---|---|
| Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., Gebru, T. (2019). *Model Cards for Model Reporting*. Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT* '19). | Card structure |
| Gebru, T. et al. (2021). *Datasheets for Datasets*. Communications of the ACM 64(12). | Data documentation linked from 5 and 6 |
| [Internal evaluation report] | Source of section 7 numbers |

### 11.2 Glossary

| Term | Meaning |
|---|---|
| TP, FP, FN | True positives, false positives, false negatives for the positive class |
| FNR | False negative rate: share of real positives the model missed |
| Disaggregated result | A metric computed separately for one group or condition |
| Intersectional result | A metric for a combination of groups, such as channel and language |
| Calibration | How closely predicted probabilities match observed frequencies |

## Standard and licence note

Structure aligned with the model card framework described in Mitchell et
al. (2019), *Model Cards for Model Reporting* (FAT* '19), covering its
sections on model details, intended use, factors, metrics, evaluation and
training data, quantitative analyses, ethical considerations, and caveats
and recommendations. This template is original text written for this
collection and reproduces no text from the paper; read the paper for the
authors' full guidance. Template licence: MIT.