Model Card Template (Mitchell et al. 2019)

Model details, intended and out-of-scope uses, factors, metrics, data, disaggregated results, ethics, caveats

IA y datosCompletaModel Cards for Model Reporting (Mitchell et al., 2019)model cardActualizada 20 oct 2026

Cómo usar: Use when releasing or handing over a trained model. Fill intended use and factors first, because the factors decide how results are disaggregated. The main pitfall is a single headline metric with no per-group results or confidence intervals.

Vista previa

Model Card: [Model name] [version]

Complete model card for a trained machine learning model: what it is, what it is for and not for, how it was trained and evaluated, how it performs across groups of people and conditions, and what users must know before relying on it. The section structure follows the model card framework of Mitchell et al. (2019), Model Cards for Model Reporting. Replace every [bracketed] value and every row marked Example.

Document control

Field	Value
Document ID	[MC-MODEL-001]
Model name	[Model name]
Model version	[1.2.0]
Card version	[0.1]
Status	[Draft / In review / Approved / Published / Retired]
Owner	[@model-owner]
Classification	[Internal / Confidential / Public]
Last updated	YYYY-MM-DD
Related documents	[Datasheet for training data], [ML system design], [AI risk assessment], [Evaluation report]

Approvals

Role	Name	Decision	Date
Model owner	[@name]	[Approved / Changes requested]	YYYY-MM-DD
Responsible AI / ethics reviewer	[@name]	[Decision]	YYYY-MM-DD
Product owner	[@name]	[Decision]	YYYY-MM-DD
Privacy or legal reviewer	[@name]	[Decision]	YYYY-MM-DD

Revision history

Version	Date	Author	Change summary
0.1	YYYY-MM-DD	[@name]	First draft for review
0.2	YYYY-MM-DD	[@name]	Example: Added results for speakers over 65 after fairness review
1.0	YYYY-MM-DD	[@name]	Example: Published with model 1.2.0

Model details
Intended use
Factors
Metrics
Evaluation data
Training data
Quantitative analyses
Ethical considerations
Caveats and recommendations
Versioning, maintenance and contact
References and glossary

1. Model details

1.1 Summary

[Two or three sentences in plain language. Example: "A gradient-boosted classifier that estimates the probability that an incoming support ticket is urgent, so that agents can sort their queue. It reads the ticket text and metadata only; it does not read customer account data."]

1.2 Identification and provenance

Attribute	Value
Developer / organisation	[Team, organisation]
Model date	YYYY-MM-DD
Model version	[1.2.0]
Artefact location and hash	[Registry URI], `sha256:[hash]`
Model type	Example: Gradient-boosted decision trees over TF-IDF and metadata features
Architecture details	Example: 600 trees, max depth 8, learning rate 0.05
Base or foundation model	[None / Name and version of the pre-trained model it was fine-tuned from]
Input	Example: Ticket subject and body (English, up to 4 000 characters), channel, product area
Output	Example: Probability of "urgent" in [0, 1] and a binary label at threshold 0.62
Training framework	[Framework and version]
Licence	[Internal use only / Open licence name]
Citation	[How to cite this model, if published]

1.3 Training approach

[Describe the algorithm, objective, important hyperparameters, fairness constraints or interventions, and how the decision threshold was chosen. Example: "Trained with log loss on 18 months of labelled tickets; class weights balance the 9% urgent rate; threshold chosen to give 90% recall on the validation set."]

2. Intended use

2.1 Primary intended uses

ID	Use	Decision supported	Human in the loop
IU-01	Example: Rank the open ticket queue by urgency for support agents	Order in which agents read tickets	Yes, agent decides what to work on
IU-02	Example: Trigger a page to the on-call lead when 5 or more urgent tickets arrive in 10 minutes	Whether to escalate	Yes, lead confirms incident
[IU-nn]	[Use]	[Decision]	[Yes / No, describe]

2.2 Primary intended users

[Who operates the model output, with what training. Example: "Tier-1 and tier-2 support agents trained on the triage runbook; support leads."]

2.3 Out-of-scope uses

ID	Use that is out of scope	Why
OOS-01	Example: Deciding refunds, credits or account closure	Not trained or evaluated for these outcomes
OOS-02	Example: Tickets in languages other than English	Performance drops sharply (see 7.3)
OOS-03	Example: Evaluating agent performance	Scores reflect ticket text, not agent behaviour
[OOS-nn]	[Use]	[Reason]

3. Factors

3.1 Relevant factors

Factor type	Factor	Why it may affect performance	Evaluated
Group	Example: Customer writing in a second language	Phrasing and vocabulary differ from training majority	Yes
Group	Example: Customer age band (self-declared, sample only)	Writing style; accessibility needs	Partly, sample of 2 000
Instrumentation	Example: Channel: email, web form, chat transcript	Length and structure of text differ	Yes
Environment	Example: Incident days versus normal days	Ticket mix shifts, many near-duplicates	Yes
[Group / Instrumentation / Environment]	[Factor]	[Reason]	[Yes / No / Partly]

3.2 Evaluation factors

[Which factors section 7 reports on and why these were chosen. Explain any relevant factor that was not evaluated, for example because labels for that group are not lawfully collectable, and what that means for users.]

4. Metrics

4.1 Performance measures

Metric	Definition	Why it matters here
Recall (urgent)	$\frac{TP}{TP + FN}$	A missed urgent ticket delays a customer in trouble
Precision (urgent)	$\frac{TP}{TP + FP}$	False alarms waste agent attention and erode trust
False negative rate	$\frac{FN}{TP + FN}$	Compared across groups to detect unequal harm
AUROC	Area under the ROC curve	Threshold-free ranking quality
Calibration error	Expected calibration error over 10 bins	Scores are shown to agents as probabilities
[Metric]	[Definition]	[Reason]

4.2 Decision threshold

[Threshold value, how it was chosen, and the trade-off it represents. Example: "0.62, chosen on validation data to reach recall of at least 0.90 while keeping precision above 0.55."]

4.3 Variation approaches

[How uncertainty is shown. Example: "95% bootstrap confidence intervals from 1 000 resamples of the test set, stratified by channel. Group gaps are reported with their own intervals."]

5. Evaluation data

Attribute	Value
Dataset(s)	Example: Held-out tickets from YYYY-MM-DD to YYYY-MM-DD, never used in training
Size	Example: 24 000 tickets, 2 180 labelled urgent
Motivation	Example: Most recent period, closest to production mix
Labelling	Example: Two agents labelled independently; disagreements resolved by a lead; Cohen's kappa 0.81
Preprocessing	Example: Same pipeline as training: lower-casing, signature removal, PII masking
Known gaps	Example: Only 3% of tickets from chat; no tickets from the new Japanese site
Datasheet	[Link to datasheet for this dataset]

6. Training data

Attribute	Value
Source	Example: Support tickets from all public channels, YYYY-MM to YYYY-MM
Size	Example: 310 000 tickets, 9% urgent
Collection and consent basis	[Legal basis or consent mechanism; reference to privacy notice]
Personal data handling	Example: Names, emails and phone numbers masked before training
Distribution over factors	Example: Email 61%, web form 33%, chat 6%; second-language writers estimated 14%
Exclusions	Example: Tickets under 10 words; spam; internal test tickets
Datasheet	[Link to datasheet for this dataset]

7. Quantitative analyses

7.1 Overall results

Metric	Value	95% CI	Target	Met
Recall (urgent)	Example: 0.91	0.89 to 0.92	at least 0.90	Yes
Precision (urgent)	Example: 0.58	0.56 to 0.60	at least 0.55	Yes
AUROC	Example: 0.94	0.93 to 0.95	at least 0.92	Yes
Expected calibration error	Example: 0.031	0.025 to 0.037	at most 0.05	Yes

7.2 Disaggregated results by single factor

Factor	Group	n	Recall	Precision	FNR	Gap vs overall FNR
Channel	Email	Example: 14 600	0.92	0.60	0.08	-0.01
Channel	Web form	Example: 8 000	0.91	0.57	0.09	0.00
Channel	Chat	Example: 1 400	0.82	0.49	0.18	+0.09 (above tolerance)
Writer	Second language	Example: 3 300	0.86	0.55	0.14	+0.05
Day type	Incident day	Example: 2 100	0.95	0.71	0.05	-0.04
[Factor]	[Group]	[n]	[value]	[value]	[value]	[value]

The fairness tolerance used in this card is: [Example: no group's false negative rate may exceed the overall rate by more than 0.05 absolute].

7.3 Intersectional results

Group combination	n	Recall	FNR	Note
Example: Chat and second language	260	0.74	0.26	Wide interval (0.66 to 0.81); treat with caution
Example: Non-English tickets (out of scope)	410	0.52	0.48	Confirms OOS-02
[Combination]	[n]	[value]	[value]	[Note]

7.4 Interpretation

[What the numbers mean for users. Example: "The model misses roughly one in five urgent chat tickets. Chat tickets must not rely on the urgency score alone until the chat-specific model in roadmap item RM-14 ships."]

8. Ethical considerations

Topic	Consideration	Mitigation	Residual concern
Sensitive data	Example: Tickets may mention health or financial hardship	PII masking; no sensitive attributes as features	Free text can still reveal sensitive facts
Human life and safety	Example: Urgent tickets may involve service outages for hospitals	Hospital accounts always routed to priority queue regardless of score	None known
Unequal performance	Example: Higher miss rate for chat and second-language writers	Group-specific monitoring; agent guidance in 9	Gap remains until RM-14
Automation bias	Example: Agents may skip low-score tickets	Queue shows age as well as score; maximum wait rule	Monitored monthly
[Topic]	[Consideration]	[Mitigation]	[Residual]

9. Caveats and recommendations

9.1 Caveats

[Example: Not evaluated on tickets longer than 4 000 characters; text is truncated.]
[Example: Performance after major product launches is unknown until labels arrive, usually 14 days.]
[Example: Age band results come from a self-declared sample and may not represent all customers.]

9.2 Recommendations for users

[Example: Read every chat ticket older than 15 minutes regardless of score.]
[Example: Report misclassified tickets with the "wrong urgency" button; reports feed the next retraining.]
[Example: Do not display the raw probability to customers.]

10. Versioning, maintenance and contact

10.1 Version lineage

Model version	Date	Change	Card version
Example: 1.0.0	YYYY-MM-DD	First production model	1.0
Example: 1.2.0	YYYY-MM-DD	Retrained on 6 more months; chat features added	2.0
[Version]	YYYY-MM-DD	[Change]	[Card version]

10.2 Monitoring and retraining

[What is monitored in production, thresholds that trigger review, and the retraining cadence. Example: "Weekly recall per channel from agent feedback; retrain quarterly or when recall drops below 0.87 for two consecutive weeks."]

10.3 Contact

Purpose	Contact
Questions about the model	[@model-owner], [team channel]
Report a harmful or incorrect output	[Issue tracker or form]
Responsible AI review	[@reviewer]

11. References and glossary

11.1 References

Reference	Use
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., Gebru, T. (2019). Model Cards for Model Reporting. Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT* '19).	Card structure
Gebru, T. et al. (2021). Datasheets for Datasets. Communications of the ACM 64(12).	Data documentation linked from 5 and 6
[Internal evaluation report]	Source of section 7 numbers

11.2 Glossary

Term	Meaning
TP, FP, FN	True positives, false positives, false negatives for the positive class
FNR	False negative rate: share of real positives the model missed
Disaggregated result	A metric computed separately for one group or condition
Intersectional result	A metric for a combination of groups, such as channel and language
Calibration	How closely predicted probabilities match observed frequencies

Standard and licence note

Structure aligned with the model card framework described in Mitchell et al. (2019), Model Cards for Model Reporting (FAT* '19), covering its sections on model details, intended use, factors, metrics, evaluation and training data, quantitative analyses, ethical considerations, and caveats and recommendations. This template is original text written for this collection and reproduces no text from the paper; read the paper for the authors' full guidance. Template licence: MIT.

Plantillas relacionadas

IA y datos