# AI System Card: [Product or system name] [version]

> Public or customer-facing documentation for a deployed AI product at the
> system level: not one model, but the whole system of models, retrieval,
> tools, filters and human processes that users actually interact with. It
> covers components, intended and prohibited uses, capabilities and
> limitations, evaluation results, red-teaming, safeguards, data handling,
> human oversight, feedback channels and change history. Based on common
> industry practice for system cards, building on model cards. Replace
> every `[bracketed]` value and every row marked *Example*.

<!--
Guidance: How to use this template
- Write for the people who use, buy, audit or are affected by the system.
  Plain language first; technical detail in tables and links.
- Report results for the system as deployed, with its safeguards switched
  on, and say so. Model-only results belong in each [Model card].
- Be specific about limitations. A card that lists no limitations is not
  credible.
- Publish a new version of the card with every material change, and keep
  the changelog (section 13) honest.
- Related documents: [Model card] for each model, [Datasheet] for key
  datasets, [AI risk assessment], [AI usage policy] for acceptable use.
-->

## Document control

<!-- Guidance: Identify exactly which release of the system this card describes and who approved publication. -->

| Field | Value |
|---|---|
| Document ID | [SYSCARD-001] |
| System name | [Name] |
| System version / release | [2.3.0, released YYYY-MM-DD] |
| Card version | [1.0] |
| Status | [Draft / Published / Superseded] |
| Audience | [Public / Customers / Internal] |
| Owner | [@product-owner] |
| Contributors | [@ml-lead], [@safety-lead], [@privacy-lead] |
| Last updated | YYYY-MM-DD |
| Contact | [ai-feedback@example.com] |

### Approvals

| Role | Name | Decision | Date |
|---|---|---|---|
| Product owner | [@name] | [Approve for publication] | YYYY-MM-DD |
| Safety / responsible AI lead | [@name] | [Decision] | YYYY-MM-DD |
| Legal / privacy | [@name] | [Decision] | YYYY-MM-DD |
| Communications | [@name] | [Decision] | YYYY-MM-DD |

## Revision history

<!-- Guidance: Track changes to the card itself; changes to the system go in section 13. -->

| Card version | Date | Author | Change summary |
|---|---|---|---|
| 0.1 | YYYY-MM-DD | [@name] | First draft |
| 1.0 | YYYY-MM-DD | [@name] | *Example:* Published with system release 2.3.0 |

## Contents

<!-- Guidance: Keep aligned with the headings below. -->

1. [System overview](#1-system-overview)
2. [Components and models](#2-components-and-models)
3. [Intended uses](#3-intended-uses)
4. [Prohibited and out-of-scope uses](#4-prohibited-and-out-of-scope-uses)
5. [Capabilities and limitations](#5-capabilities-and-limitations)
6. [Evaluations](#6-evaluations)
7. [Red-teaming summary](#7-red-teaming-summary)
8. [Safeguards and mitigations](#8-safeguards-and-mitigations)
9. [Data handling and privacy](#9-data-handling-and-privacy)
10. [Human oversight](#10-human-oversight)
11. [Incidents and feedback](#11-incidents-and-feedback)
12. [Versioning and support](#12-versioning-and-support)
13. [Changelog](#13-changelog)

## 1. System overview

<!-- Guidance: In one or two paragraphs: what the system does, for whom, where it runs, and what a user can and cannot expect. -->

[Plain-language description.] *Example:* Helpdesk Assistant is an AI chat
assistant embedded in our customer support portal. It answers questions
about our products using our public documentation, can look up the status
of a customer's own orders, and hands the conversation to a human agent
when it cannot help. It is available in English, French and German to
logged-in customers in the EU and UK.

| Item | Value |
|---|---|
| Deployment surface | *Example:* Web portal chat widget, mobile app |
| Users | *Example:* Logged-in customers aged 18+ |
| Availability | *Example:* 24/7; human hand-off 08:00 to 20:00 CET |
| Interaction disclosure | *Example:* Users are told at the start that they are chatting with an AI system |
| Regulatory classification | *Example:* Limited-risk chatbot (transparency obligations) per internal legal review |

## 2. Components and models

<!-- Guidance: List every model and major component, including filters and retrieval. Users of the card should be able to see which parts are third-party. -->

```mermaid
flowchart LR
  U[User] --> IF["Input safety filter"]
  IF --> O["Orchestrator"]
  O --> R["Retrieval over help docs"]
  O --> T["Order status tool"]
  O --> L["Language model"]
  L --> OF["Output safety filter"]
  OF --> U
  O --> H["Human hand-off"]
```

| Component | Type | Provider | Version | Role | Model card |
|---|---|---|---|---|---|
| *Example:* Language model | LLM | [Vendor] | [model-id, date] | Generates answers | [Link] |
| *Example:* Embedding model | Embeddings | In-house | v4 | Retrieval | [Link] |
| *Example:* Input safety classifier | Classifier | [Vendor] | [v] | Blocks abuse, self-harm, injection | [Link] |
| *Example:* Output filter | Rules + classifier | In-house | v2 | PII and policy check | [Link] |
| *Example:* Order status tool | API | In-house | v7 | Read-only lookup of the user's own orders | n/a |

## 3. Intended uses

<!-- Guidance: Describe the tasks the system was designed and evaluated for, and the users it was designed for. -->

| ID | Intended use | Users | Notes |
|---|---|---|---|
| IU-01 | *Example:* Answer product and how-to questions from documentation | Customers | Cites sources |
| IU-02 | *Example:* Report the status of the user's own orders | Logged-in customers | Read-only |
| IU-03 | *Example:* Route complex issues to a human agent with a summary | Customers, agents | |

## 4. Prohibited and out-of-scope uses

<!-- Guidance: List uses that are forbidden by policy and uses that are simply not supported, and how each is prevented. -->

| ID | Use | Status | How it is prevented |
|---|---|---|---|
| PU-01 | *Example:* Medical, legal or financial advice | Out of scope | Topic classifier; refusal with referral text |
| PU-02 | *Example:* Decisions about refunds, credit or eligibility | Out of scope | No tools for these actions; hand-off |
| PU-03 | *Example:* Use by children under 18 | Not supported | Account age requirement |
| PU-04 | *Example:* Harassment, generating malware, impersonation | Prohibited | Acceptable use policy; filters; account action |
| PU-05 | *Example:* Accessing another person's order data | Prohibited | Tool scoped to the authenticated account |

## 5. Capabilities and limitations

<!-- Guidance: Be concrete. Give examples of what it does well and where it fails, including languages, topics and edge cases. -->

### 5.1 Capabilities

- [*Example:* Answers 82% of documentation questions correctly with a
  citation, measured on the evaluation set in 6.1.]
- [*Example:* Understands follow-up questions within one conversation.]
- [*Example:* Hands off to a human in under 10 seconds when asked.]

### 5.2 Known limitations

| ID | Limitation | Effect on users | Mitigation |
|---|---|---|---|
| LIM-01 | *Example:* Can state outdated information if documentation is stale | Wrong instructions | Sources shown with dates |
| LIM-02 | *Example:* Lower accuracy in German (74%) than English (85%) | More hand-offs | Improvement planned in 2.4 |
| LIM-03 | *Example:* Cannot see account details beyond order status | Cannot answer billing questions | Hand-off |
| LIM-04 | *Example:* May produce confident but wrong answers (hallucinations) | Misleading answers | Grounding check; feedback button |
| LIM-05 | *Example:* Not evaluated for accessibility tools beyond screen readers | Unknown quality | Planned testing |

## 6. Evaluations

<!-- Guidance: Report capability, safety and fairness evaluations of the deployed system. Give dataset, metric, result, threshold and date for each, and note uncertainty. -->

### 6.1 Capability

| Evaluation | Dataset | Metric | Result | Threshold | Date |
|---|---|---|---|---|---|
| *Example:* Documentation Q&A | 600 real anonymised questions | Correct and cited (human graded) | 0.82 (95% CI 0.79 to 0.85) | >= 0.80 | YYYY-MM-DD |
| *Example:* Order status | 200 scripted conversations | Task success | 0.97 | >= 0.95 | YYYY-MM-DD |
| *Example:* Hand-off quality | 150 hand-offs | Agent rating of summary (1 to 5) | 4.3 | >= 4.0 | YYYY-MM-DD |

### 6.2 Safety

| Evaluation | Dataset | Metric | Result | Threshold |
|---|---|---|---|---|
| *Example:* Harmful content requests | 1,000 adversarial prompts | Policy-violating responses | 0.3% | <= 0.5% |
| *Example:* Prompt injection (direct and via documents) | 300 cases | Successful injections | 1.0% | <= 2% |
| *Example:* Personal data leakage | 200 probing conversations | Leaks of other users' data | 0 | 0 |
| *Example:* Self-harm escalation | 120 cases | Correct crisis response and hand-off | 100% | 100% |

### 6.3 Fairness and bias

| Evaluation | Groups compared | Metric | Result | Gap threshold |
|---|---|---|---|---|
| *Example:* Q&A accuracy by language | EN / FR / DE | Correct and cited | 0.85 / 0.80 / 0.74 | 10 points: **DE fails** |
| *Example:* Refusal rate by dialect | Standard vs regional phrasing | Unwarranted refusals | 2.1% vs 3.4% | 2 points: pass |
| *Example:* Tone by inferred gender of name | Paired prompts | Rated politeness | No significant difference | p >= 0.05 |

### 6.4 Methodology and caveats

<!-- Guidance: Readers need to know how much weight to put on the numbers. State how data was sampled, who graded it, and what the evaluations do not cover. -->

- **Sampling:** [*Example:* questions drawn at random from three months of
  real traffic, stratified by language and topic, personal data removed
  before grading.]
- **Grading:** [*Example:* two trained graders per item using a written
  rubric; disagreements resolved by a third; inter-rater agreement
  (Cohen's kappa) 0.81.]
- **Configuration:** [*Example:* all results measured on the production
  configuration with every safeguard in section 8 enabled.]
- **Not covered:** [*Example:* voice input, languages other than EN, FR and
  DE, conversations longer than 30 turns.]
- **Uncertainty:** [*Example:* intervals are 95% Wilson intervals; small
  subgroups (under 100 items) are reported but not used for release gates.]

## 7. Red-teaming summary

<!-- Guidance: Summarise who tested, how, what they found and what was fixed. Do not publish working attack strings. -->

| Item | Detail |
|---|---|
| Teams | *Example:* Internal red team (6 people) and an external firm |
| Period | YYYY-MM-DD to YYYY-MM-DD |
| Scope | *Example:* Jailbreaks, injection through documents, data exfiltration, harmful advice, impersonation |
| Effort | *Example:* 140 person-hours, 2,300 attempts |

| ID | Finding | Severity | Status |
|---|---|---|---|
| RT-01 | *Example:* Role-play prompts bypassed the medical-advice refusal in French | High | Fixed in 2.3.0; regression test added |
| RT-02 | *Example:* Markdown links could carry query strings with conversation text | Medium | Fixed: links restricted to allow-listed domains |
| RT-03 | *Example:* Long conversations degraded refusal consistency | Medium | Mitigated: context limit and periodic policy reminder |

## 8. Safeguards and mitigations

<!-- Guidance: List the controls in layers: before the model, in the model, after the model, and in operations. -->

| Layer | Safeguard | Addresses |
|---|---|---|
| Input | *Example:* Abuse, self-harm and injection classifiers | PU-04, RT-01 |
| Model | *Example:* System instructions with refusal policy; temperature 0.2 | LIM-04 |
| Retrieval | *Example:* Only approved public documentation indexed | LIM-01 |
| Tools | *Example:* Read-only, scoped to the signed-in user | PU-05 |
| Output | *Example:* PII filter, link allow-list, grounding check | RT-02 |
| Operations | *Example:* Rate limits, abuse monitoring, kill switch | PU-04 |
| User interface | *Example:* AI disclosure, source citations, feedback buttons, "talk to a person" always visible | Over-reliance |

## 9. Data handling and privacy

<!-- Guidance: Explain what data the system processes, why, where, for how long, and whether it is used for training. -->

| Data | Purpose | Storage location | Retention | Used for training? |
|---|---|---|---|---|
| *Example:* Conversation text | Answering, quality review | EU region | 30 days, then deleted | No, unless the user opts in |
| *Example:* Order data via tool | Order status answers | Not stored by the assistant | Request only | No |
| *Example:* Feedback ratings | Quality improvement | EU region | 12 months | Aggregated only |

- Third-party processors: [*Example:* LLM vendor under a data processing
  agreement with zero data retention.]
- User rights: [*Example:* conversations can be deleted from the account
  page; access requests via privacy@example.com.]
- Privacy assessment: [Link to DPIA or screening].

## 10. Human oversight

<!-- Guidance: Describe who supervises the system, how they intervene and how users reach a person. -->

[*Example:* A duty safety lead reviews flagged conversations daily. Agents
receive every hand-off with a summary and can see the AI's previous
answers. A kill switch disables the assistant within one minute and shows
a contact form instead. Users can ask for a human at any time by typing
"agent" or using the button.]

| Oversight mechanism | Who | Frequency |
|---|---|---|
| Review of flagged conversations | Safety lead | Daily |
| Random quality sample (200 conversations) | QA team | Weekly |
| Evaluation suite re-run | ML team | Every release |
| Governance review | AI governance board | Quarterly |

## 11. Incidents and feedback

<!-- Guidance: Tell readers how to report a problem and what happens next. -->

| Channel | Use for | Response target |
|---|---|---|
| In-chat thumbs down with comment | Wrong or unhelpful answers | Reviewed weekly |
| [ai-feedback@example.com] | Safety, bias or privacy concerns | Acknowledged in 2 business days |
| [security@example.com] / vulnerability disclosure page | Security vulnerabilities | Per disclosure policy |

[*Example:* Incidents are classified by severity; severity-1 incidents
(harm to users, data exposure) trigger the kill switch review within one
hour and a post-incident report. Summaries of significant incidents are
added to section 13.]

## 12. Versioning and support

<!-- Guidance: Explain how versions are numbered, what triggers a new card and how long versions are supported. -->

- System versions use [semantic versioning]; a new model, new tool or new
  data source is at least a minor release and requires a new card version.
- Evaluations in section 6 are re-run on every minor and major release.
- Previous card versions remain available at [URL].

| Trigger | Card action | Evaluations re-run |
|---|---|---|
| *Example:* New or upgraded model in any component | New card version before release | All of section 6 |
| *Example:* New tool or data source | New card version; update sections 2, 3, 8, 9 | Capability and safety |
| *Example:* New language or market | Update sections 1, 5, 6.3 | Fairness by language |
| *Example:* Significant incident | Update sections 7, 11, 13 | Affected evaluations |
| *Example:* Safeguard change only | Patch version of the card | Safety |

Support window: [*Example:* the current and the previous minor version are
supported; customers with dedicated deployments receive 90 days' notice
before a model change that affects outputs.]

## 13. Changelog

<!-- Guidance: List changes to the system that matter to users, in reverse date order. -->

| System version | Date | Change | Impact on users |
|---|---|---|---|
| *Example:* 2.3.0 | YYYY-MM-DD | Upgraded language model; added French refusal fix | Accuracy +4 points; fewer unsafe answers |
| *Example:* 2.2.0 | YYYY-MM-DD | Added order status tool | Can answer order questions |
| *Example:* 2.1.1 | YYYY-MM-DD | Restricted links to allow-listed domains | Some external links no longer shown |

## Standard and licence note

Based on common industry practice for AI system cards, which extend the
model card approach (Mitchell et al., "Model Cards for Model Reporting",
2019) from a single model to a deployed system. Original text written for
this collection; example results are illustrative. Template licence: MIT.