AI Security Review: [AI system name] [version]
Security review of an AI system against the threats and mitigations that
are specific to AI, structured on ISO/IEC 27090:2026, Cybersecurity,
Artificial intelligence (AI), Addressing security threats and compromises
to AI systems (edition 1, published 14 October 2026 by ISO and IEC, JTC 1/SC
27). The review checks that conventional information security is applied
to the AI system (clause 5), that each type of compromise is considered
(clause 6), and that mitigations and their interactions are chosen with
care (clause 7), with lifecycle and asset mapping in the spirit of Annexes
A to C. It is written in original words and does not reproduce the
standard. Replace every [bracketed] value and every row marked
Example.
Document control
| Field | Value |
|---|
| Document ID | [AISR-001] |
| AI system and version | [Name] [version] (Example: Fenwold Mutual claims triage 2.4, a fictional system) |
| Model and data versions | [Model IDs, dataset snapshot IDs, retrieval index version] |
| Review type | [Design / Pre-release / Periodic / Post-incident] |
| Version | [0.1] |
| Status | [Draft / In review / Approved / Superseded] |
| Owner | [@security-reviewer] |
| Approvers / reviewers | [@ciso], [@head-of-ai], [@system-owner] |
| Classification | [Confidential] |
| Last updated | YYYY-MM-DD |
| Review period | YYYY-MM-DD to YYYY-MM-DD |
| Related records | [ISMS risk register], [AIMS impact assessment], [Threat model], [Red team report], [AI-BOM] |
Approvals
| Role | Name | Decision | Date |
|---|
| Lead security reviewer | [@name] | [Approved / Changes requested] | YYYY-MM-DD |
| System owner (accepts residual risk) | [@name] | [Decision] | YYYY-MM-DD |
| Chief information security officer | [@name] | [Decision] | YYYY-MM-DD |
Revision history
| Version | Date | Author | Change summary |
|---|
| 0.1 | YYYY-MM-DD | [@name] | First draft |
| 1.0 | YYYY-MM-DD | [@name] | Example: Pre-release review approved with two High findings tracked |
| 1.1 | YYYY-MM-DD | [@name] | Example: Retest of AISR-F-01 and AISR-F-05 after fixes |
Contents
1. Review scope and system description
| Item | Description |
|---|
| Purpose | Example: Sorts incoming motor claims into fast-track, standard and specialist queues, and drafts a summary of uploaded documents for adjusters |
| AI components | Example: Gradient-boosted triage classifier; hosted language model for document summaries; retrieval index of policy wording |
| Deployment | Example: Insurer's cloud tenant in one region; adjuster web application; partner garage portal for uploads |
| Users and channels | Example: 300 adjusters; partner garages upload photos and invoices; policyholders upload documents through the claims portal |
| Third parties | Example: Model hosting provider; OCR service; data labelling vendor |
| In scope | [Components, environments, lifecycle stages] |
| Out of scope | [Components and the reason, for example covered by another review] |
| Methods used | Example: Architecture review, configuration review, threat walkthrough, targeted tests, interviews |
The diagram shows the data flow reviewed and where untrusted input enters.
2. Assets and lifecycle stages
| Asset ID | Asset | Group | Lifecycle stages where exposed | Owner | Sensitivity |
|---|
| AS-01 (Example) | Historic claims training data | Data | Collection, preparation, training | [@data-lead] | Personal data, confidential |
| AS-02 (Example) | Labels from the labelling vendor | Data | Preparation | [@data-lead] | Confidential |
| AS-03 (Example) | Triage classifier weights | Model | Training, deployment, operation | [@ml-lead] | Confidential, commercial value |
| AS-04 (Example) | System prompt and tool configuration of the summary model | Model | Deployment, operation | [@ml-lead] | Confidential |
| AS-05 (Example) | Policy wording retrieval index | Data | Deployment, operation | [@ml-lead] | Internal |
| AS-06 (Example) | Uploaded documents and photos | Data | Operation | [@claims-ops] | Personal data |
| AS-07 (Example) | Summaries and triage decisions | Output | Operation | [@claims-ops] | Personal data, decision record |
| AS-08 (Example) | Scoring and summary APIs | Service | Operation | [@platform] | Availability critical |
| [AS-nn] | [Asset] | [Data / Model / Output / Service] | [Stages] | [@owner] | [Sensitivity] |
Lifecycle stages used in this review: inception and design, data
collection and preparation, training and tuning, verification and
validation, deployment, operation and monitoring, retirement.
3. Conventional information security applied to the AI system (clause 5)
| Control area | Question for the AI system | Finding | Control reference | Status |
|---|
| Asset inventory | Are models, datasets, prompts and indexes in the asset inventory with owners? | Example: Retrieval index missing from inventory | [Annex A ref] | Gap |
| Access control | Who can read training data, change prompts, publish models? Is it least privilege? | Example: Data scientists have write access to production registry | [Annex A ref] | Gap (AISR-F-06) |
| Supplier security | Do contracts with model host, OCR and labelling vendors cover AI-specific duties (data use, incident notice, model changes)? | Example: Model host contract silent on model version changes | [Annex A ref] | Partial |
| Secure development | Are pipelines, notebooks and training code under the secure development lifecycle? | Example: Yes, same pipeline as application code | [Annex A ref] | Met |
| Logging and monitoring | Are prompts, retrieved passages, outputs and model versions logged with privacy controls? | Example: Prompts logged; retrieved passages not logged | [Annex A ref] | Partial |
| Configuration and change | Are prompt and model changes change-controlled? | Example: Prompt edits deployed without review | [Annex A ref] | Gap |
| Data leakage prevention | Can outputs carry personal data to the wrong user? | Example: Summaries scoped to the claim; tested | [Annex A ref] | Met |
| Incident management | Does the incident process cover model misbehaviour and poisoning? | Example: Not yet | [Annex A ref] | Gap |
| Backup and recovery | Can the previous model and index be restored quickly? | Example: Registry keeps last 5 versions | [Annex A ref] | Met |
| [Area] | [Question] | [Finding] | [Ref] | [Met / Partial / Gap] |
4. Types of compromise considered (clause 6)
| Threat ID | Threat type | Applies? | Where it could occur | Evidence reviewed | Linked finding |
|---|
| TH-01 | Data poisoning (training or fine-tuning data manipulated) | Example: Yes | AS-01, AS-02: vendor labels and claims history | Data pipeline review, vendor contract | AISR-F-03 |
| TH-02 | Model poisoning or backdoor (model or artefact altered) | Example: Yes | AS-03: registry write access, unsigned artefacts | Registry permissions, deployment logs | AISR-F-06 |
| TH-03 | Evasion (inputs crafted to get a chosen output) | Example: Yes | Feature builder: invoice amounts and photo metadata | Robustness tests | None (controls adequate) |
| TH-04 | Membership inference (learning whether a record was in training data) | Example: Yes, limited | AS-08: classifier returns raw probabilities to partners | API response review | AISR-F-04 |
| TH-05 | Model inversion (reconstructing training data features) | Example: Low | AS-08: same exposure as TH-04 | API response review | AISR-F-04 |
| TH-06 | Model stealing or extraction | Example: Yes | AS-08: partner API without per-client rate limits | Gateway configuration | AISR-F-02 |
| TH-07 | Prompt injection, direct and indirect | Example: Yes | AS-06 into the summary model: text hidden in uploaded PDFs | Injection tests on 40 crafted documents | AISR-F-01 |
| TH-08 | Output injection (model output carrying active content into another system) | Example: Yes | AS-07 rendered as HTML in the adjuster application | Front-end review, test | AISR-F-05 |
| TH-09 | Leakage of training data or of model inputs | Example: Yes | AS-04, AS-06: system prompt disclosure, cross-claim retrieval | Extraction tests | None (controls adequate) |
| [TH-nn] | [Further threat type from the standard] | [Yes / No / Low] | [Asset and location] | [Evidence] | [Finding] |
5. Mitigations and their interactions (clause 7)
| Mitigation ID | Mitigation | Threats addressed | Status | Interaction with other threats or mitigations |
|---|
| MT-01 (Example) | Treat retrieved and uploaded text as data: delimiting, instruction hierarchy, injection classifier | TH-07 | Partial | Classifier adds 120 ms latency; false positives send claims to manual review |
| MT-02 (Example) | Encode model output as text; strict content security policy in the adjuster application | TH-08 | Planned | None negative; also limits damage from TH-07 |
| MT-03 (Example) | Per-client rate limits and query pattern monitoring on partner API | TH-06, TH-04 | Planned | Must not block legitimate batch uploads at month end |
| MT-04 (Example) | Return score band instead of raw probability to partners | TH-04, TH-05, TH-06 | Planned | Less detail for partners; agreed with business |
| MT-05 (Example) | Provenance checks and outlier detection on vendor labels | TH-01 | Partial | Outlier filter may remove rare but genuine claim types; check group coverage after filtering |
| MT-06 (Example) | Signed model artefacts, verified at deployment; registry write limited to pipeline identity | TH-02 | Planned | Adds a release step; break-glass procedure needed |
| MT-07 (Example) | Adversarial and noise augmentation in training | TH-03 | Met | Small accuracy loss (0.4 points); fairness metrics rechecked |
| MT-08 (Example) | Retrieval scoped to the claim and policy; output filter for other claim numbers | TH-09 | Met | None |
| [MT-nn] | [Mitigation] | [Threats] | [Met / Partial / Planned] | [Interaction] |
5.1 Defence in depth check
[Explain, for the two highest threats, which layers stop them if one layer
fails. Example: "For TH-07, if the injection classifier (MT-01) misses a
hidden instruction, the summary model has no tools that write data, and
output encoding (MT-02) prevents rendered links or scripts; the adjuster
still reviews the source documents before deciding."]
6. Lifecycle and asset mapping (Annex A)
| Threat | Data collection and preparation | Training and tuning | Verification and validation | Deployment | Operation and monitoring |
|---|
| TH-01 Data poisoning | AS-01, AS-02 | AS-01 | | | AS-05 (index updates) |
| TH-02 Model poisoning | | AS-03 | AS-03 | AS-03 | |
| TH-03 Evasion | | | Robustness tests | | AS-06 |
| TH-04 Membership inference | | | | | AS-08 |
| TH-05 Model inversion | | | | | AS-08 |
| TH-06 Model extraction | | | | | AS-08 |
| TH-07 Prompt injection | | | Injection tests | AS-04 | AS-06 |
| TH-08 Output injection | | | | | AS-07 |
| TH-09 Data and input leakage | AS-01 | | | AS-04 | AS-06, AS-07 |
7. Conventional attacks in AI form and federated learning (Annexes B and C)
7.1 Conventional attacks in AI form
| Conventional attack | AI form checked | Result |
|---|
| Supply chain compromise | Tampered pre-trained model, poisoned open dataset, malicious model file format | Example: Only safe weight formats loaded; hashes pinned in AI-BOM |
| Denial of service | Resource exhaustion through long prompts or expensive queries | Example: Token and page limits on uploads |
| Injection | Instructions hidden in documents and images reaching the model | See TH-07 |
| Privilege escalation | Model or agent tools acting with more rights than the user | Example: Summary model has read-only access to one claim |
| Data exfiltration | Outputs or logs carrying data to an attacker | See TH-09 |
| [Attack] | [AI form] | [Result] |
7.2 Federated machine learning
| Question | Answer |
|---|
| Does the system train across parties or devices without pooling data? | Example: No; section not applicable |
| If yes: how are malicious participants and poisoned updates detected? | [Answer] |
| If yes: what protects participants' data from inference through shared updates? | [Answer] |
| If yes: how is the aggregation server secured and audited? | [Answer] |
8. Crosswalk to other frameworks
| Threat | OWASP Top 10 for LLM Applications 2026 | MITRE ATLAS (dated release, [v2026.10]) | NIST AI 100-2 E2025 class |
|---|
| TH-01 Data poisoning | LLM05:2026 Data and Model Poisoning | [Technique ID from the current release] | Poisoning (availability and integrity) |
| TH-02 Model poisoning | LLM04:2026 Supply Chain; LLM05:2026 | [Technique ID] | Poisoning, backdoor |
| TH-03 Evasion | Not an LLM entry | [Technique ID] | Evasion |
| TH-04 Membership inference | LLM02:2026 Sensitive Information Disclosure | [Technique ID] | Privacy attack |
| TH-05 Model inversion | LLM02:2026 | [Technique ID] | Privacy attack |
| TH-06 Model extraction | LLM06:2026 Unbounded Consumption (abuse pattern) | [Technique ID] | Privacy attack (model extraction) |
| TH-07 Prompt injection | LLM01:2026 Prompt Injection | [Technique ID] | Direct and indirect prompt injection |
| TH-08 Output injection | LLM10:2026 Improper Output Handling | [Technique ID] | [Class] |
| TH-09 Leakage | LLM02:2026; LLM08:2026 Hidden Context Exposure | [Technique ID] | Privacy attack |
Fill the ATLAS column from the release you checked and record its date;
technique IDs change between releases.
9. Findings
9.1 Rating scale
Likelihood (L) and impact (I) are each rated 1 to 5; score = L x I.
| Band | Score | Expected response |
|---|
| Low | 1 to 5 | Fix in normal backlog |
| Medium | 6 to 11 | Fix within Example: 90 days |
| High | 12 to 19 | Fix before release or within Example: 30 days in operation |
| Critical | 20 to 25 | Stop release; fix or disable the feature |
9.2 Findings register
| Finding ID | Title | Threat | Asset | L | I | Score | Band | Owner |
|---|
| AISR-F-01 (Example) | Hidden text in uploaded PDFs changes summary content (7 of 40 crafted documents succeeded) | TH-07 | AS-06, AS-07 | 4 | 4 | 16 | High | [@ml-lead] |
| AISR-F-02 (Example) | Partner API has no per-client rate limit; extraction feasible | TH-06 | AS-08 | 3 | 3 | 9 | Medium | [@platform] |
| AISR-F-03 (Example) | Vendor labels accepted without provenance or outlier checks | TH-01 | AS-02 | 2 | 4 | 8 | Medium | [@data-lead] |
| AISR-F-04 (Example) | Raw probabilities returned to partners | TH-04, TH-05 | AS-08 | 2 | 3 | 6 | Medium | [@platform] |
| AISR-F-05 (Example) | Summary output rendered as HTML in the adjuster application | TH-08 | AS-07 | 3 | 5 | 15 | High | [@frontend-lead] |
| AISR-F-06 (Example) | Unsigned model artefacts; data scientists can write to production registry | TH-02 | AS-03 | 2 | 4 | 8 | Medium | [@ml-platform] |
| [AISR-F-nn] | [Title] | [TH] | [AS] | [L] | [I] | [Score] | [Band] | [@owner] |
Example summary: 6 findings, 2 High (AISR-F-01, AISR-F-05), 4 Medium,
none Critical or Low. The injection success rate in AISR-F-01 is 7 of 40,
or 17.5%.
10. Action plan and retest
| Finding | Mitigation | Due | Retest method | Retest result | Status |
|---|
| AISR-F-01 (Example) | MT-01 complete, MT-02 | YYYY-MM-DD | Re-run the 40 crafted documents plus 40 new ones | Example: 0 of 80 succeeded | Closed |
| AISR-F-05 (Example) | MT-02 | YYYY-MM-DD | Front-end test with script and link payloads | [Result] | Open |
| AISR-F-02 (Example) | MT-03 | YYYY-MM-DD | Load test with extraction query pattern | [Result] | Open |
| AISR-F-03 (Example) | MT-05 | YYYY-MM-DD | Seeded poisoned labels detected | [Result] | Open |
| AISR-F-04 (Example) | MT-04 | YYYY-MM-DD | API response check | [Result] | Open |
| AISR-F-06 (Example) | MT-06 | YYYY-MM-DD | Deploy unsigned artefact in staging; expect rejection | [Result] | Open |
11. Conclusion
| Item | Value |
|---|
| Overall assessment | [Acceptable / Acceptable with conditions / Not acceptable] |
| Conditions | Example: Release after AISR-F-05 is closed; AISR-F-02 to F-06 tracked in the ISMS risk register |
| Residual risk accepted by | [@system-owner], YYYY-MM-DD |
| Next review | Example: in 12 months, or earlier on retraining, a new upload channel, new tools or an incident |
12. Glossary and references
| Term | Meaning in this review |
|---|
| Poisoning | Manipulating training data or a model so that it behaves as an attacker wants |
| Evasion | Crafting inputs at use time so that the model gives a wrong or chosen output |
| Membership inference | Working out whether a given record was part of the training data |
| Model inversion | Reconstructing sensitive features of training data from model outputs |
| Model extraction | Building a copy of a model by querying it |
| Prompt injection | Instructions placed in input or retrieved content that override the intended instructions |
| Output injection | Model output that carries active content or commands into a downstream system |
References (checked October 2026; check the current versions):
Related templates: ai-data/agentic-threat-model, ai-data/ai-red-team,
ai-data/guardrails-spec, ai-data/ai-bom, ai-data/agent-tool-permission-design,
security/threat-model, security/supply-chain-security,
compliance/statement-of-applicability.
Standard and licence note
Structure aligned with ISO/IEC 27090:2026, Cybersecurity, Artificial
intelligence (AI), Addressing security threats and compromises to AI
systems (edition 1, October 2026): clause 5 (application of information
security), clause 6 (types of potential compromise), clause 7 (mitigations
and their interactions) and Annexes A (lifecycle and asset mapping), B
(AI-specific versions of conventional attacks) and C (federated machine
learning) are cited by number and topic only. This template is original
text written for this collection and reproduces no wording, tables or
definitions from the standard; threat types listed are those named in
public summaries by BSI and other public sources. Official text: ISO and
IEC (UK adoption by BSI), purchase or licensed access:
https://www.iso.org/standard/56581.html or
https://knowledge.bsigroup.com/products/cybersecurity-artificial-intelligence-ai-addressing-security-threats-and-compromises-to-ai-systems.
OWASP entry IDs and short names are cited with attribution to the OWASP
GenAI Security Project (CC BY-SA 4.0); no OWASP text is adapted. The
system, organisation and figures in the examples are fictional. Check the
current official versions before use. Template licence: MIT.