Guardrails and Safety Filter Specification Template (OWASP LLM Top 10 2026)

Policy taxonomy, input, output and tool-call guards, thresholds and actions, false-positive and latency budgets, guard evaluation, OWASP LLM 2026 mapping

KI & DatenVollständigOWASP Top 10 for LLM Applications 2026; OWASP Top 10 for Agentic Applications 2026CC BY-SA 4.0guardrails spec mathAktualisiert 21.10.2026

Im Editor öffnen.md herunterladen

Anleitung: Use it when you add or change safety filters, injection detectors or tool-call rules around an LLM application or agent. Fill the policy taxonomy first, then each guard's threshold, action and budgets from evaluation data. The main pitfall is treating a classifier as the only control in front of a tool or data store.

Vorschau

Guardrails and Safety Filter Specification: [Application name] [version]

Specification of the guardrails around an LLM application or agent: the policy taxonomy, input guards (prompt injection, jailbreak, personal data), output guards (toxicity, grounding, data leakage, format), tool-call guards, thresholds and actions (block, redact, warn, escalate), false-positive and latency budgets, evaluation of the guards themselves, monitoring and change management. Each guard is mapped to the OWASP Top 10 for LLM Applications 2026 (LLM01:2026 to LLM10:2026) and, for agents, to the OWASP Top 10 for Agentic Applications 2026. Replace every [bracketed] value and every row marked Example. Worked examples use a customer support assistant for an online bank that answers questions from help articles and can look up a customer's own transactions.

Document control

Field	Value
Document ID	[GRD-2026-001]
Version	[0.1]
Status	[Draft / In review / Approved / Superseded]
Owner	[@safety-engineer]
Approvers / reviewers	[@app-owner], [@security-lead], [@privacy-officer], [@trust-and-safety]
Last updated	YYYY-MM-DD
Classification	[Confidential; thresholds help attackers]
Application	[Name], [version or commit]
Guard bundle version	[guards-vN], deployed as [identifier]
Model(s) guarded	[Provider, model, snapshot date]
Frameworks used	OWASP Top 10 for LLM Applications 2026; OWASP Top 10 for Agentic Applications 2026
Related documents	[Prompt specification], [Agentic threat model], [Red team report]

Approvals

Role	Name	Decision	Date
Application owner	[@name]	[Approved / Approved with conditions / Rejected]	YYYY-MM-DD
Security lead	[@name]	[Decision]	YYYY-MM-DD
Privacy officer	[@name]	[Decision]	YYYY-MM-DD
Trust and safety lead	[@name]	[Decision]	YYYY-MM-DD

Revision history

Version	Date	Author	Change summary
0.1	YYYY-MM-DD	[@name]	First draft
0.2	YYYY-MM-DD	[@name]	Example: Lowered injection threshold after red team finding RT-F-002
1.0	YYYY-MM-DD	[@name]	Example: Approved for production with guards-v3

Scope and architecture
Policy taxonomy
Input guards
Output guards
Tool-call guards
Thresholds and actions
False-positive and latency budgets
Evaluation of guards
OWASP mapping
Monitoring and response
Change management
Glossary and references

1. Scope and architecture

1.1 Application summary

[Application name] answers [customer questions about cards, transfers and fees] using [retrieved help articles] and can call [get_transactions for the signed-in customer only]. It serves [about 200,000 conversations per month] in [web and mobile chat] in [languages].

1.2 Guard placement

The diagram shows every point where a guard inspects content or an action. Retrieved text and tool results pass the context guards before they reach the model.

1.3 Guard inventory

ID	Guard	Stage	Mechanism	Version	Owner
GRD-IN-001 (Example)	Prompt injection detector	Input and context	Fine-tuned classifier	[v4]	[@safety-engineer]
GRD-IN-002 (Example)	Jailbreak detector	Input	Classifier plus pattern list	[v2]	[@safety-engineer]
GRD-IN-003 (Example)	Personal data detector	Input	Named entity model plus regular expressions for IBAN and card numbers	[v5]	[@privacy-eng]
GRD-IN-004 (Example)	Topic scope	Input	Embedding similarity to allowed topics	[v1]	[@product]
GRD-OUT-001 (Example)	Toxicity and harassment	Output	Moderation model	[v3]	[@trust-and-safety]
GRD-OUT-002 (Example)	Grounding check	Output	Claim-to-source entailment model	[v2]	[@ml-engineer]
GRD-OUT-003 (Example)	Data leakage	Output	Personal data detector plus hidden-context canary match	[v5]	[@privacy-eng]
GRD-OUT-004 (Example)	Format and rendering	Output	Schema validation, Markdown sanitiser, link allowlist	[v2]	[@app-engineer]
GRD-TOOL-001 (Example)	Tool allowlist and argument validation	Tool call	Deterministic policy engine enforcing TGR-001 to TGR-003 and TGR-005	[v3]	[@platform-engineer]
GRD-TOOL-002 (Example)	Tool-call rate and loop limit	Tool call	Counter per session enforcing TGR-004	[v1]	[@platform-engineer]
[GRD-XX-NNN]	[Guard]	[Stage]	[Mechanism]	[vN]	[@owner]

2. Policy taxonomy

ID	Category	Includes	Excludes	Default action	Severity
POL-01 (Example)	Prompt injection	Instructions to ignore rules, role overrides, hidden instructions in documents	Questions about how the assistant works	Block	High
POL-02 (Example)	Jailbreak attempt	Role play or encoding meant to bypass safety behaviour	Fiction requests without policy impact	Block	High
POL-03 (Example)	Personal data from the user	Card numbers, IBANs, government IDs typed into chat	Customer's own first name	Redact before logging and model call	Medium
POL-04 (Example)	Personal data of others	Data about a person who is not the signed-in customer	Public company names	Block in output	High
POL-05 (Example)	Harassment and hate	Abuse aimed at people or groups	Customer quoting abuse they received, for a complaint	Warn and de-escalate	Medium
POL-06 (Example)	Self-harm signals	Statements of intent or crisis	Discussion of insurance terms	Escalate to human with support resources	Critical
POL-07 (Example)	Ungrounded financial statements	Fees, rates or rules not supported by retrieved articles	General greetings	Block and regenerate	High
POL-08 (Example)	Hidden context exposure	System prompt, internal rules, tool schemas	Public help article text	Block	Medium
POL-09 (Example)	Out of scope	Investment advice, legal advice, unrelated topics	Questions about bank products	Redirect	Low
POL-10 (Example)	Unsafe rendering	Scripts, external images, links outside the allowlist	Links to the bank's help centre	Strip	High
[POL-NN]	[Category]	[Includes]	[Excludes]	[Action]	[Severity]

Labelling guide: [link to the annotation guidelines with at least five positive and five negative examples per category].

3. Input guards

3.1 Prompt injection and jailbreak

Item	GRD-IN-001 Prompt injection	GRD-IN-002 Jailbreak
Inspects	User message, retrieved chunks, tool results	User message and last [3] turns
Policy	POL-01	POL-02
Decision	Score 0 to 1; threshold in section 6	Score 0 to 1 plus pattern list
On hit (user message)	Block, fallback reply, log	Block, fallback reply, log
On hit (retrieved chunk)	Drop chunk, mark source for review	n/a
Known limits	Novel phrasings and other languages; see evaluation in section 8	Multi-turn escalation

Design rules:

Mark untrusted content (retrieved text, tool results) as data in the prompt and never let it change tool selection on its own.
Do not rely on the injection detector to protect tools; tool-call guards in section 5 enforce permissions independently.
Normalise input before scoring: decode common encodings, remove zero-width characters, and score each language the product supports.

3.2 Personal data in input

GRD-IN-003 detects [card numbers (with Luhn check), IBANs, national ID formats for supported countries, email addresses and phone numbers]. Detected values are replaced with typed placeholders such as [CARD_1] before the text is logged or sent to the model, and the mapping is kept only in memory for the session so the tool layer can use it where the task needs it.

3.3 Topic scope

GRD-IN-004 compares the message with the allowed topic set. Below the similarity threshold the assistant replies with a scoped redirect rather than a refusal, and the message is not sent to the model.

4. Output guards

Guard	Checks	Policy	On hit	Regenerations
GRD-OUT-001 (Example)	Toxic, harassing or hateful language	POL-05	Block and regenerate with stricter instruction	1, then fallback
GRD-OUT-002 (Example)	Each factual claim about fees, limits or rules is entailed by a retrieved source	POL-07	Remove unsupported claim or regenerate; cite sources	1, then "I could not confirm this" reply
GRD-OUT-003 (Example)	Personal data of anyone other than the signed-in customer; canary strings from hidden context	POL-04, POL-08	Block and log as security event	0
GRD-OUT-004 (Example)	JSON schema for structured replies; Markdown sanitised; links on allowlist; no external images	POL-10	Strip unsafe parts; on schema failure regenerate	2
[GRD-OUT-NNN]	[Checks]	[POL-NN]	[Action]	[n]

Notes:

Streaming: when answers stream to the user, run fast guards on each chunk and hold the final [N] tokens until the slower guards pass, or disable streaming for high-risk intents.
Grounding: a grounding guard needs the retrieved sources as input; log the source IDs with the decision so reviewers can check it.
Rendering: output encoding and sanitisation happen in the client or renderer as well, because the guard can be bypassed by a bug elsewhere.

5. Tool-call guards

Rule ID	Tool	Rule	Enforcement point	On violation
TGR-001 (Example)	`get_transactions`	`customer_id` must equal the session's customer; model-supplied IDs are ignored	Tool gateway	Reject, log security event
TGR-002 (Example)	`get_transactions`	Date range at most [90] days; page size at most [50]	Tool gateway	Clamp and log
TGR-003 (Example)	Any	Only tools on the allowlist for this intent may be called	Orchestrator	Reject
TGR-004 (Example)	Any	At most [5] tool calls per turn and [20] per session	Orchestrator	Stop, fallback reply
TGR-005 (Example)	`create_dispute` (if enabled)	Requires explicit customer confirmation shown by the client, not by the model	Client and gateway	Hold until confirmed
[TGR-NNN]	[Tool]	[Rule]	[Point]	[Action]

Agent deployments: align these rules with the [Agent tool permission design] and the OWASP agentic categories ASI02 (Tool Misuse and Exploitation) and ASI03 (Identity and Privilege Abuse).

6. Thresholds and actions

6.1 Actions

Action	Meaning	User sees	Logged as
Allow	Pass through	Normal answer	Decision only
Redact	Replace the matched span with a placeholder	Answer with placeholder	Decision and span type
Warn	Allow but append a notice or soften	Answer with notice	Decision and score
Block	Do not send; use fallback	Fallback reply	Decision, score, excerpt reference
Escalate	Hand to a human or a support flow	Handoff message	Decision and case ID

6.2 Threshold table

Guard	Score range	Allow below	Warn	Block at or above	Escalate	Basis
GRD-IN-001 (Example)	0 to 1	0.50	0.50 to 0.80 (context chunks: drop)	0.80	n/a	Section 8 evaluation, run EV-12
GRD-IN-002 (Example)	0 to 1	0.60	n/a	0.60	n/a	Section 8, run EV-12
GRD-OUT-001 (Example)	0 to 1	0.40	0.40 to 0.70	0.70	n/a	Moderation vendor guidance plus own test set
GRD-OUT-002 (Example)	Entailment 0 to 1	n/a	n/a	Below 0.60	n/a	Calibration set GS-03
POL-06 detector (Example)	0 to 1	0.30	n/a	n/a	0.30	Low threshold by design: false negatives are costly
[Guard]	[Range]	[Value]	[Range]	[Value]	[Value]	[Evidence]

Where a threshold is used per language or per channel, list each row separately.

7. False-positive and latency budgets

7.1 False-positive budget

The false-positive rate (FPR) and the false-negative rate (FNR) of a guard on a labelled set are:

\text{FPR} = \frac{FP}{FP + TN} \qquad \text{FNR} = \frac{FN}{FN + TP}

Guard	Max FPR on benign traffic	Max FNR on attack or policy set	Measured on
GRD-IN-001 (Example)	[1.0%]	[10%]	Benign set BN-01 (5,000), attack set AT-01 (1,200)
GRD-IN-003 (Example)	[0.5%]	[2%]	PII set PII-02
GRD-OUT-002 (Example)	[3%]	[8%]	Grounding set GS-03
Combined, all guards (Example)	[2.5%] of conversations with any block	n/a	Production sample
[Guard]	[Value]	[Value]	[Set]

7.2 Latency budget

Stage	Guard(s)	p50 budget	p95 budget	Mode
Input (Example)	GRD-IN-001 to GRD-IN-004	[40 ms]	[120 ms]	Parallel
Context (Example)	GRD-IN-001 on chunks	[30 ms]	[90 ms]	Batched
Output (Example)	GRD-OUT-001 to GRD-OUT-004	[150 ms]	[400 ms]	Parallel, before release of final tokens
Tool call (Example)	TGR rules	[5 ms]	[20 ms]	Inline
Total added (Example)	All	[250 ms]	[600 ms]	n/a
[Stage]	[Guards]	[Value]	[Value]	[Mode]

Failure mode: if a guard times out or errors, [fail closed for GRD-OUT-003 and TGR rules; fail open with logging for GRD-IN-004]. Record the choice per guard.

8. Evaluation of guards

Set	Purpose	Size	Source	Refresh
BN-01 (Example)	Benign production-like traffic	5,000	Sampled, anonymised conversations	Quarterly
AT-01 (Example)	Injection and jailbreak attacks	1,200	Red team library, public benchmarks, new findings	Monthly
PII-02 (Example)	Personal data detection	2,000	Synthetic generator plus real formats	Quarterly
GS-03 (Example)	Grounded and ungrounded answers	800	Human-labelled	Quarterly
ML-01 (Example)	Multilingual variants of AT-01	1,200	Translated and native-written	Quarterly
[Set]	[Purpose]	[n]	[Source]	[Cadence]

Report for each guard: precision, recall, FPR and FNR with 95% intervals, broken down by language and channel; the confusion matrix; and the 20 most frequent false positives with a fix proposal. Feed red team findings into AT-01 within [one week]. Keep a private hold-out portion of each attack set so guard tuning does not overfit.

Adaptive attacks: guards are tested by people who know the guard exists. Include cases crafted to evade the specific guard (encodings, splitting across turns, payloads in retrieved documents) and report them separately from static benchmark results.

9. OWASP mapping

The IDs and short names below follow the OWASP Top 10 for LLM Applications 2026 (published August 2026). Check the current list at https://genai.owasp.org/ before review.

OWASP 2026 ID	Risk	Guards	Other controls (outside this document)
LLM01:2026	Prompt Injection	GRD-IN-001, GRD-IN-002, TGR-003	Data marking in prompts, least-privilege tools
LLM02:2026	Sensitive Information Disclosure	GRD-IN-003, GRD-OUT-003, TGR-001	Access control in retrieval and tools
LLM03:2026	Excessive Agency	TGR-001 to TGR-005	Agent tool permission design
LLM04:2026	Supply Chain	None	Model and package provenance, AI bill of materials
LLM05:2026	Data and Model Poisoning	GRD-IN-001 on retrieved chunks (partial)	Content ingestion review, index provenance
LLM06:2026	Unbounded Consumption	TGR-004, input length limit	Rate limiting, cost budgets
LLM07:2026	Misinformation	GRD-OUT-002	Retrieval quality, evaluation plan
LLM08:2026	Hidden Context Exposure	GRD-OUT-003 canary match	Keep secrets and authorisation out of prompts
LLM09:2026	Vector and Embedding Weaknesses	None	Tenant filters in the vector store
LLM10:2026	Improper Output Handling	GRD-OUT-004	Output encoding in the client

For agents, also map to the OWASP Top 10 for Agentic Applications 2026:

ASI ID	Category	Guards
ASI01	Agent Goal Hijack	GRD-IN-001 on all untrusted inputs, TGR-003
ASI02	Tool Misuse and Exploitation	TGR-001 to TGR-005
ASI03	Identity and Privilege Abuse	TGR-001
ASI06	Memory & Context Poisoning	GRD-IN-001 on memory writes and recalls
ASI09	Human-Agent Trust Exploitation	GRD-OUT-002 grounding with visible sources
[ASInn]	[Category]	[Guards]

10. Monitoring and response

Signal	Threshold for alert	Receiver	Response
Example: Block rate per guard	Change of more than [50%] week on week	[@safety-engineer]	Check for attack wave or false-positive regression
Example: GRD-OUT-003 hits	Any	[@security-on-call]	Treat as possible data leak; open incident
Example: TGR-001 rejections	More than [10] per hour	[@security-on-call]	Investigate targeted abuse
Example: Guard p95 latency	Above budget for [15 minutes]	[@platform-on-call]	Scale or fail over
Example: User appeals of blocks	More than [20] per week	[@product]	Review false positives
[Signal]	[Threshold]	[Receiver]	[Response]

Logging: record guard ID, version, score, decision and a reference to the content in an access-controlled store; do not copy raw personal data into general logs. Retain guard logs for [90 days].

Serious events (data leak, self-harm escalation failure, tool rule bypass) follow the [AI incident report] process.

11. Change management

Change type	Evidence required	Approver	Rollout
Threshold change	Before and after FPR and FNR on all sets	Guard owner and security lead	Canary [10%] for [48 h]
New guard model version	Full section 8 evaluation, latency test	Guard owner, security lead, trust and safety	Shadow mode, then canary
New policy category	Labelling guide, test set, legal or policy sign-off	Application owner, trust and safety	Shadow mode first
Pattern list update	Regression on BN-01 and AT-01	Guard owner	Direct, with rollback
Emergency block rule	Incident ticket	Security on-call	Immediate; review within [5] working days

Keep every guard version, threshold and pattern list in version control with the change reason. Rollback must be possible within [15 minutes].

12. Glossary and references

Term	Meaning
Guardrail	A control that checks or constrains model inputs, outputs or actions against a policy
Fail closed	When a guard errors, block the request
Fail open	When a guard errors, let the request continue and log it
Grounding	Whether a claim is supported by the provided sources
Canary string	A unique marker placed in hidden context so that its appearance in output reveals a leak
False positive	A benign request the guard wrongly blocks or changes
Adaptive attack	An attack designed with knowledge of the specific guard

References:

OWASP GenAI Security Project, OWASP Top 10 for LLM Applications 2026, August 2026: https://genai.owasp.org/resource/owasp-genai-llm-top-10-2026/
OWASP GenAI Security Project, OWASP Top 10 for Agentic Applications for 2026, December 2025: https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/
NIST AI 100-2 E2025, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations, March 2025: https://csrc.nist.gov/pubs/ai/100/2/e2025/final
[Prompt specification], [Agent tool permission design], [Agentic threat model], [AI red team plan and report], [LLM observability specification].

Standard and licence note

Based on the OWASP Top 10 for LLM Applications 2026 and the OWASP Top 10 for Agentic Applications for 2026 by the OWASP GenAI Security Project (https://genai.owasp.org/), licensed under Creative Commons Attribution-ShareAlike 4.0 (https://creativecommons.org/licenses/by-sa/4.0/), and on NIST AI 100-2 E2025 (a NIST publication). The OWASP IDs and short names are used for mapping; all descriptions, policies, thresholds and examples are original text, and example values are illustrative rather than recommended settings. Because the mapping tables adapt the OWASP structure, that adapted structure is shared under CC BY-SA 4.0 and any further adaptation must carry the same licence. Check the current official versions of the OWASP lists before use. Apart from the adapted OWASP structure, the original text is under the template licence. Template licence: MIT.