# Enterprise AI Platform Reference Architecture: [Organisation name]

> Reference architecture for the shared platform on which an organisation builds, runs and governs
> generative AI and agent workloads: the capability map, model access through a gateway, the data
> and retrieval layer, the agent runtime with tools over the Model Context Protocol (MCP,
> specification revision 2026-07-28) and agent-to-agent calls over the Agent2Agent (A2A) protocol
> (specification 1.0.1, a patch release of 1.0.0; protocol version 1.0), evaluation and guardrails, identity and data permissions, observability
> with the OpenTelemetry semantic conventions for generative AI (Development status), cost
> management, governance hooks for ISO/IEC 42001:2023 and Regulation (EU) 2024/1689 (the AI Act) as
> amended by Regulation (EU) 2026/1744, tenancy for business units, build or buy decisions and the
> roadmap. It connects the detailed design templates of this library instead of repeating them.
> Replace every `[bracketed]` value and every row marked *Example*. The regulatory rows are a
> documentation format, not legal advice.

<!--
Guidance: How to use this template
- Write one reference architecture per organisation (or per legal entity when platforms are
  separate). Individual AI products then cite it and record only how they deviate.
- Fill sections 1 and 2 first: the drivers and the capability map decide what the platform owns and
  what each product team owns.
- Keep this document at reference level. Detailed designs live in the linked templates: LLM gateway
  (`system-design/design-llm-gateway`), enterprise search
  (`system-design/design-enterprise-rag-search`), RAG (`ai-data/rag-system-design`), agent
  orchestration (`system-design/design-agent-orchestration-platform`), multi-agent behaviour
  (`ai-data/multi-agent-system-design`), guardrails (`ai-data/guardrails-spec`), observability
  (`ai-data/llm-observability-spec`) and MCP servers (`ai-data/mcp-server-spec`).
- Review each workload built on the platform with the GenAI well-architected review
  (`architecture/genai-well-architected-review`).
- Check protocol revisions, semantic convention status and regulatory dates on the official pages
  before each major version of this document; all three move quickly.
-->

## Document control

<!-- Guidance: Identify the platform, its owner and the protocol and regulatory baselines it assumes. A change of gateway, agent runtime, tenancy model or regulatory baseline is a new major version. -->

| Field | Value |
|---|---|
| Document ID | [EAP-RA-001] |
| Platform name | [Name of the enterprise AI platform] |
| Version | [0.1] |
| Status | [Draft / In review / Approved / Superseded] |
| Owner | [@ai-platform-lead] |
| Approvers/reviewers | [@chief-architect], [@ciso], [@data-protection-officer], [@ai-governance-lead], [@finops-lead] |
| Business units in scope | [BU 1], [BU 2], [BU 3] |
| Protocol baseline | [MCP 2026-07-28], [A2A specification 1.0.1, protocol version 1.0] |
| Telemetry baseline | [OpenTelemetry GenAI semantic conventions, repository commit or tag, YYYY-MM-DD] |
| Management system | [ISO/IEC 42001:2023 AI management system, certified / aligned / not adopted] |
| Regulatory baseline | [Regulation (EU) 2024/1689 as amended by 2026/1744, consolidated text checked YYYY-MM-DD], [other jurisdictions] |
| Last updated | YYYY-MM-DD |
| Classification | [Internal / Confidential] |

### Approvals

| Role | Name | Decision | Date |
|---|---|---|---|
| Chief architect | [@name] | [Approve / Approve with conditions / Reject] | YYYY-MM-DD |
| Chief information security officer | [@name] | [Decision] | YYYY-MM-DD |
| Data protection officer | [@name] | [Decision] | YYYY-MM-DD |
| AI governance board | [@name] | [Decision] | YYYY-MM-DD |
| Finance and FinOps | [@name] | [Decision] | YYYY-MM-DD |

### Revision history

| Version | Date | Author | Change summary |
|---|---|---|---|
| 0.1 | YYYY-MM-DD | [@name] | First draft |
| 1.0 | YYYY-MM-DD | [@name] | *Example:* Approved; gateway and retrieval layer live for two business units |
| 1.1 | YYYY-MM-DD | [@name] | *Example:* Added agent runtime with MCP tool registry and A2A federation |

## Contents

<!-- Guidance: Keep this list aligned with the headings. -->

1. [Scope, drivers and principles](#1-scope-drivers-and-principles)
2. [Capability map](#2-capability-map)
3. [Reference architecture overview](#3-reference-architecture-overview)
4. [Model access and gateway](#4-model-access-and-gateway)
5. [Data and retrieval layer](#5-data-and-retrieval-layer)
6. [Agent runtime, tools and protocols](#6-agent-runtime-tools-and-protocols)
7. [Evaluation and guardrails](#7-evaluation-and-guardrails)
8. [Identity, data permissions and security](#8-identity-data-permissions-and-security)
9. [Observability and cost](#9-observability-and-cost)
10. [Governance hooks](#10-governance-hooks)
11. [Tenancy for business units](#11-tenancy-for-business-units)
12. [Build, buy or reuse](#12-build-buy-or-reuse)
13. [Operating model](#13-operating-model)
14. [Roadmap](#14-roadmap)
15. [Risks and traceability](#15-risks-and-traceability)
16. [Glossary and references](#16-glossary-and-references)

## 1. Scope, drivers and principles

<!-- Guidance: State why a shared platform exists and which disputes the principles settle. Questions: which workloads must use the platform and which may stay outside? What does a product team get on day one? Pitfall: a platform defined by the tools it bought rather than the outcomes it guarantees. -->

### 1.1 Scope

| In scope | Out of scope |
|---|---|
| *Example:* Shared access to hosted and self-hosted models through one gateway | Training foundation models from scratch |
| *Example:* Retrieval services over approved enterprise content with permission filtering | Business-unit data products that do not feed AI workloads |
| *Example:* Agent runtime, tool registry and evaluation services | The behaviour design of each individual agent |
| *Example:* Embedded copilots in ERP, CRM and ITSM suites, as governed consumers of the platform controls | The vendor's own model hosting inside those suites |
| [Item] | [Item] |

### 1.2 Drivers

| ID | Driver | Measure of success |
|---|---|---|
| EAP-DRV-001 *(Example)* | Business units buy overlapping AI tools and models with no shared controls | Share of production AI workloads routed through the gateway reaches [90 %] within [12 months] |
| EAP-DRV-002 *(Example)* | Each team re-implements retrieval, guardrails and logging | Time from approved use case to first production release falls from [16] to [6] weeks |
| EAP-DRV-003 *(Example)* | No inventory of AI systems for regulatory and management-system duties | Every production AI system has an inventory record and a risk classification |
| EAP-DRV-004 *(Example)* | AI spend is invisible until the invoice arrives | Monthly showback per business unit and per use case, variance against budget under [10 %] |
| [EAP-DRV-NNN] | [Driver] | [Measure] |

### 1.3 Principles

| ID | Principle | Consequence |
|---|---|---|
| EAP-P-001 *(Example)* | One door to models: every model call passes the gateway | No direct provider keys in product code; keys live in the gateway only |
| EAP-P-002 *(Example)* | Permissions follow the data: a user's AI answer never shows what the user cannot open in the source system | Retrieval filters by the caller's identity, not the agent's service identity |
| EAP-P-003 *(Example)* | Tools are products with owners | Every MCP server and A2A agent is registered, versioned and has an accountable owner |
| EAP-P-004 *(Example)* | No production without evaluation | A release gate blocks promotion until the evaluation suite passes agreed thresholds |
| EAP-P-005 *(Example)* | Paved road, not a cage | Teams may leave the paved road with an approved exception that names the compensating controls |
| EAP-P-006 *(Example)* | Model portability | Product code depends on the gateway API, not on one provider's SDK |
| [EAP-P-NNN] | [Principle] | [Consequence] |

## 2. Capability map

<!-- Guidance: List the platform capabilities in layers and say who provides each one. This map is the backbone of the document: every later section expands one row group. Pitfall: capabilities without an owner or a maturity target. -->

Assess current and target maturity on a scale of 0 (absent) to 4 (managed and measured). Provider is
Platform (built or run by the platform team), Vendor (bought service), Product (left to product
teams) or Shared.

| ID | Layer | Capability | Provider | Current | Target | Detailed design |
|---|---|---|---|---|---|---|
| CAP-01 *(Example)* | Experience | Chat and assistant front ends, embedded copilots | Product / Vendor | 2 | 3 | [Product designs] |
| CAP-02 *(Example)* | Model access | Unified model API, routing, fallback, quotas, caching | Platform | 1 | 4 | `system-design/design-llm-gateway` |
| CAP-03 *(Example)* | Model access | Model catalogue and approval of new models | Platform | 1 | 3 | Section 4.2 |
| CAP-04 *(Example)* | Model access | Self-hosted model serving | Platform | 0 | 2 | `system-design/design-llm-inference-serving` |
| CAP-05 *(Example)* | Data | Content connectors, ingestion and chunking | Platform | 1 | 3 | `ai-data/rag-system-design` |
| CAP-06 *(Example)* | Data | Hybrid retrieval with permission filtering | Platform | 1 | 4 | `system-design/design-enterprise-rag-search` |
| CAP-07 *(Example)* | Agents | Durable agent runtime and human-in-the-loop pauses | Platform | 0 | 3 | `system-design/design-agent-orchestration-platform` |
| CAP-08 *(Example)* | Agents | Tool registry (MCP servers) and agent registry (A2A Agent Cards) | Platform | 0 | 3 | `ai-data/mcp-server-spec`, `ai-data/a2a-agent-card-spec` |
| CAP-09 *(Example)* | Agents | Sandboxed code and browser execution | Platform | 0 | 2 | `system-design/design-agent-sandbox-execution` |
| CAP-10 *(Example)* | Trust | Input, output and tool-call guardrails | Platform | 1 | 3 | `ai-data/guardrails-spec` |
| CAP-11 *(Example)* | Trust | Offline and online evaluation service | Platform | 1 | 3 | `ai-data/llm-evaluation-plan`, `ai-data/agent-evaluation-plan` |
| CAP-12 *(Example)* | Trust | Identity propagation, delegated authorisation, secrets | Shared | 2 | 4 | Section 8 |
| CAP-13 *(Example)* | Operations | Traces, metrics, prompt capture under policy | Platform | 1 | 3 | `ai-data/llm-observability-spec` |
| CAP-14 *(Example)* | Operations | Cost metering, budgets and showback | Platform | 0 | 3 | `ai-data/inference-cost-model` |
| CAP-15 *(Example)* | Governance | AI system inventory, risk classification, evidence store | Shared | 1 | 3 | Section 10 |
| [CAP-NN] | [Layer] | [Capability] | [Provider] | [0-4] | [0-4] | [Link] |

## 3. Reference architecture overview

<!-- Guidance: Show the layers and the paths a request takes. Questions: where is the trust boundary between product code and the platform? Which components see raw prompts? Pitfall: a diagram of products and logos with no flows. -->

```mermaid
flowchart TB
  subgraph EXP["Experience layer"]
    APP["Product apps and assistants"]
    COP["Embedded copilots in ERP, CRM, ITSM"]
  end
  subgraph AGT["Agent layer"]
    RT["Agent runtime"] --> TREG["Tool registry (MCP)"] & AREG["Agent registry (A2A)"] & SBX["Sandbox execution"]
  end
  subgraph MOD["Model access layer"]
    GW["LLM gateway"] --> GR["Guardrail service"] & HOST["Hosted model providers"] & SELF["Self-hosted models"]
  end
  subgraph DAT["Data layer"]
    SRC["Source systems"] --> CON["Connectors and ingestion"] --> IDX["Vector and keyword indexes"]
    RET["Retrieval service"] --> IDX
  end
  subgraph CTL["Control plane"]
    IAM["Identity and policy"]
    EVAL["Evaluation service"]
    OBS["Observability and cost"] -.-> INV["AI inventory and evidence"]
  end
  APP --> RT & GW
  RT --> GW & RET
  IAM -.-> GW & RET & TREG
  GW & RT -.-> OBS
  EVAL -.-> GW
  COP -.-> IAM & OBS
```

Read the diagram with these rules. Solid arrows are request paths; dotted arrows are control and
telemetry paths. Product code reaches models only through the gateway, either directly or through
the agent runtime. The retrieval service and the tool registry enforce the end user's permissions,
which the identity and policy service supplies. Embedded vendor copilots run their own model stack,
so the platform governs them through identity, logging and evaluation (PATH-D).

### 3.1 Request paths

| Path | Steps | Used for |
|---|---|---|
| PATH-A *(Example)* | App, gateway, guardrails, model | Drafting, summarisation, classification without enterprise data |
| PATH-B *(Example)* | App, retrieval (permission-filtered), gateway, model, citation check | Question answering over enterprise content |
| PATH-C *(Example)* | App, agent runtime, gateway and tools (MCP), optional remote agents (A2A), human approval, action | Multi-step tasks that change records |
| PATH-D *(Example)* | Embedded copilot in a vendor suite, vendor model stack, platform identity and log export | Vendor-native assistants, designed with `architecture/genai-enterprise-app-integration` |
| [PATH-X] | [Steps] | [Use] |

## 4. Model access and gateway

<!-- Guidance: Decide how every workload reaches every model. Questions: which models are approved for which data classes? Who may add a model? What happens when a provider is down or deprecates a model? Pitfall: approving a model once and never re-evaluating it when the provider changes its version. -->

The gateway is the single control point for model use. It authenticates the calling workload,
applies per-tenant quotas, routes to an approved model, applies guardrails, meters tokens and cost,
and emits telemetry. Design it in detail with the LLM gateway design
(`system-design/design-llm-gateway`); this section records the enterprise decisions that the gateway
enforces.

### 4.1 Gateway decisions

| ID | Decision | Choice | Reason |
|---|---|---|---|
| GW-001 *(Example)* | API shape | One chat and embeddings API close to the most common provider format, with adapters per provider | Product teams change provider without code change |
| GW-002 *(Example)* | Credentials | Provider keys held only in the gateway secret store; workloads use workload identity | No key sprawl; one rotation point |
| GW-003 *(Example)* | Routing | Policy-based: data class and region first, then cost and latency tier | Data residency is a hard constraint; cost is a preference |
| GW-004 *(Example)* | Fallback | Secondary provider for tier 1 workloads only, with an evaluation run that proves quality parity | Fallback to an untested model is a silent quality change |
| GW-005 *(Example)* | Caching | Exact cache on; semantic cache off by default, allowed per use case after review | A semantic cache can return one user's answer to another |
| GW-006 *(Example)* | Streaming | Token streaming with output guardrails on rolling windows | Users expect streaming; guardrails must still apply |
| [GW-NNN] | [Decision] | [Choice] | [Reason] |

### 4.2 Model catalogue

| Model ID | Provider and hosting | Approved data classes | Regions | Approved uses | Evaluation record | Review date |
|---|---|---|---|---|---|---|
| MDL-001 *(Example)* | [Hosted provider A], provider region EU | Public, Internal, Confidential | EU | Chat, summarisation, agents | [EVAL-2026-014] | YYYY-MM-DD |
| MDL-002 *(Example)* | [Self-hosted open-weight model], own cluster | Up to Restricted | EU, on premises | Classification, extraction | [EVAL-2026-021] | YYYY-MM-DD |
| MDL-003 *(Example)* | [Hosted embeddings model] | Public, Internal, Confidential | EU | Embeddings | [EVAL-2026-009] | YYYY-MM-DD |
| [MDL-NNN] | [Provider] | [Classes] | [Regions] | [Uses] | [Record] | YYYY-MM-DD |

Approve a model through these steps: intake with the provider's model documentation and terms;
vendor risk review with the AI vendor risk questionnaire
(`compliance/ai-vendor-risk-questionnaire`); data processing and residency review; evaluation on the
platform's standard suites and on the first use case's suite; entry in the catalogue with an owner
and a review date. Re-run the evaluation whenever the provider announces a new model version behind
the same name, and record the provider's deprecation dates so that dependent workloads migrate in
time.

## 5. Data and retrieval layer

<!-- Guidance: Decide which enterprise content AI may read, how it is indexed and how permissions travel with it. Questions: which sources are approved? How fast does a revoked permission take effect in the index? Pitfall: indexing content under a service account and filtering nothing. -->

The retrieval layer turns approved enterprise content into permission-aware context. Its detailed
design belongs in the enterprise RAG search design (`system-design/design-enterprise-rag-search`)
and, for a single use case, the RAG system design (`ai-data/rag-system-design`).

### 5.1 Approved sources

| Source ID | System | Content | Classification | Permission model | Sync mode | Permission latency target | Owner |
|---|---|---|---|---|---|---|---|
| SRC-001 *(Example)* | Document management | Policies, procedures | Internal | Site and folder groups | Incremental, 15 min | 15 min | [@records-lead] |
| SRC-002 *(Example)* | ITSM knowledge base | Knowledge articles | Internal | Knowledge base user criteria | Incremental, 30 min | 30 min | [@itsm-owner] |
| SRC-003 *(Example)* | CRM | Accounts, cases (selected fields) | Confidential | Record sharing rules, queried live | Live query, no index | Immediate | [@crm-owner] |
| SRC-004 *(Example)* | Data warehouse | Curated metrics through a semantic layer | Confidential | Row-level security | Live query through a governed tool | Immediate | [@data-platform-lead] |
| [SRC-NNN] | [System] | [Content] | [Class] | [Model] | [Mode] | [Target] | [@owner] |

### 5.2 Layer rules

- Index only content that has an owner, a classification and a retention rule. Content without an
  owner stays out of the index.
- Store the source access control list or group references with every chunk and filter at query time
  with the end user's identity. Prefer a live query to the source for records whose permissions
  change often.
- Delete chunks when the source document is deleted or its retention expires, and prove it with a
  reconciliation report.
- Keep structured data behind governed tools (a semantic layer or a read-only API) rather than
  copying tables into a vector index.
- Record the embedding model and chunking version on every index, so a model change triggers a
  planned re-index rather than mixed vectors.

## 6. Agent runtime, tools and protocols

<!-- Guidance: Decide where agents run, how they reach tools and other agents, and what stops them. Questions: which actions may an agent take without a human? How is a tool registered, reviewed and retired? Pitfall: letting each team connect any MCP server it finds. -->

Agents run on a shared durable runtime that checkpoints each step, can pause for human approval and
resumes after failures. Design the runtime with the agent orchestration platform design
(`system-design/design-agent-orchestration-platform`) and the behaviour of each agent system with
the multi-agent system design (`ai-data/multi-agent-system-design`).

### 6.1 Protocol choices

| Concern | Protocol and version | Platform rule |
|---|---|---|
| Agent to tool and data | Model Context Protocol, specification revision 2026-07-28 (JSON-RPC 2.0 messages; servers offer tools, resources and prompts; optional extensions such as Tasks for long-running operations) | Only MCP servers in the tool registry are reachable from the runtime; remote servers use the authorization flow defined by the specification |
| Agent to agent | Agent2Agent protocol, specification 1.0.1 (protocol version 1.0), with Agent Cards describing skills and security schemes | Remote agents are registered with their Agent Card, owner and allowed callers; cross-organisation agents need a contract and a risk review |
| Agent to platform services | Platform APIs (gateway, retrieval, evaluation) | Agents call models only through the gateway |

Both protocols are stewarded by the Agentic AI Foundation under the Linux Foundation. Check the
current revisions at modelcontextprotocol.io and a2a-protocol.org before each platform release, and
record the revision each server or agent implements.

### 6.2 Tool registry

| Tool ID | MCP server | Owner | Operations exposed | Side effects | Approval needed | Data classes | Status |
|---|---|---|---|---|---|---|---|
| TOOL-001 *(Example)* | ITSM server | [@itsm-owner] | Search incidents, create incident, add work note | Creates records | Create: none below priority 2; priority 1 and 2 need a human | Internal | Approved |
| TOOL-002 *(Example)* | Finance read server | [@finance-it] | Read invoice status, read supplier master | None | None | Confidential | Approved |
| TOOL-003 *(Example)* | Payments server | [@treasury-it] | Propose payment run | Proposes only, never releases | Always; two-person rule in the ERP | Restricted | Pilot |
| [TOOL-NNN] | [Server] | [@owner] | [Operations] | [Effects] | [Rule] | [Classes] | [Status] |

Register each tool with its permission design from the agent tool permission design
(`ai-data/agent-tool-permission-design`) and its catalogue entry in the tool catalogue
(`ai-data/tool-catalogue`). Review new MCP servers like any integration: security review, threat
model with the agentic threat model (`ai-data/agentic-threat-model`), and a pinned version.

### 6.3 Autonomy levels

| Level | Description | Allowed when |
|---|---|---|
| L0 | Suggest only: the agent drafts, a person acts | Default for new agents |
| L1 | Act with approval: each side effect needs a person to confirm | Side effects are reversible and the evaluation suite passes |
| L2 | Act within limits: side effects below a threshold run without approval, the rest need approval | Six weeks at L1 without a severity 1 or 2 incident, and an owner sign-off |
| L3 | Act and report: the agent acts and a person reviews after the fact | Only for low-impact, reversible actions listed by the AI governance board |

## 7. Evaluation and guardrails

<!-- Guidance: Make quality and safety measurable and enforced by the platform. Questions: what evaluation must pass before release? Which guardrails are mandatory for every workload, and which are per use case? Pitfall: a guardrail product switched on with default thresholds and never tested on your own traffic. -->

### 7.1 Evaluation service

The evaluation service runs three kinds of checks. Offline suites run in the release pipeline
against curated test sets. Online checks sample production traffic and score it with automated
graders and human review. Regression runs repeat the offline suites whenever a model, prompt,
retrieval index or tool version changes. Plan the suites per use case with the LLM evaluation plan
(`ai-data/llm-evaluation-plan`) and, for agents, the agent evaluation plan
(`ai-data/agent-evaluation-plan`).

| Gate ID | Applies to | Metric | Threshold | Blocking |
|---|---|---|---|---|
| EVG-001 *(Example)* | All RAG workloads | Answer groundedness on the use case test set | At least [0.90] | Yes |
| EVG-002 *(Example)* | All RAG workloads | Citation correctness (cited passage supports the claim) | At least [0.95] | Yes |
| EVG-003 *(Example)* | All agents | Task success on the scenario suite | At least [0.85] | Yes |
| EVG-004 *(Example)* | All agents | Unsafe action rate on the adversarial suite | 0 | Yes |
| EVG-005 *(Example)* | All workloads | Prompt injection resistance on the platform red-team set | At least [0.98] blocked | Yes |
| EVG-006 *(Example)* | Customer-facing workloads | Human rating on a sample of [200] conversations | Mean at least [4.0 of 5] | No, reviewed by the owner |
| [EVG-NNN] | [Scope] | [Metric] | [Threshold] | [Yes / No] |

Thresholds are set by each use case owner and approved by the AI governance board; the values above
are illustrations.

### 7.2 Guardrail baseline

| Guard | Stage | Mandatory for | Action | Specification |
|---|---|---|---|---|
| Prompt injection and jailbreak detection | Input and retrieved content | All workloads | Block or strip, log | `ai-data/guardrails-spec` |
| Personal data detection and redaction | Input, output, logs | All workloads | Redact before logging; block for restricted classes | `ai-data/guardrails-spec` |
| Grounding check | Output | RAG workloads | Warn or withhold uncited claims | `ai-data/guardrails-spec` |
| Tool-call policy | Before each tool call | All agents | Deny calls outside the agent's scope; require approval per section 6.3 | `ai-data/agent-tool-permission-design` |
| Output content policy | Output | Customer-facing workloads | Block and fall back to a safe reply | `ai-data/guardrails-spec` |
| Spend and loop limits | Runtime | All agents | Stop the run and alert the owner | `ai-data/multi-agent-system-design` |

## 8. Identity, data permissions and security

<!-- Guidance: Define whose identity each call carries and how access is decided. Questions: does the agent act as the user, as itself, or as the user with reduced scope? How is consent recorded for delegated actions? Pitfall: one powerful service account behind every agent. -->

### 8.1 Identity model

| Actor | Identity | Credential | Authorisation decision |
|---|---|---|---|
| End user | Workforce identity from the enterprise identity provider | Session token from single sign-on | Source system permissions for data; platform roles for features |
| Workload (product app) | Workload identity per environment | Short-lived token from the platform | Gateway quota and model catalogue rules for its tenant |
| Agent | Agent identity registered in the agent registry | Short-lived token issued per run | Intersection of the agent's scope and the delegating user's permissions |
| MCP server | Service identity | Mutual TLS or OAuth client credentials | Tool registry entry and server-side checks |
| Remote A2A agent | Partner or internal agent identity | Security scheme declared in its Agent Card | Allowed-caller list and contract |

The rule is "least of both": when an agent acts for a user, it may do only what both the agent's
scope and the user's permissions allow. Pass the user context to tools as a delegated,
audience-restricted token, not as a shared secret.

### 8.2 Security controls

| ID | Control | Implementation |
|---|---|---|
| SEC-001 *(Example)* | Network isolation of model traffic | Private connectivity to providers where offered; egress allow-list from the runtime |
| SEC-002 *(Example)* | Secrets out of prompts and code | Gateway holds provider keys; tools fetch secrets at call time from the vault |
| SEC-003 *(Example)* | Untrusted content handling | Retrieved documents and tool outputs are marked as data and passed through injection guards |
| SEC-004 *(Example)* | Sandboxed execution | Generated code runs only in the sandbox service with no network by default |
| SEC-005 *(Example)* | Supply chain | AI bill of materials per workload (`ai-data/ai-bom`); pinned MCP server versions |
| SEC-006 *(Example)* | Threat modelling | Each agent system threat-modelled against the OWASP Top 10 for Agentic Applications 2026 in `ai-data/agentic-threat-model` |
| [SEC-NNN] | [Control] | [Implementation] |

## 9. Observability and cost

<!-- Guidance: Make every model call, retrieval and tool call traceable and attributable to a tenant and a use case. Questions: who may read captured prompts? How is cost allocated when one agent run spans several models? Pitfall: capturing full prompts by default and creating an unmanaged store of personal data. -->

### 9.1 Telemetry

Emit traces, metrics and events using the OpenTelemetry semantic conventions for generative AI. They
have Development status and are maintained in the `semantic-conventions-genai` repository, so pin a
commit or tag and review changes on a schedule. Specify attributes, spans, retention and prompt
capture rules in the LLM observability specification (`ai-data/llm-observability-spec`). Every span
carries the tenant (business unit), use case ID, AI system inventory ID, model ID and, for agents,
the run ID.

| Signal | Minimum content | Retention | Access |
|---|---|---|---|
| Traces | Model calls, retrieval, tool calls, agent steps, guardrail decisions | [30 days] | Platform and product engineers |
| Metrics | Tokens, latency, errors, cost, guardrail triggers, evaluation scores | [13 months] | All tenants for their own data |
| Prompt and completion capture | Off by default; sampled and redacted where a use case approves it | [30 days] | Named reviewers only, access logged |
| Audit events | Approvals, autonomy changes, model catalogue changes, exceptions | [7 years or per retention schedule] | Governance and audit |

### 9.2 Cost management and showback

Meter cost at the gateway per request and attribute it to tenant and use case. Report cost in the
FinOps format the organisation uses (for example FOCUS 1.4 columns) so AI spend appears next to
cloud spend. Model unit economics per use case with the inference cost model
(`ai-data/inference-cost-model`).

*Example* monthly showback, using illustrative list prices of USD 2.50 per million input tokens and
USD 10.00 per million output tokens, and a 30-day month:

| Tenant | Requests per day | Input tokens per request | Output tokens per request | Input cost per day | Output cost per day | Cost per month |
|---|---|---|---|---|---|---|
| Retail *(Example)* | 40,000 | 2,500 | 400 | USD 250 | USD 160 | USD 12,300 |
| Finance *(Example)* | 5,000 | 6,000 | 800 | USD 75 | USD 40 | USD 3,450 |
| Engineering *(Example)* | 20,000 | 4,000 | 1,000 | USD 200 | USD 200 | USD 12,000 |
| Total | 65,000 | | | USD 525 | USD 400 | USD 27,750 |

Check the arithmetic: Retail uses 40,000 x 2,500 = 100 million input tokens (USD 250) and 40,000 x
400 = 16 million output tokens (USD 160) per day, so USD 410 per day and USD 12,300 per month. The
three tenants together cost USD 925 per day. Retail and Engineering together are about 88 % of the
total, so their caching and model-tier choices matter most.

Set a monthly budget per tenant with alerts at [80 %] and [100 %], and a hard stop only for
non-production environments. Review the top ten use cases by cost each month with the FinOps cost
review (`ops/finops-cost-review`).

## 10. Governance hooks

<!-- Guidance: Show where the platform produces the evidence that governance needs, so that compliance is a by-product of using the platform. Questions: which records does each regulation or management-system clause need, and which platform component produces them? Pitfall: a separate governance spreadsheet that drifts from what runs in production. -->

### 10.1 AI system lifecycle hooks

```mermaid
flowchart LR
  A["Use case intake"] --> B["Risk classification"] --> C["Design and build on platform"] --> D["Evaluation gate"]
  D --> E["Launch review"] --> F["Production with monitoring"] --> G["Periodic review or retirement"]
  F --> H["Incident"] --> C
```

| Hook | Platform evidence produced | Template |
|---|---|---|
| Intake | Inventory record with owner, purpose, data, users | `ai-data/ai-use-case-intake` |
| Risk classification | Regulatory applicability and risk tier | `compliance/ai-regulatory-applicability-register`, `ai-data/ai-risk-assessment` |
| Impact assessment | Impact on people and groups, where required | `ai-data/ai-impact-assessment-42005`, `ai-data/ai-act-fria` |
| Evaluation gate | Evaluation results stored against the release | Section 7.1 |
| Launch review | Sign-off record with conditions | `ai-data/ai-launch-review` |
| Human oversight | Oversight design and autonomy level | `ai-data/human-oversight-design` |
| Monitoring | Dashboards, drift and incident records | `ai-data/model-monitoring-plan`, `ai-data/ai-incident-report` |
| Transparency | System card and user disclosures | `ai-data/ai-system-card`, `ai-data/ai-content-transparency-plan` |

### 10.2 ISO/IEC 42001:2023 mapping

ISO/IEC 42001:2023 specifies an AI management system. The platform supports, but does not replace,
the organisation's management system (`compliance/iso-42001-ai-management`). Only clause and Annex A
numbers are cited below; obtain the standard for the requirements.

| ISO/IEC 42001 reference | Topic in brief | Platform contribution |
|---|---|---|
| Clause 6.1 | Planning for AI risks and impacts | Risk classification hook and risk register links |
| Clause 8 | Operation of AI processes and assessments | Release pipeline, evaluation gate, change records |
| Clause 9.1 | Monitoring and measurement | Metrics, evaluation scores and dashboards from section 9 |
| Annex A.4 | Resources for AI systems | Model catalogue, data sources, tool registry as resource records |
| Annex A.6 | AI system life cycle | Lifecycle hooks in section 10.1 |
| Annex A.7 | Data for AI systems | Source register in section 5.1 with provenance and classification |
| Annex A.8 | Information for interested parties | System cards and disclosure records |
| Annex A.9 | Use of AI systems | Autonomy levels, usage policy enforcement, logging |
| Annex A.10 | Third-party and customer relationships | Model provider and MCP server vendor reviews |

Record applicability decisions in the statement of applicability
(`compliance/iso-42001-statement-of-applicability`).

### 10.3 EU AI Act hooks

Dates below follow Regulation (EU) 2024/1689 as amended by Regulation (EU) 2026/1744. Check the
consolidated text on EUR-Lex before relying on them.

| Article | Duty in brief | Applies from | Platform hook |
|---|---|---|---|
| Article 4 | Measures that support AI literacy of staff (wording amended by 2026/1744) | 2 February 2025 | Literacy plan (`ai-data/ai-literacy-plan`) linked to platform onboarding |
| Article 5 | Prohibited practices (with prohibitions added by 2026/1744 applying from 2 December 2026) | 2 February 2025 | Intake screening question that blocks prohibited use cases |
| Article 26 | Deployer duties for high-risk systems: use per instructions, human oversight, monitoring, log keeping | 2 December 2027 for Annex III systems; 2 August 2028 for Annex I systems | Oversight design, log retention, monitoring dashboards |
| Article 27 | Fundamental rights impact assessment for certain deployers | 2 December 2027 for the Annex III systems it covers | FRIA record linked to the inventory |
| Article 50 | Transparency: providers design systems that tell people they interact with AI (50(1)) and mark synthetic content (50(2)); deployers disclose deep fakes and emotion recognition or biometric categorisation (50(3), 50(4)) | 2 August 2026; Article 50(2) marking from 2 December 2026 for generative systems placed on the market before 2 August 2026 | Disclosure components and content marking at the gateway |
| Articles 53 and 55 | Duties of general-purpose AI model providers | 2 August 2025 | Applies only if the organisation itself provides a GPAI model; otherwise collect the provider's documentation in the model catalogue |

Map other jurisdictions in the regulatory applicability register. When the platform fine-tunes or
substantially modifies a model, check whether the organisation becomes a provider for that model or
system.

## 11. Tenancy for business units

<!-- Guidance: Decide how business units share the platform without sharing data, quotas or blame. Questions: which resources are pooled and which are dedicated? Can one unit's spike starve another? Pitfall: tenancy only in the billing report, not in the data path. -->

Treat each business unit, and where needed each legal entity, as a tenant. Apply the patterns of the
multi-tenancy design (`architecture/multi-tenancy-design`) to the AI platform.

| Resource | Model | Isolation mechanism | Notes |
|---|---|---|---|
| Gateway | Pooled | Tenant ID in every request; per-tenant quotas and rate limits | Priority classes protect tier 1 workloads |
| Model capacity | Pooled, with reserved capacity for tier 1 | Reserved throughput assigned per tenant | Reserved capacity is a cost decision per tenant |
| Retrieval indexes | Bridge: shared service, index per tenant | Separate indexes and encryption keys per tenant | Cross-tenant search only through an approved federation tool |
| Agent runtime | Pooled | Namespaces per tenant; run budgets per tenant | Tenant-specific tools visible only in that tenant |
| Prompt capture store | Silo | Separate store and keys per tenant | Required where works council or local law demands it |
| Evaluation data | Silo | Per-tenant datasets and access groups | Test sets may contain confidential data |

| Tenant | Legal entity | Data residency | Tier | Reserved capacity | Monthly budget | Platform contact |
|---|---|---|---|---|---|---|
| Retail *(Example)* | [Entity A] | EU | 1 | [Units] | USD 14,000 | [@retail-ai-lead] |
| Finance *(Example)* | [Entity A] | EU | 1 | [Units] | USD 4,000 | [@finance-ai-lead] |
| Engineering *(Example)* | [Entity B] | EU and US | 2 | None | USD 13,000 | [@eng-ai-lead] |
| [Tenant] | [Entity] | [Region] | [Tier] | [Units] | USD [amount] | [@contact] |

## 12. Build, buy or reuse

<!-- Guidance: Decide per capability whether to build, buy a product, use a cloud provider service or reuse an existing internal service. Questions: which capabilities differentiate the organisation? Where is lock-in acceptable? Pitfall: buying an all-in-one suite that overlaps the gateway, the agent runtime and the vendor copilots at once. -->

| Capability | Options considered | Decision | Lock-in exit path | Review trigger |
|---|---|---|---|---|
| LLM gateway *(Example)* | Open-source gateway, cloud provider gateway, commercial product, build | Open-source gateway with internal extensions | Gateway API is internal; adapters can move | A requirement the gateway cannot meet in two releases |
| Retrieval *(Example)* | Cloud search service, vector database product, extend existing enterprise search | Cloud search service with hybrid retrieval | Index can be rebuilt from sources | Permission latency target missed |
| Agent runtime *(Example)* | Cloud agent service, workflow engine plus agent library, commercial agent platform | Workflow engine plus agent library | MCP and A2A keep tools and agents portable | Need for features the engine lacks |
| Guardrails *(Example)* | Cloud content safety service, open-source guard models, commercial product | Combination: cloud service for content, open-source models for injection | Guard decisions logged in a neutral schema | False-positive budget exceeded |
| Evaluation *(Example)* | Open-source framework, commercial evaluation product | Open-source framework with internal test sets | Test sets are owned data | Scale or reviewer workflow needs |
| [Capability] | [Options] | [Decision] | [Exit path] | [Trigger] |

Record each significant choice with the build vs buy analysis (`architecture/build-vs-buy-analysis`)
and an architecture decision in the decision log (`architecture/architecture-decision-log`).

## 13. Operating model

<!-- Guidance: Say who does what between the platform team, product teams, governance and security. Pitfall: a platform team that becomes a ticket queue for every prompt change. -->

R = responsible, A = accountable, C = consulted, I = informed.

| Activity | Platform team | Product team | AI governance | Security | Data owner |
|---|---|---|---|---|---|
| Add a model to the catalogue | R | C | A | C | I |
| Onboard a data source | R | C | I | C | A |
| Register an MCP server | C | R | I | A | C |
| Set evaluation thresholds for a use case | C | R | A | I | C |
| Raise an agent's autonomy level | C | R | A | C | I |
| Respond to an AI incident | R | R | A | R | C |
| Approve an exception to the paved road | C | R | A | C | I |
| Publish monthly showback | A | I | I | I | I |

Publish platform service levels per service (*Example:* gateway 99.9 % monthly availability with
under [50 ms] added latency at the 95th percentile and 24x7 support; retrieval and agent runtime
99.5 % with business-hours support plus on-call) and review them with the tenants every quarter.

## 14. Roadmap

<!-- Guidance: Sequence the platform in increments that each deliver value to a named workload. Pitfall: building every layer before the first use case ships. -->

| Increment | Capabilities delivered | First workloads | Exit criteria | Target date |
|---|---|---|---|---|
| R1 *(Example)* | Gateway, model catalogue, basic telemetry and showback | Drafting assistant (Retail), code review assistant (Engineering) | 80 % of model calls via gateway; monthly showback live | YYYY-MM-DD |
| R2 *(Example)* | Retrieval with permission filtering, guardrail baseline, evaluation gate | Policy question answering (all tenants) | EVG-001 and EVG-002 pass; permission latency targets met | YYYY-MM-DD |
| R3 *(Example)* | Agent runtime, tool registry, autonomy levels L0 and L1 | IT service desk agent | No severity 1 incident in pilot; audit events complete | YYYY-MM-DD |
| R4 *(Example)* | A2A federation, sandbox execution, L2 autonomy | Finance close assistant | Governance board approves L2 for listed actions | YYYY-MM-DD |
| [R-N] | [Capabilities] | [Workloads] | [Criteria] | YYYY-MM-DD |

## 15. Risks and traceability

<!-- Guidance: Keep the risks that could undo the platform and trace drivers to capabilities and evidence. Record decisions in the decision log, not here. Pitfall: risks without an owner or a date. -->

### 15.1 Risks

| ID | Risk | Likelihood | Impact | Mitigation | Owner |
|---|---|---|---|---|---|
| EAP-R-001 *(Example)* | Teams bypass the gateway with direct provider accounts | High | High | Block provider endpoints at egress; procurement routes new AI contracts to the platform | [@ai-platform-lead] |
| EAP-R-002 *(Example)* | Index exposes content to users who lost access | Medium | High | Permission latency monitoring; live queries for volatile sources | [@data-platform-lead] |
| EAP-R-003 *(Example)* | Protocol revisions break registered MCP servers | Medium | Medium | Pin revisions; compatibility tests in the tool registry pipeline | [@agent-runtime-lead] |
| EAP-R-004 *(Example)* | Provider retires a model with short notice | Medium | Medium | Catalogue tracks deprecation dates; second model pre-evaluated for tier 1 | [@ai-platform-lead] |
| EAP-R-005 *(Example)* | Regulatory dates or obligations change again | Medium | Medium | Quarterly review of the applicability register | [@ai-governance-lead] |
| [EAP-R-NNN] | [Risk] | [L] | [I] | [Mitigation] | [@owner] |

### 15.2 Traceability matrix

| Driver | Principles | Capabilities | Evidence |
|---|---|---|---|
| EAP-DRV-001 *(Example)* | EAP-P-001, EAP-P-006 | CAP-02, CAP-03 | Gateway share metric in monthly report |
| EAP-DRV-002 *(Example)* | EAP-P-005 | CAP-05, CAP-06, CAP-10, CAP-11 | Lead time per use case from intake to launch review |
| EAP-DRV-003 *(Example)* | EAP-P-003, EAP-P-004 | CAP-15 | Inventory completeness check |
| EAP-DRV-004 *(Example)* | EAP-P-001 | CAP-14 | Showback variance report |
| [EAP-DRV-NNN] | [Principles] | [Capabilities] | [Evidence] |

## 16. Glossary and references

<!-- Guidance: Define the terms readers outside the platform team need, and list the versions of every protocol and law this document assumes. -->

### 16.1 Glossary

| Term | Meaning in this document |
|---|---|
| Agent | Software that uses a model to plan and take steps, calling tools or other agents |
| Agent Card | A2A document in which an agent describes its skills, endpoint and security schemes |
| Gateway | Single internal API through which workloads reach models |
| MCP server | Service that exposes tools, resources or prompts to AI applications over MCP |
| Paved road | Supported default path that meets platform controls without extra approval |
| Permission latency | Time between a permission change in a source system and its effect on AI answers |
| Showback | Reporting cost to a consumer without an internal charge |
| Tenant | Business unit or legal entity whose data, quotas and cost are isolated on the platform |

### 16.2 References

| Reference | Version or date checked | Use |
|---|---|---|
| Model Context Protocol specification, modelcontextprotocol.io | Revision 2026-07-28, checked 2026-10-23 | Section 6 |
| Agent2Agent (A2A) Protocol Specification, a2a-protocol.org and github.com/a2aproject/A2A | 1.0.1 (GitHub release v1.0.1, 28 May 2026, a patch release of 1.0.0; the specification site still labels 1.0.0 as the latest released version; protocol version string `1.0`), checked 2026-10-23 | Section 6 |
| OpenTelemetry semantic conventions for generative AI, github.com/open-telemetry/semantic-conventions-genai | Development status, no tagged release seen 2026-10-23 | Section 9 |
| ISO/IEC 42001:2023, Artificial intelligence: Management system | Edition 1, 2023 | Section 10.2 |
| Regulation (EU) 2024/1689 (AI Act) as amended by Regulation (EU) 2026/1744, EUR-Lex | Consolidated text, check before use | Section 10.3 |
| OWASP Top 10 for Agentic Applications 2026, genai.owasp.org | December 2025 | Section 8 |
| FinOps Open Cost and Usage Specification (FOCUS) | 1.4 | Section 9.2 |
| [Internal AI policy] | [Version] | [Use] |

## Standard and licence note

Based on the public Model Context Protocol specification (revision 2026-07-28), the Agent2Agent
protocol specification 1.0.1, the OpenTelemetry semantic conventions for generative AI (Development
status) and Regulation (EU) 2024/1689 as amended by Regulation (EU) 2026/1744 (EU legislative texts
are free to reuse); the ISO/IEC 42001:2023 mapping is structure aligned with that standard and cites
clause and Annex A numbers only, with no text reproduced. The architecture, tables and wording are
original to this collection. Check the current official versions and the consolidated legal text
before use. The regulatory rows are a documentation format, not legal advice. Product and protocol
names are trademarks of their owners; this template is not affiliated with or endorsed by them.
Template licence: MIT.