Enterprise AI Platform Reference Architecture: [Organisation name]

Reference architecture for the shared platform on which an organisation builds, runs and governs generative AI and agent workloads: the capability map, model access through a gateway, the data and retrieval layer, the agent runtime with tools over the Model Context Protocol (MCP, specification revision 2026-07-28) and agent-to-agent calls over the Agent2Agent (A2A) protocol (specification 1.0.1, a patch release of 1.0.0; protocol version 1.0), evaluation and guardrails, identity and data permissions, observability with the OpenTelemetry semantic conventions for generative AI (Development status), cost management, governance hooks for ISO/IEC 42001:2023 and Regulation (EU) 2024/1689 (the AI Act) as amended by Regulation (EU) 2026/1744, tenancy for business units, build or buy decisions and the roadmap. It connects the detailed design templates of this library instead of repeating them. Replace every [bracketed] value and every row marked Example. The regulatory rows are a documentation format, not legal advice.

Document control

Approvals

Revision history

Contents

1. Scope, drivers and principles
2. Capability map
3. Reference architecture overview
4. Model access and gateway
5. Data and retrieval layer
6. Agent runtime, tools and protocols
7. Evaluation and guardrails
8. Identity, data permissions and security
9. Observability and cost
10. Governance hooks
11. Tenancy for business units
12. Build, buy or reuse
13. Operating model
14. Roadmap
15. Risks and traceability
16. Glossary and references

1. Scope, drivers and principles

1.1 Scope

1.2 Drivers

1.3 Principles

2. Capability map

Assess current and target maturity on a scale of 0 (absent) to 4 (managed and measured). Provider is Platform (built or run by the platform team), Vendor (bought service), Product (left to product teams) or Shared.

3. Reference architecture overview

Read the diagram with these rules. Solid arrows are request paths; dotted arrows are control and telemetry paths. Product code reaches models only through the gateway, either directly or through the agent runtime. The retrieval service and the tool registry enforce the end user's permissions, which the identity and policy service supplies. Embedded vendor copilots run their own model stack, so the platform governs them through identity, logging and evaluation (PATH-D).

3.1 Request paths

4. Model access and gateway

The gateway is the single control point for model use. It authenticates the calling workload, applies per-tenant quotas, routes to an approved model, applies guardrails, meters tokens and cost, and emits telemetry. Design it in detail with the LLM gateway design (system-design/design-llm-gateway); this section records the enterprise decisions that the gateway enforces.

4.1 Gateway decisions

4.2 Model catalogue

Approve a model through these steps: intake with the provider's model documentation and terms; vendor risk review with the AI vendor risk questionnaire (compliance/ai-vendor-risk-questionnaire); data processing and residency review; evaluation on the platform's standard suites and on the first use case's suite; entry in the catalogue with an owner and a review date. Re-run the evaluation whenever the provider announces a new model version behind the same name, and record the provider's deprecation dates so that dependent workloads migrate in time.

5. Data and retrieval layer

The retrieval layer turns approved enterprise content into permission-aware context. Its detailed design belongs in the enterprise RAG search design (system-design/design-enterprise-rag-search) and, for a single use case, the RAG system design (ai-data/rag-system-design).

5.1 Approved sources

5.2 Layer rules

• Index only content that has an owner, a classification and a retention rule. Content without an owner stays out of the index.
• Store the source access control list or group references with every chunk and filter at query time with the end user's identity. Prefer a live query to the source for records whose permissions change often.
• Delete chunks when the source document is deleted or its retention expires, and prove it with a reconciliation report.
• Keep structured data behind governed tools (a semantic layer or a read-only API) rather than copying tables into a vector index.
• Record the embedding model and chunking version on every index, so a model change triggers a planned re-index rather than mixed vectors.

6. Agent runtime, tools and protocols

Agents run on a shared durable runtime that checkpoints each step, can pause for human approval and resumes after failures. Design the runtime with the agent orchestration platform design (system-design/design-agent-orchestration-platform) and the behaviour of each agent system with the multi-agent system design (ai-data/multi-agent-system-design).

6.1 Protocol choices

Both protocols are stewarded by the Agentic AI Foundation under the Linux Foundation. Check the current revisions at modelcontextprotocol.io and a2a-protocol.org before each platform release, and record the revision each server or agent implements.

6.2 Tool registry

Register each tool with its permission design from the agent tool permission design (ai-data/agent-tool-permission-design) and its catalogue entry in the tool catalogue (ai-data/tool-catalogue). Review new MCP servers like any integration: security review, threat model with the agentic threat model (ai-data/agentic-threat-model), and a pinned version.

6.3 Autonomy levels

7. Evaluation and guardrails

7.1 Evaluation service

The evaluation service runs three kinds of checks. Offline suites run in the release pipeline against curated test sets. Online checks sample production traffic and score it with automated graders and human review. Regression runs repeat the offline suites whenever a model, prompt, retrieval index or tool version changes. Plan the suites per use case with the LLM evaluation plan (ai-data/llm-evaluation-plan) and, for agents, the agent evaluation plan (ai-data/agent-evaluation-plan).

Thresholds are set by each use case owner and approved by the AI governance board; the values above are illustrations.

7.2 Guardrail baseline

8. Identity, data permissions and security

8.1 Identity model

The rule is "least of both": when an agent acts for a user, it may do only what both the agent's scope and the user's permissions allow. Pass the user context to tools as a delegated, audience-restricted token, not as a shared secret.

8.2 Security controls

9. Observability and cost

9.1 Telemetry

Emit traces, metrics and events using the OpenTelemetry semantic conventions for generative AI. They have Development status and are maintained in the semantic-conventions-genai repository, so pin a commit or tag and review changes on a schedule. Specify attributes, spans, retention and prompt capture rules in the LLM observability specification (ai-data/llm-observability-spec). Every span carries the tenant (business unit), use case ID, AI system inventory ID, model ID and, for agents, the run ID.

9.2 Cost management and showback

Meter cost at the gateway per request and attribute it to tenant and use case. Report cost in the FinOps format the organisation uses (for example FOCUS 1.4 columns) so AI spend appears next to cloud spend. Model unit economics per use case with the inference cost model (ai-data/inference-cost-model).

Example monthly showback, using illustrative list prices of USD 2.50 per million input tokens and USD 10.00 per million output tokens, and a 30-day month:

Check the arithmetic: Retail uses 40,000 x 2,500 = 100 million input tokens (USD 250) and 40,000 x 400 = 16 million output tokens (USD 160) per day, so USD 410 per day and USD 12,300 per month. The three tenants together cost USD 925 per day. Retail and Engineering together are about 88 % of the total, so their caching and model-tier choices matter most.

Set a monthly budget per tenant with alerts at [80 %] and [100 %], and a hard stop only for non-production environments. Review the top ten use cases by cost each month with the FinOps cost review (ops/finops-cost-review).

10. Governance hooks

10.1 AI system lifecycle hooks

10.2 ISO/IEC 42001:2023 mapping

ISO/IEC 42001:2023 specifies an AI management system. The platform supports, but does not replace, the organisation's management system (compliance/iso-42001-ai-management). Only clause and Annex A numbers are cited below; obtain the standard for the requirements.

Record applicability decisions in the statement of applicability (compliance/iso-42001-statement-of-applicability).

10.3 EU AI Act hooks

Dates below follow Regulation (EU) 2024/1689 as amended by Regulation (EU) 2026/1744. Check the consolidated text on EUR-Lex before relying on them.

Map other jurisdictions in the regulatory applicability register. When the platform fine-tunes or substantially modifies a model, check whether the organisation becomes a provider for that model or system.

11. Tenancy for business units

Treat each business unit, and where needed each legal entity, as a tenant. Apply the patterns of the multi-tenancy design (architecture/multi-tenancy-design) to the AI platform.

12. Build, buy or reuse

Record each significant choice with the build vs buy analysis (architecture/build-vs-buy-analysis) and an architecture decision in the decision log (architecture/architecture-decision-log).

13. Operating model

R = responsible, A = accountable, C = consulted, I = informed.

Publish platform service levels per service (Example: gateway 99.9 % monthly availability with under [50 ms] added latency at the 95th percentile and 24x7 support; retrieval and agent runtime 99.5 % with business-hours support plus on-call) and review them with the tenants every quarter.

14. Roadmap

15. Risks and traceability

15.1 Risks

15.2 Traceability matrix

16. Glossary and references

16.1 Glossary

16.2 References

Standard and licence note

Based on the public Model Context Protocol specification (revision 2026-07-28), the Agent2Agent protocol specification 1.0.1, the OpenTelemetry semantic conventions for generative AI (Development status) and Regulation (EU) 2024/1689 as amended by Regulation (EU) 2026/1744 (EU legislative texts are free to reuse); the ISO/IEC 42001:2023 mapping is structure aligned with that standard and cites clause and Annex A numbers only, with no text reproduced. The architecture, tables and wording are original to this collection. Check the current official versions and the consolidated legal text before use. The regulatory rows are a documentation format, not legal advice. Product and protocol names are trademarks of their owners; this template is not affiliated with or endorsed by them. Template licence: MIT.