Generative AI Well-Architected Review: [Workload name]
Review record for one generative AI workload (an assistant, a RAG
application, an LLM-augmented workflow or an agent) organised by the
six well-architected pillars: operational excellence, security,
reliability, performance efficiency, cost optimisation and
sustainability. The question sets are original to this collection and
are mapped to the focus areas of the AWS Well-Architected Generative AI
Lens (lens version of 19 November 2025, which added an agentic AI
section and updated responsible AI guidance), with cross-references to
the Microsoft Azure Well-Architected Framework AI workload guidance and
the Google Cloud Well-Architected Framework AI and ML perspective. It
ends with a findings register (High and Medium risk), an improvement
plan and a re-review cadence. Replace every [bracketed] value and
every row marked Example.
Document control
| Field | Value |
|---|
| Document ID | [GWAR-001] |
| Workload and version | [Workload name] [v1.4] |
| Foundation model(s) and versions | [Provider, model ID, version or snapshot date] |
| Hosting | [Managed model API / self-hosted model / mixed] |
| Review type | [Initial / Periodic / Pre-launch / After model change / After incident] |
| Review date | YYYY-MM-DD |
| Version | [0.1] |
| Status | [Draft / In review / Approved] |
| Owner | [@workload-owner] |
| Approvers/reviewers | [@lead-reviewer], [@ml-engineer], [@security-engineer], [@responsible-ai-lead] |
| Lens versions used | [AWS Generative AI Lens, 19 Nov 2025], [Azure WAF AI workloads, checked YYYY-MM-DD], [Google WAF AI and ML perspective, checked YYYY-MM-DD] |
| Classification | [Internal / Confidential] |
| Last updated | YYYY-MM-DD |
| Next review | YYYY-MM-DD |
Approvals
| Role | Name | Decision | Date |
|---|
| Workload owner | [@name] | [Accept plan / Changes requested] | YYYY-MM-DD |
| Lead reviewer | [@name] | [Decision] | YYYY-MM-DD |
| Security | [@name] | [Decision] | YYYY-MM-DD |
| Responsible AI or AI governance | [@name] | [Decision] | YYYY-MM-DD |
Revision history
| Version | Date | Author | Change summary |
|---|
| 0.1 | YYYY-MM-DD | [@name] | Pre-review questionnaire completed by the team |
| 0.2 | YYYY-MM-DD | [@name] | Example: Ratings after review session; 3 High, 2 Medium findings |
| 1.0 | YYYY-MM-DD | [@name] | Example: Improvement plan approved |
Contents
1. Review scope and method
1.1 Scope
| In scope | Out of scope |
|---|
| Example: Customer support assistant, its retrieval pipeline, guardrails, tools and evaluation harness in production | General hosting and network review (covered by the cloud architecture review) |
| Example: Fine-tuned classification model used for routing | Foundation model training by the provider |
| [Item] | [Item] |
1.2 Method
- Pre-review questionnaire completed by the team on YYYY-MM-DD.
- Review session of [3 hours] on YYYY-MM-DD with [participants].
- Evidence inspected: [evaluation reports, golden data set, traces,
guardrail configuration, prompt catalogue, cost reports for the last
[3] months, incident records].
- Each question receives a current-state description, the evidence and
one rating.
1.3 Rating scale
| Rating | Meaning | Expected response |
|---|
| High risk | Gap likely to cause harmful output, data exposure, significant outage, cost overrun or compliance breach within 12 months | Action in this quarter or formal acceptance |
| Medium risk | Gap that raises impact or effort but is unlikely to cause a severe event alone | Action in the next two quarters |
| None | Practice in place with evidence, or not applicable with a stated reason | No action |
1.4 Question IDs
Questions use the prefix GWR-[pillar]-NN (OPS, SEC, REL, PERF, COST,
SUS). The column "Lens focus area" names the related focus area of the
AWS Generative AI Lens (identifiers GENOPS01 to GENSUS03), so that teams
on AWS can open the matching best practices in the Well-Architected Tool.
2. Workload context
2.1 Business and use case
| Item | Value |
|---|
| Purpose | Example: Answers customer questions about orders and returns and can create a return label |
| Users | Example: About 40,000 customers per day through web chat; 120 support agents internally |
| Business criticality | Example: Tier 2: fallback to human agents exists |
| Lifecycle stage | [Scoping / Model selection / Customisation / Development and integration / Deployment / Continuous improvement] |
| Agency level | [Single model call / LLM-augmented workflow / Autonomous agent / Hybrid] |
| Regulatory context | Example: Consumer protection, data protection; AI transparency duties for chatbots |
| Responsible AI owner | [@name] |
2.2 Architecture summary
Describe each component, the model versions, the data sources indexed
and the tools the model can call. Link to the RAG system design
(ai-data/rag-system-design) or agent specification
(ai-data/ai-agent-spec) where they exist.
2.3 Targets
| Target | Value | Measured by |
|---|
| Answer quality (Example) | Groundedness score of 0.9 or more on the golden set | Offline evaluation, weekly |
| Harmful output rate (Example) | Below [0.1 %] of sampled responses | Guardrail logs and human review |
| Latency (Example) | Time to first token under 1.5 s at p95 | Tracing |
| Availability (Example) | 99.5 % monthly | Synthetic probes |
| Cost (Example) | Under [value] per resolved conversation | Cost and usage reports |
| [Target] | [Value] | [Method] |
3. Responsible AI and agentic preamble
3.1 Responsible AI
| ID | Question | Current state | Evidence | Rating |
|---|
| GWR-RAI-01 | Which responsible AI dimensions matter most for this use case (for example fairness, explainability, privacy and security, safety, controllability, veracity and robustness, governance, transparency), and who owns each? | Example: Safety, veracity and privacy prioritised; owners named | Risk profile | None |
| GWR-RAI-02 | Is there a documented split of responsibilities between the model producer, the model provider and your team? | [State] | [Evidence] | [Rating] |
| GWR-RAI-03 | Are users told they are interacting with an AI system, and is generated content labelled where required? | [State] | [Evidence] | [Rating] |
| GWR-RAI-04 | Have harms been assessed for the affected groups, and are results reviewed before each major change? | [State] | [Evidence] | [Rating] |
Record the full risk analysis in the GenAI risk profile
(ai-data/genai-risk-profile).
3.2 Agentic scope
| ID | Question | Current state | Evidence | Rating |
|---|
| GWR-AGT-01 | Is the agency level justified by task complexity, and would a simpler workflow meet the targets? | Example: Workflow with one tool call; autonomy not needed | Design review | None |
| GWR-AGT-02 | Is every tool listed with its permissions, side effects and approval rules? | [State] | [Evidence] | [Rating] |
| GWR-AGT-03 | Are there stop conditions, step limits and time limits for every loop? | [State] | [Evidence] | [Rating] |
| GWR-AGT-04 | Are irreversible actions gated by a human or a deterministic check? | [State] | [Evidence] | [Rating] |
For agentic workloads also review against the separate AWS
Well-Architected Agentic AI Lens and record the threat analysis in the
Agentic threat model (ai-data/agentic-threat-model).
4. Operational excellence
| ID | Question | Lens focus area | Current state | Evidence | Rating |
|---|
| GWR-OPS-01 | Is functional quality evaluated on a schedule against a maintained golden data set, and are regressions blocking? | GENOPS01 Model performance evaluation | Example: Weekly run, not blocking releases | Evaluation dashboard | Medium |
| GWR-OPS-02 | Is user feedback collected, triaged and fed into evaluation sets? | GENOPS01 | [State] | [Evidence] | [Rating] |
| GWR-OPS-03 | Are all layers monitored: application, retrieval, model metrics (tokens, latency, errors, throttling) and guardrail actions? | GENOPS02 Monitor and manage operational health | [State] | [Evidence] | [Rating] |
| GWR-OPS-04 | Are there protections against overload, such as rate limits, queues and quota alarms? | GENOPS02 | [State] | [Evidence] | [Rating] |
| GWR-OPS-05 | Are prompt templates versioned, reviewed and deployed like code? | GENOPS03 Observability in workloads | [State] | [Evidence] | [Rating] |
| GWR-OPS-06 | Are agent and RAG workflows traced end to end, with the retrieved context and tool calls visible? | GENOPS03 | [State] | [Evidence] | [Rating] |
| GWR-OPS-07 | Are infrastructure, prompts, guardrails and evaluation jobs deployed through IaC and pipelines? | GENOPS04 Automate lifecycle management | [State] | [Evidence] | [Rating] |
| GWR-OPS-08 | Is there a documented decision on whether to customise a model (prompting, retrieval, fine-tuning) with evidence? | GENOPS05 Model customization | [State] | [Evidence] | [Rating] |
Specify telemetry in the LLM observability specification
(ai-data/llm-observability-spec).
5. Security
| ID | Question | Lens focus area | Current state | Evidence | Rating |
|---|
| GWR-SEC-01 | Is access to model endpoints limited to named identities with least privilege? | GENSEC01 Endpoint security | Example: Shared key used by three services | Key inventory | High |
| GWR-SEC-02 | Does traffic between the application and model endpoints stay on private network paths where offered? | GENSEC01 | [State] | [Evidence] | [Rating] |
| GWR-SEC-03 | Does the model (or orchestrator) access data stores only with the permissions of the requesting user or a narrowly scoped role? | GENSEC01 | [State] | [Evidence] | [Rating] |
| GWR-SEC-04 | Are responses filtered for harmful, incorrect or sensitive content before reaching users or tools? | GENSEC02 Response validation | [State] | [Evidence] | [Rating] |
| GWR-SEC-05 | Are control-plane and data-access events for AI services logged and monitored? | GENSEC03 Event monitoring | [State] | [Evidence] | [Rating] |
| GWR-SEC-06 | Are prompts stored in a controlled catalogue, and are user inputs sanitised and checked for injection? | GENSEC04 Prompt security | [State] | [Evidence] | [Rating] |
| GWR-SEC-07 | Are agent permissions bounded so that a manipulated model cannot exceed the intended scope? | GENSEC05 Excessive agency | [State] | [Evidence] | [Rating] |
| GWR-SEC-08 | Are training, fine-tuning and retrieval data filtered and provenance-checked to prevent poisoning? | GENSEC06 Data poisoning | [State] | [Evidence] | [Rating] |
Specify filters in the Guardrails specification (ai-data/guardrails-spec)
and test them with the AI red team plan (ai-data/ai-red-team).
6. Reliability
| ID | Question | Lens focus area | Current state | Evidence | Rating |
|---|
| GWR-REL-01 | Are throughput quotas known, monitored and sized for peak with headroom, and is load balanced across capacity? | GENREL01 Manage throughput quotas | Example: Quota hit twice last month | Throttling metrics | High |
| GWR-REL-02 | Are network paths to model endpoints and supporting services redundant? | GENREL02 Network reliability | [State] | [Evidence] | [Rating] |
| GWR-REL-03 | Does the orchestrator handle model errors, malformed output and tool failures with retries, fallbacks and user-facing messages? | GENREL03 Prompt remediation and recovery actions | [State] | [Evidence] | [Rating] |
| GWR-REL-04 | Do agentic workflows have timeouts at each step and for the whole run? | GENREL03 | [State] | [Evidence] | [Rating] |
| GWR-REL-05 | Are prompts and models pinned to versions in catalogues, with tested rollback? | GENREL04 Prompt management | [State] | [Evidence] | [Rating] |
| GWR-REL-06 | Can inference, embeddings and agent capabilities be served from more than one region or location if the target requires it? | GENREL05 Distributed availability | [State] | [Evidence] | [Rating] |
| GWR-REL-07 | Are long-running distributed jobs (fine-tuning, batch inference, indexing) checkpointed and restartable? | GENREL06 Distributed compute tasks | [State] | [Evidence] | [Rating] |
7. Performance efficiency
| ID | Question | Lens focus area | Current state | Evidence | Rating |
|---|
| GWR-PERF-01 | Is there a ground-truth data set of prompts and expected responses that reflects real traffic? | GENPERF01 Establish performance evaluation processes | [State] | [Evidence] | [Rating] |
| GWR-PERF-02 | Are latency, throughput and quality metrics collected per model and per prompt version? | GENPERF01 | [State] | [Evidence] | [Rating] |
| GWR-PERF-03 | Are model endpoints load tested at expected peak before release? | GENPERF02 Maintaining model performance | [State] | [Evidence] | [Rating] |
| GWR-PERF-04 | Are inference parameters (temperature, maximum tokens, stop sequences) set deliberately and tested? | GENPERF02 | [State] | [Evidence] | [Rating] |
| GWR-PERF-05 | Was the model selected by comparing candidates on the workload's own evaluation set? | GENPERF02 | [State] | [Evidence] | [Rating] |
| GWR-PERF-06 | Are managed services used for hosting and customisation where they meet requirements? | GENPERF03 Optimize consumption of high-performance compute | [State] | [Evidence] | [Rating] |
| GWR-PERF-07 | Are embedding models, vector dimensions and index settings tested for retrieval quality and latency? | GENPERF04 Vector store optimization | [State] | [Evidence] | [Rating] |
Record evaluation design in the LLM evaluation plan
(ai-data/llm-evaluation-plan).
8. Cost optimisation
| ID | Question | Lens focus area | Current state | Evidence | Rating |
|---|
| GWR-COST-01 | Is the smallest model that meets the quality target used, with routing for harder requests? | GENCOST01 Model selection and cost optimization | [State] | [Evidence] | [Rating] |
| GWR-COST-02 | Has the inference pricing model (on demand, provisioned, batch, self-hosted) been chosen from measured usage? | GENCOST02 Generative AI pricing model | [State] | [Evidence] | [Rating] |
| GWR-COST-03 | Are prompt length and response length controlled, and is prompt caching used where supported? | GENCOST03 Cost-aware prompting | [State] | [Evidence] | [Rating] |
| GWR-COST-04 | Are requests that should not reach the model filtered cheaply before inference? | GENCOST03 | [State] | [Evidence] | [Rating] |
| GWR-COST-05 | Are embedding sizes and vector store capacity sized to the retrieval need? | GENCOST04 Cost-informed vector stores | [State] | [Evidence] | [Rating] |
| GWR-COST-06 | Do agent workflows have stopping conditions and budgets per run? | GENCOST05 Cost-informed agents | Example: No step limit; one run used 140 model calls | Trace sample | High |
| GWR-COST-07 | Is cost reported per conversation, task or outcome, and reviewed in FinOps reviews? | GENCOST01 to GENCOST05 | [State] | [Evidence] | [Rating] |
Review spend with the FinOps cost review (ops/finops-cost-review).
9. Sustainability
| ID | Question | Lens focus area | Current state | Evidence | Rating |
|---|
| GWR-SUS-01 | Are auto scaling and serverless options used so that idle capacity is minimal? | GENSUS01 Energy-efficient infrastructure and services | [State] | [Evidence] | [Rating] |
| GWR-SUS-02 | Is model customisation done with efficient methods (for example parameter-efficient fine-tuning) only when justified? | GENSUS01 | [State] | [Evidence] | [Rating] |
| GWR-SUS-03 | Are data processing and storage (indexes, logs, training sets) minimised and expired by policy? | GENSUS02 Consume sustainable data processing and storage services | [State] | [Evidence] | [Rating] |
| GWR-SUS-04 | Are smaller or distilled models and optimised inference techniques used where quality allows? | GENSUS03 Consume energy efficient models | [State] | [Evidence] | [Rating] |
10. Findings register
| Finding ID | Question | Pillar | Risk | Finding | Impact | Action ID or acceptance |
|---|
| FND-001 (Example) | GWR-SEC-01 | Security | High | One shared API key gives three services full model access | Key leak allows abuse and cost spikes | ACT-001 |
| FND-002 (Example) | GWR-REL-01 | Reliability | High | Quota exhausted at peak twice in 30 days | Failed conversations, fallback overload | ACT-002 |
| FND-003 (Example) | GWR-COST-06 | Cost optimisation | High | Agent loop has no step or budget limit | Runaway cost, latency | ACT-003 |
| FND-004 (Example) | GWR-OPS-01 | Operational excellence | Medium | Evaluation runs weekly but does not block releases | Quality regressions reach users | ACT-004 |
| FND-005 (Example) | GWR-RAI-03 | Responsible AI | Medium | AI disclosure missing in the mobile channel | Transparency obligation not met | ACT-005 |
| [FND-NNN] | [GWR-ID] | [Pillar] | [High / Medium] | [Finding] | [Impact] | [ACT-NNN / RA-NNN] |
10.1 Summary by pillar
| Pillar | High | Medium | None | Not applicable |
|---|
| Responsible AI and agentic preamble | [n] | [n] | [n] | [n] |
| Operational excellence | [n] | [n] | [n] | [n] |
| Security | [n] | [n] | [n] | [n] |
| Reliability | [n] | [n] | [n] | [n] |
| Performance efficiency | [n] | [n] | [n] | [n] |
| Cost optimisation | [n] | [n] | [n] | [n] |
| Sustainability | [n] | [n] | [n] | [n] |
11. Improvement plan
| Action ID | Findings | Action | Owner | Due | Verification | Status |
|---|
| ACT-001 (Example) | FND-001 | Replace shared key with per-service identities and least-privilege policies | @platform-engineer | YYYY-MM-DD | Access review shows one role per service | Open |
| ACT-002 (Example) | FND-002 | Raise quota, add queueing and a smaller fallback model at peak | @ml-engineer | YYYY-MM-DD | Load test at 1.5x peak without throttling | Open |
| ACT-003 (Example) | FND-003 | Add step limit, run budget and timeout to the agent loop | @agent-developer | YYYY-MM-DD | Trace sample shows limits enforced | Open |
| ACT-004 (Example) | FND-004 | Make evaluation a blocking release gate | @ml-engineer | YYYY-MM-DD | Pipeline fails on regression | Open |
| ACT-005 (Example) | FND-005 | Add AI disclosure to mobile chat | @product-owner | YYYY-MM-DD | Screenshot and release note | Open |
| [ACT-NNN] | [FND-NNN] | [Action] | [@owner] | YYYY-MM-DD | [Verification] | [Open / Done] |
11.1 Risk acceptances
| Acceptance ID | Finding | Reason | Compensating control | Accepted by | Expiry |
|---|
| RA-001 (Example) | [FND-NNN] | Single-region inference accepted until the provider offers the model in a second region | Fallback to human agents within 2 minutes | @workload-owner | YYYY-MM-DD |
| [RA-NNN] | [FND-NNN] | [Reason] | [Control] | [@name] | YYYY-MM-DD |
12. Re-review cadence and sign-off
| Trigger | Re-review scope |
|---|
| Calendar (Example) | Full review every 6 months |
| Model change (new model, version or provider) (Example) | Operational excellence, security, reliability, performance and cost questions |
| New tool or increase in agency (Example) | Agentic preamble, security, reliability, cost |
| New data source indexed (Example) | Security (GWR-SEC-03, GWR-SEC-08), performance (GWR-PERF-07) |
| Serious incident or harmful output event (Example) | Affected pillar plus responsible AI preamble |
| New lens version published (Example) | Update the question mapping, then full review at next cycle |
Sign-off: the workload owner confirms the improvement plan and the
acceptances; the lead reviewer confirms that the ratings reflect the
evidence; AI governance confirms the responsible AI preamble. Record
decisions in the Approvals table above. For a launch decision, attach
this review to the AI launch review (ai-data/ai-launch-review).
13. Framework cross-reference
13.1 AWS Well-Architected lenses
| Lens | Version | Use in this review |
|---|
| Generative AI Lens | 19 November 2025 (first published 15 April 2025) | Focus areas GENOPS01 to GENSUS03 mapped in sections 4 to 9; lifecycle stages in section 2.1; responsible AI and agentic patterns in section 3 |
| Agentic AI Lens | 10 June 2026 | Deeper review of autonomous and multi-agent workloads |
| Responsible AI Lens | 19 November 2025 | Deeper review of responsible AI dimensions |
13.2 Microsoft Azure Well-Architected Framework, AI workloads
The Azure guidance for AI workloads is organised in design methodology
and principles, workload personas, technical design areas (application design, application
platform, training data design, grounding data design, data platform)
and operational design areas (workload operations, MLOps and GenAIOps,
testing and evaluation, responsible AI); the grouping into technical and
operational areas is this template's own framing. The Azure framework uses five
pillars (reliability, security, cost optimisation, operational
excellence, performance efficiency) and has no separate sustainability
pillar.
| Section of this review | Azure AI workload design area |
|---|
| 2.2 Architecture | Application design, application platform |
| 3.1 Responsible AI | Responsible AI |
| 4 Operational excellence | Workload operations, MLOps and GenAIOps, testing and evaluation |
| 5 Security | Application design, data platform, responsible AI |
| 6 Reliability | Application platform, workload operations |
| 7 Performance efficiency | Grounding data design, testing and evaluation |
| 8 Cost optimisation | Application platform, data platform |
13.3 Google Cloud Well-Architected Framework, AI and ML perspective
The AI and ML perspective (as published, last updated October 2024)
gives AI-specific recommendations for operational excellence, security,
reliability, cost optimisation and performance optimisation. The
framework itself also has a sustainability pillar and names its security
pillar "Security, privacy, and compliance".
| Section of this review | Google pillar in the AI and ML perspective |
|---|
| 4 Operational excellence | Operational excellence |
| 5 Security | Security |
| 6 Reliability | Reliability |
| 7 Performance efficiency | Performance optimization |
| 8 Cost optimisation | Cost optimization |
| 9 Sustainability | Sustainability pillar of the framework (the perspective listed no sustainability page when checked on 2026-10-22) |
14. Glossary and references
14.1 Glossary
| Term | Meaning |
|---|
| Foundation model | Large pre-trained model adapted to many tasks by prompting, retrieval or fine-tuning |
| RAG | Retrieval-augmented generation: grounding model answers in retrieved documents |
| Golden data set | Curated prompts with expected responses used for repeatable evaluation |
| Guardrail | Control that filters or blocks model inputs or outputs |
| Agency | Ability of a system to choose and take actions, such as calling tools |
| Lens | Well-Architected extension with questions for a specific technology domain |
| Groundedness | Degree to which an answer is supported by the provided context |
14.2 References
| Reference | Version or date | Use |
|---|
| AWS Well-Architected Framework, Generative AI Lens, docs.aws.amazon.com/wellarchitected/latest/generative-ai-lens | 19 November 2025 | Focus area mapping |
| AWS Well-Architected Framework, Agentic AI Lens | 10 June 2026 | Agentic deep review |
| AWS Well-Architected Framework, Responsible AI Lens | 19 November 2025 | Responsible AI deep review |
| Microsoft Azure Well-Architected Framework, AI workload documentation, learn.microsoft.com/azure/well-architected/ai | Checked YYYY-MM-DD | Section 13.2 |
| Google Cloud Well-Architected Framework, AI and ML perspective, docs.cloud.google.com/architecture/framework/perspectives/ai-ml | Last updated October 2024 | Section 13.3 |
| [Internal AI policy] | [Version] | [Use] |
Standard and licence note
Structure aligned with the six pillars of the AWS Well-Architected
Framework and the focus areas of its Generative AI Lens (version of 19
November 2025), with cross-references to the Microsoft Azure
Well-Architected Framework AI workload guidance and the Google Cloud
Well-Architected Framework AI and ML perspective. Lens identifiers,
focus area, pillar and design area names belong to their publishers and
are used for mapping only. The review questions, ratings and examples
are original text written for this collection and reproduce no best
practice text from these sources. Check the current official versions
before use; lenses and perspectives are updated frequently. Template
licence: MIT.