System Design: [Customer support AI agent name]

Reference design for an AI agent that handles customer support conversations across chat, messaging and email: it identifies intent, answers from the knowledge base, takes bounded actions such as refunds with limits and approvals, hands off to human agents with full context, and records every decision for audit. It covers channels, intent and routing, knowledge retrieval, tool actions, human handoff, conversation state, deflection and CSAT metrics, safety and escalation, and audit. Oversight follows the four dimensions of Singapore's Model AI Governance Framework for Agentic AI (IMDA, launched 22 January 2026; current Version 1.5, published 20 May 2026, updated 5 June 2026); threats are mapped to the OWASP Top 10 for LLM Applications 2026. It follows the 12-section skeleton of the System Design Document template and is pre-filled with a worked design for an online retailer to adapt. Replace every [bracketed] value and every row marked Example; all numbers are illustrative examples, not benchmarks. This is a documentation format, not legal advice.

Document control

Approvals

Revision history

Contents

1. Overview
2. Requirements
3. Capacity estimation
4. API design
5. Data model
6. High-level design
7. Detailed design
8. Scalability and performance
9. Reliability, security and operations
10. Trade-offs and alternatives
11. Delivery plan and risks
12. Appendices

1. Overview

1.1 Problem statement

[Example: The support team handles 1.2 million conversations a month. About 60% are order status, returns and simple refunds that follow written policy. Median first response time in peak season is 9 hours, CSAT has fallen for three quarters and seasonal hiring cannot keep up. A rule-based chatbot answers only 12% of conversations and frustrates customers who cannot reach a person.]

1.2 Proposed solution

[Example: An AI agent on every text channel that identifies the customer, classifies intent, answers from the versioned policy and help-centre knowledge base with citations, and calls order system tools to check status, change addresses, issue return labels and refund. A deterministic policy engine approves refunds within set limits and sends larger ones to a human approval queue. The agent hands off to a human whenever the customer asks, when confidence or policy coverage is low, or when safety signals appear, passing a summary so the customer never repeats themselves. Every model turn, tool call and decision is logged for audit.]

The agent never: [Example: changes payment methods, issues refunds above its limit without approval, makes promises not in published policy, discusses legal claims, or hides that it is an AI.]

1.3 Decisions requested from reviewers

1.4 45-minute interview path

2. Requirements

2.1 Core functional requirements

2.2 Non-functional requirements

2.3 Below the line (out of scope)

• [Example: voice calls (see Real-Time Voice Agent System Design (system-design/design-voice-agent))]
• [Example: outbound proactive messages and marketing]
• [Example: complaints that are formal regulatory complaints; routed to the complaints team without agent handling]
• [Example: agent-initiated changes to payment instruments or account ownership]
• [Item deferred, with the release that will take it]

3. Capacity estimation

3.1 Assumptions

All values are Example planning inputs.

3.2 Conversation load

Per day: 1.2 million / 30 = 40 000 conversations. Busiest hour: 4 000 conversations. Concurrent conversations at that hour: 4 000 x 8 / 60, about 533.

Consequence: the orchestrator must hold conversation state outside the process (533 concurrent at peak, many idle between customer turns), so any instance can serve the next turn.

3.3 Model tokens and cost

Model calls per day: 40 000 x 6 x 2 = 480 000. Input tokens: 480 000 x 8 000 = 3.84 billion; output: 480 000 x 250 = 120 million. At Example prices of USD 2 per million input and USD 8 per million output: 7 680 + 960 = USD 8 640 per day, about USD 0.22 per conversation.

Consequence: model cost is small next to human cost (USD 4 per conversation). Optimise for correct resolution and safe handoff, not for token savings; use a larger model where it improves policy accuracy.

3.4 Human handoff load

Handoffs per day at 50%: 20 000. Busiest hour: 2 000 handoffs. Human agents needed in that hour: 2 000 x 10 / 60, about 333 concurrent, before occupancy and shrinkage factors.

Consequence: the handoff rate drives staffing more than any other number. Workforce planning uses the agent's forecast handoff rate by intent, and the handoff queue must expose estimated wait time so the agent can tell the customer.

3.5 Refund actions

Refund requests per day: 40 000 x 0.08 = 3 200. Automatic: 2 240. Sent for approval: 960, about 96 per hour over a 10-hour approval window.

Consequence: finance needs an approval queue staffed for about 100 decisions per hour with a 2-hour target, and the payment system must accept idempotency keys so a retried tool call never pays twice.

3.6 Summary

4. API design

4.1 Endpoints

4.2 Tool contract example

The model fills the proposal; the orchestrator adds identity and the idempotency key and passes it to the policy engine, never directly to the payment system.

{"tool": "propose_refund", "arguments": {"order_id": "[order id]", "items": ["[sku]"], "amount": 34.90, "currency": "USD", "reason_code": "item_damaged", "evidence": ["photo_uploaded"]}}

{"decision": "auto_approved", "policy_rule": "RF-02", "policy_version": "[version]", "refund_id": "[id]", "idempotency_key": "[conversation id]:[order id]:[sku]"}

4.3 Errors and outcomes

5. Data model

5.1 Entities

The conversation is the unit of state; actions and approvals hang off it and reference the policy version that allowed them.

5.2 Access patterns

5.3 Storage choices

6. High-level design

7. Detailed design

7.1 Turn flow

One refund conversation that ends with a handoff request. The model never calls the refund service; the policy engine does, after its checks.

7.2 Deep dive 1: tool actions with limits and approvals

Choice: C. Reason: money decisions need deterministic, testable rules with an owner; the model is good at understanding the request and gathering evidence, not at enforcing limits.

The policy engine reads the customer's 30-day total and records the new action in one database transaction that locks the customer's limit row, so two concurrent proposals (for example, from chat and email at once) cannot both pass RF-02.

Tool permission design follows Agent Tool Permission Design Template (ai-data/agent-tool-permission-design). Read tools (order status) need verified identity; write tools (address change, cancellation, refund) go through the policy engine; no tool can change payment methods.

7.3 Deep dive 2: human handoff and conversation state

Choice: B at launch, C later. Reason: B fixes the main complaint (repeating details) and the summary is checked against structured facts from tools, not only model text.

Mandatory handoff triggers:

Conversation states: bot_active, awaiting_customer, awaiting_approval, queued_for_human, human_active, closed. Only one actor (bot or human) may write to the customer at a time; the state machine enforces it so the bot never replies after a human has taken over.

7.4 Deep dive 3: grounding on versioned policy

Choice: C. Each policy chunk carries policy_id, version, effective_from, locale and market. Retrieval filters by market and date; the answer cites the policy link; the policy version is logged with the turn. When retrieval finds no policy for a question about rights or money, the agent says it will check and hands off (HO-05) rather than improvising. Retrieval design follows RAG System Design Template (ai-data/rag-system-design).

7.5 Safety, disclosure and oversight

• AI disclosure. The first message states that the customer is talking with an AI assistant and how to reach a person. In the EU, Article 50(1) of the AI Act (Regulation (EU) 2024/1689) requires that people are informed when they interact with an AI system unless it is obvious; it applies from 2 August 2026 (check the consolidated text). See AI Content Transparency Plan Template (ai-data/ai-content-transparency-plan).
• Guardrails. Input and output checks (personal data, abusive content, off-topic, promises outside policy) per Guardrails Specification Template (ai-data/guardrails-spec).
• Automated decisions. Where a refusal is fully automated and has a significant effect on the customer, check data protection rules on automated decision-making for each market and offer human review.
• Oversight. The table maps the design to the four dimensions of the Model AI Governance Framework for Agentic AI (IMDA, Singapore, launched 22 January 2026), paraphrased. Detailed oversight design belongs in Human Oversight Design Template (ai-data/human-oversight-design).

7.6 How real systems do it

How real systems do it (paraphrased from Moffatt v. Air Canada, 2024 BCCRT 149, British Columbia Civil Resolution Tribunal, 14 February 2024). A customer relied on an airline website chatbot that described a bereavement fare refund process that differed from the airline's published policy page. The airline argued it was not responsible for what the chatbot said. The tribunal disagreed: the chatbot was part of the airline's website, the airline was responsible for all information on it, and the customer had no reason to think one part of the site was more reliable than another. It found negligent misrepresentation and ordered the airline to pay the difference in fare plus interest and fees. Design lesson: the organisation owns what its agent says. Ground answers on the current policy version, cite it, and hand off when no policy applies (7.4).

8. Scalability and performance

Chat reply latency budget (target p95 6 s, Example):

Send a typing indicator on channels that support it while tools run.

9. Reliability, security and operations

9.1 Failure-mode table

9.2 Security

Payment card data never enters the model context: card fields in channels are masked by the adapter, and refunds return to the original payment method by reference.

9.3 Metrics

Define each metric so it cannot be improved by hiding problems.

Observability follows LLM and Agent Observability Specification Template (ai-data/llm-observability-spec); evaluation follows Agent Evaluation Plan Template (ai-data/agent-evaluation-plan).

9.4 Audit

Each turn record holds the customer message ID, model and prompt version, retrieved policy IDs and versions, tool calls with arguments and results, policy engine rule and decision, approver for approved actions, and the reply sent. Records are write-once and retained for [period]. Weekly sampling by quality assurance uses the same scorecard as for human agents.

10. Trade-offs and alternatives

11. Delivery plan and risks

11.1 Milestones

11.2 Rollout and rollback

[Example: Shadow mode first: the agent drafts replies and actions that humans send or discard. Then a share of traffic by intent, starting with order status. Any increase in false resolution or complaints above threshold switches that intent back to humans by configuration.]

11.3 Risks

12. Appendices

12.1 Traceability

12.2 Open questions

12.3 Glossary

12.4 References

Standard and licence note

Based on the Model AI Governance Framework for Agentic AI published by Singapore's Infocomm Media Development Authority (launched 22 January 2026), Article 50 of Regulation (EU) 2024/1689, and the OWASP Top 10 for LLM Applications 2026 by the OWASP GenAI Security Project (https://genai.owasp.org/), licensed under Creative Commons Attribution-ShareAlike 4.0 (https://creativecommons.org/licenses/by-sa/4.0/). The framework's dimensions and the tribunal decision are paraphrased in original words. The OWASP IDs and short names are used for the mapping in 9.2; all descriptions and controls are original text. Because that mapping adapts the OWASP structure, the adapted structure is shared under CC BY-SA 4.0 and any further adaptation must carry the same licence. This is a documentation format, not legal advice. Check the current official version of each framework, law and list before use. Apart from the adapted OWASP structure, the original text is under the template licence. Template licence: MIT.