Chat System Design Template (Reference Design)

Real-time 1:1 and group chat: WebSocket gateways, per-conversation ordering, fan-out, receipts, offline sync, push, E2EE

Architektur & SystemdesignVollständigdesign chat system mathAktualisiert 20.10.2026

Im Editor öffnen.md herunterladen

Anleitung: Use when designing messaging. Change the user and message inputs in 3.1 first and recompute gateway count and storage. The main pitfall is ordering by client clocks; keep server-assigned per-conversation sequence numbers.

Vorschau

System Design: [Chat system name]

Reference design for a real-time chat system with one-to-one and group conversations, multi-device delivery, presence, receipts, offline sync and push notifications. It follows the structure of the System Design Document template and is pre-filled with a worked design so you can adapt it rather than start from a blank page. Every value marked Example is an assumption: replace it with your own numbers and re-run the formulas.

Document control

Field	Value
Document ID	[SDD-CHAT-001]
System / feature	[Chat platform: 1:1 and group messaging]
Version	[0.1]
Status	[Draft / In review / Approved / Implemented / Superseded]
Author(s)	[@author]
Reviewers	[@reviewer-1], [@reviewer-2]
Last updated	YYYY-MM-DD
Related	[PRD], [Threat model], [ADR: message store], [ADR: E2EE]

Approvals

Role	Name	Decision	Date
Tech lead	[@name]	[Approved / Changes requested]	YYYY-MM-DD
Staff / principal engineer	[@name]	[Decision]	YYYY-MM-DD
SRE / operations	[@name]	[Decision]	YYYY-MM-DD
Security and privacy	[@name]	[Decision]	YYYY-MM-DD
Mobile client lead	[@name]	[Decision]	YYYY-MM-DD

Revision history

Version	Date	Author	Change summary
0.1	YYYY-MM-DD	[@name]	First draft for design review
0.2	YYYY-MM-DD	[@name]	Example: Replaced client timestamps with server-assigned per-conversation sequence numbers
0.3	YYYY-MM-DD	[@name]	Example: Added large-group fan-out on read above 1 000 members
1.0	YYYY-MM-DD	[@name]	Example: Approved

Overview
Requirements
Capacity estimation
API design
Data model
High-level design
Detailed design
Scalability and performance
Reliability, security and operations
Trade-offs and alternatives
Delivery plan and risks
Appendices

1. Overview

1.1 Problem statement

[What problem exists today. Example: Users coordinate through email and a third-party messenger. Messages are lost between devices, there is no shared history for teams, and we cannot meet data-residency commitments. We need a first-party chat service that works on web, iOS and Android, including when a device has been offline for days.]

1.2 Proposed solution

[One paragraph. Example: Clients hold one persistent WebSocket connection to a stateless gateway tier. A chat service assigns every message a monotonically increasing sequence number within its conversation, persists it to a wide-column store partitioned by conversation, and fans it out to the recipients' devices through a routing layer that knows which gateway holds each device's connection. Devices that are offline receive a push notification and catch up by asking for everything after the last sequence number they have seen. Presence and typing indicators are ephemeral and never touch durable storage.]

1.3 Decisions requested from reviewers

ID	Question	Recommended option	Section
Q-01	Example: How are messages ordered?	Server-assigned sequence per conversation	7.2
Q-02	Example: Fan-out on write for all groups?	Write fan-out up to 1 000 members, read fan-out above	7.3, 10
Q-03	Example: Is end-to-end encryption in scope for v1?	1:1 chats in v1, groups in v2	7.6
Q-04	Example: Which message store?	Wide-column store, partition by conversation and time bucket	5.2, 10
[Q-nn]	[Question]	[Option]	[Section]

2. Requirements

2.1 Functional requirements

ID	Requirement	Priority
FR-01	Example: Send and receive text messages in 1:1 conversations	Must
FR-02	Example: Group conversations up to 1 000 members; broadcast channels up to 100 000 members	Must
FR-03	Example: Deliver every message to all of a user's devices (up to 5)	Must
FR-04	Example: Show delivered and read receipts per message (per member in groups up to 100)	Must
FR-05	Example: Show online / last-seen presence and typing indicators	Should
FR-06	Example: Sync history after a device has been offline, in order, without gaps	Must
FR-07	Example: Push notification to offline devices within 5 s	Must
FR-08	Example: Attachments (images, files up to 100 MB) via object storage	Should
FR-09	Example: Edit and delete own messages within 24 h	Could
[FR-nn]	[Requirement]	[Must / Should / Could]

2.2 Non-functional requirements

ID	Quality	Target	Measured as
NFR-01	Latency	Example: Send-to-receive p99 at or below 300 ms when both parties are online in the same region	Client-reported timing, sampled
NFR-02	Availability	Example: 99.95% monthly for send and sync	Synthetic send/receive probes
NFR-03	Durability	Example: An acknowledged message is never lost	Replication RF=3, restore drills
NFR-04	Ordering	Example: All members see a conversation in the same order	Sequence-gap detector on clients
NFR-05	Scale	Example: 15 million concurrent connections at peak	Gateway connection gauge
NFR-06	Privacy	Example: Message content encrypted in transit and at rest; E2EE for 1:1	Security review
[NFR-nn]	[Quality]	[Target]	[Measurement]

2.3 Out of scope

[Example: voice and video calls (separate real-time media design)]
[Example: full-text server-side search of message content]
[Example: federation with external chat networks]
[Item deferred, with the release that will take it]

3. Capacity estimation

3.1 Assumptions

Input	Symbol	Value	Source
Daily active users	$U$	Example: 50 million	Growth plan
Messages sent per user per day	$m$	Example: 40	Comparable products
Average stored message (payload + metadata)	$s$	Example: 300 bytes	Schema in 5.2
Average delivered copies per message (recipients x devices)	$d$	Example: 3	Mix of 1:1 and small groups
Peak concurrent users as share of DAU	$c$	Example: 30%	Traffic history
Connections per gateway node	$g$	Example: 100 000	Load test of gateway
Peak to average factor	$k$	Example: 3	Evening peak
Retention	$Y$	Example: 2 years hot, then cold	Product decision

3.2 Throughput

Messages per day and average send rate:

M_{day} = U \times m = 5 \times 10^{7} \times 40 = 2 \times 10^{9} \ \text{messages/day}

QPS_{send} = \frac{2 \times 10^{9}}{86\,400} \approx 23\,000 \ \text{messages/s}, \qquad QPS_{peak} = k \cdot QPS_{send} \approx 70\,000 \ \text{messages/s}

Deliveries (one per recipient device) at peak:

D_{peak} = d \times QPS_{peak} \approx 3 \times 70\,000 = 210\,000 \ \text{deliveries/s}

3.3 Connections and gateways

C_{peak} = c \times U = 0.3 \times 5 \times 10^{7} = 1.5 \times 10^{7} \ \text{connections}

N_{gw} = \frac{C_{peak}}{g} \times 1.5 = \frac{1.5 \times 10^{7}}{10^{5}} \times 1.5 = 225 \ \text{gateway nodes}

The factor 1.5 keeps headroom for losing one availability zone of three without reconnect storms overloading the survivors.

3.4 Storage

S_{day} = M_{day} \times s = 2 \times 10^{9} \times 300 \ \text{B} = 600 \ \text{GB/day}

S_{2y} = 600 \ \text{GB} \times 730 \approx 438 \ \text{TB}, \qquad \text{with RF} = 3: \approx 1.3 \ \text{PB raw}

Attachments are stored separately in object storage and are usually larger than the message store; estimate them from your media mix.

3.5 Bandwidth

B_{out} \approx D_{peak} \times 500 \ \text{B (framed, encrypted)} \approx 105 \ \text{MB/s} \approx 840 \ \text{Mbit/s}

Heartbeats every 30 s add $1.5 \times 10^{7} / 30 = 500\,000$ small frames per second, terminated at the gateway and never forwarded.

3.6 Summary

Quantity	Estimate	Design consequence
Send rate	~23 000/s average, ~70 000/s peak	Partitioned chat service and log
Deliveries	~210 000/s peak	Routing layer must be in memory, not a database lookup per message
Connections	15 million peak	~225 gateway nodes across 3 zones
Storage	~600 GB/day, ~1.3 PB raw for 2 years	Wide-column store with time buckets; tiering after 2 years
[Quantity]	[Estimate]	[Consequence]

4. API design

4.1 Endpoints and frames

Real-time traffic uses WebSocket frames; history, conversation management and attachments use HTTPS.

Channel	Operation	Purpose	Idempotent
WebSocket	`send`	Send a message; carries a client-generated `clientMsgId`	Yes, deduplicated by `clientMsgId`
WebSocket	`msg` (server to client)	Deliver a message with `conversationId` and `seq`	Client deduplicates by `seq`
WebSocket	`ack`	Device confirms receipt up to a sequence number	Yes
WebSocket	`read`	User marks a conversation read up to `seq`	Yes, monotonic
WebSocket	`typing`, `presence`	Ephemeral signals	Not stored
HTTPS GET	`/v1/conversations/{id}/messages?after={seq}&limit=200`	Catch-up sync and history	Yes
HTTPS POST	`/v1/conversations`	Create a 1:1 or group conversation	With `Idempotency-Key`
HTTPS POST	`/v1/attachments`	Get a pre-signed upload URL	With `Idempotency-Key`

4.2 Frame examples

A send frame from the client, the acknowledgement to the sender, and the delivery frame each recipient device receives:

{"type": "send", "clientMsgId": "01J9ZK7Q4V3M2N8R5T6Y", "conversationId": "c_8f2a", "body": {"text": "Running 5 minutes late"}}

{"type": "sent", "clientMsgId": "01J9ZK7Q4V3M2N8R5T6Y", "conversationId": "c_8f2a", "seq": 18342, "serverTs": "2026-03-14T18:02:11.412Z"}

{"type": "msg", "conversationId": "c_8f2a", "seq": 18342, "sender": "u_19", "body": {"text": "Running 5 minutes late"}, "serverTs": "2026-03-14T18:02:11.412Z"}

4.3 Errors, limits and versioning

Code	Meaning	Client action
`not_member`	Sender is not in the conversation	Remove conversation locally
`rate_limited`	Too many sends; includes `retryAfterMs`	Back off and retry
`too_large`	Body over 64 KB	Use an attachment
`resync_required`	Server cannot replay the requested range from cache	Call the history endpoint

Rate limit: [Example: 20 messages per second per device, 600 per minute per user].
Versioning: protocol version is negotiated in the WebSocket subprotocol header (chat.v1); the server supports the current and previous versions.

5. Data model

5.1 Entities

The core entities are conversations, their members, messages and per-device cursors that record how far each device has synced.

5.2 Storage choices

Data	Store	Partition key	Main queries	Why
Messages	Example: Wide-column store (Cassandra / ScyllaDB class)	(`conversation_id`, `bucket`), clustered by `seq` desc	Latest N; range after `seq`	Append-heavy, range reads within one partition
Conversations, memberships	Example: Relational database, sharded by `conversation_id`	`conversation_id`	Members of a conversation; `last_seq` counter	Transactional membership changes
User inbox index	Example: Wide-column store	`user_id`, clustered by last activity	Conversation list with unread counts	One read renders the inbox
Device cursors	Example: Key-value store	`device_id`	Delivered position per conversation	Small, very frequent writes
Presence, routing	Example: In-memory store with TTL	`user_id`, `device_id`	Which gateway holds this device	Ephemeral; lost data rebuilds on reconnect
Attachments	Object storage + CDN	Content hash	Get by URL	Large blobs out of the message path

bucket is the number of whole 10-day periods since the conversation was created. It caps partition size for very active groups (target below 100 MB per partition).

6. High-level design

Clients connect to gateways through a layer-4 load balancer; everything behind the gateways is stateless except the stores and the log.

Component	Responsibility	Scales by
WebSocket gateway	Terminate TLS and WebSocket, authenticate, heartbeats, hold device sessions	Connections per node
Chat service	Validate membership, deduplicate, assign `seq`, persist, publish to log	Stateless replicas, partitioned by conversation
Message log	Ordered, durable hand-off from write path to fan-out	Partitions (by `conversation_id`)
Fan-out workers	Expand recipients and devices, route to gateways or push	Consumer group size
Presence service	Track online state and last seen; publish changes to subscribers	Sharded by `user_id`
Push service	Batch and send notifications through platform providers	Stateless replicas
[Component]	[Responsibility]	[Scaling unit]

7. Detailed design

7.1 Send path (critical path)

The sender gets its sent acknowledgement as soon as the message is durable; delivery to recipients happens asynchronously from the log.

Steps that fail before the store write return an error and the client retries with the same clientMsgId. If the service crashes after the store write but before replying, the retry finds the existing clientMsgId in a short-lived deduplication index (24 h TTL) and returns the original seq.

7.2 Ordering and consistency model

Order is per conversation. The chat service increments conversation.last_seq with a single-row atomic update in the shard that owns the conversation. Every member sees messages in seq order, regardless of client clocks or which gateway delivered them.
Gaps are detectable. A client that receives seq 18344 after 18342 knows 18343 is missing and requests after=18342. This makes delivery at-least-once on the wire and exactly-once in the client view.
Read-your-writes for the sender: the sender shows the message immediately (optimistic, keyed by clientMsgId) and replaces it with the server copy when sent arrives.
Eventual consistency elsewhere: inbox order, unread counts, presence and receipts may lag by seconds. They are derived views and can be rebuilt from the message store and memberships.
Hot conversations (for example a broadcast channel) serialize on one counter row. At Example: 2 000 sends per second per row this is sufficient; above it, batch sequence allocation in the chat service leader for that conversation.

7.3 Fan-out

For groups up to Example: 1 000 members, fan-out workers push each message to every member's online devices (fan-out on write). For broadcast channels above that size, the worker sends only a lightweight "channel has new messages up to seq" signal to online subscribers, and clients pull the content (fan-out on read). This bounds the worst case: one post to a 100 000 member channel produces 100 000 small signals instead of 300 000 full payloads, and offline members cost nothing until they open the channel.

7.4 Delivery and read receipts

Receipt	Produced when	Stored as	Shown to
Sent	Message is durable	`seq` returned to sender	Sender
Delivered	Any recipient device acks the `seq`	`device_cursor.delivered_seq` (max)	Sender, aggregated per member
Read	Recipient opens conversation up to `seq`	`membership.read_seq` (monotonic max)	Sender; per member in groups up to 100 members, count only above that

Receipts are watermarks, not per-message rows: "read up to 18342" covers every earlier message. This keeps receipt writes proportional to activity, not to message count.

7.5 Offline sync and push

On reconnect, the device sends its delivered_seq per conversation (or a single opaque sync token). The server returns conversations whose last_seq is higher, and the client pages through GET .../messages?after= for each. Push notifications are hints only: the payload carries the conversation and sender name (or nothing, for E2EE), never the authoritative message, so a lost or duplicated push never corrupts history. Push sends are collapsed per conversation (one notification with a count) and respect the user's mute settings.

7.6 End-to-end encryption note

For 1:1 chats in v1, clients use an established double-ratchet protocol with per-device identity keys published to a key directory. The server then stores and routes ciphertext only. Consequences that must be accepted explicitly: server-side search, link previews and content moderation are not possible on encrypted content; multi-device requires encrypting to every device key; and history on a new device needs a client-side backup or device-to-device transfer. Group E2EE (sender keys or the MLS protocol) is a v2 decision.

8. Scalability and performance

Concern	Approach	Limit before redesign
Connections	Horizontal gateways; connection-count based load balancing	Example: ~150 000 connections per node before GC pauses rise
Send throughput	Chat service partitioned by `conversation_id`; message log partitions	Example: ~300 000 sends/s with 256 log partitions
Hot conversation	Single counter row; batch allocation above 2 000/s	One conversation at ~20 000 sends/s
Reconnect storm	Jittered reconnect (1-30 s) and token-bucket admission at gateways	Full-zone failover of 5 million clients
Storage growth	Time-bucketed partitions; move buckets older than 2 years to cold storage	[Limit]

Latency budget for send-to-receive, both users online (target p99 300 ms):

Step	Budget
Client to gateway (mobile network, one way)	80 ms
Gateway to chat service, membership check and `seq`	15 ms
Message store write (quorum)	20 ms
Log publish and fan-out consume	30 ms
Routing lookup and gateway delivery	15 ms
Gateway to recipient (one way)	80 ms
Headroom	60 ms

9. Reliability, security and operations

9.1 Failure modes

Failure	Impact	Detection	Mitigation
Gateway node dies	~100 000 devices disconnect	Connection gauge drop	Clients reconnect with jitter to other nodes; routing entries expire by TTL
Routing cache stale	Delivery sent to wrong gateway and dropped	Delivery ack rate	Client gap detection triggers sync; routing entries refreshed on every heartbeat
Message log lag	Delivery delayed, sends still succeed	Consumer lag seconds	Autoscale fan-out workers; alert at 10 s lag
Message store write timeout	Send fails, client retries	Write error rate	Idempotent retry via `clientMsgId`; quorum across 3 zones
Push provider outage	Offline users not notified	Provider error rate	Queue and retry with backoff; messages still sync on open
Region outage	Users homed in region cannot send	Synthetic probes	Example: Conversations homed in one region; warm standby promoted in under 15 min (RPO under 1 min)
[Failure]	[Impact]	[Signal]	[Mitigation]

9.2 Security and abuse

Threat	Control
Session hijacking	Short-lived access tokens on WebSocket upgrade; re-authenticate on token expiry
Sending to conversations the user left	Membership checked on every send and every history read
Spam and abuse	Per-user and per-device rate limits; new-account limits; user reporting (with E2EE, reported messages are forwarded by the reporter's client)
Metadata exposure	Minimise retained metadata; delete push tokens on logout; encrypt at rest
Malicious attachments	Content-type allowlist, size limits, malware scan before the download URL is issued
[Threat]	[Control]

9.3 Observability

Signal	Metric or log	Alert threshold
Send latency	p50, p99 from `send` to `sent`	p99 above 150 ms for 10 min
End-to-end delivery	Sampled send-to-ack latency	p99 above 300 ms for 10 min
Undelivered backlog	Messages older than 60 s without delivered receipt to any online device	Above 0.1%
Fan-out lag	Log consumer lag	Above 10 s
Connections	Per node and per zone	Node above 130 000
Client gap rate	Sync requests triggered by gaps	Above 1% of sessions

9.4 Cost estimate

Item	Unit cost	Quantity	Monthly cost
Gateway nodes	[$/node-month]	Example: 225	[$]
Chat and fan-out services	[$/node-month]	[n]	[$]
Message store	[$/TB-month]	Example: ~55 TB raw added per month	[$]
Message log	[$/partition-month]	Example: 256 partitions	[$]
Push provider and SMS fallback	[$/1 000 sends]	[n]	[$]
Total			[$]

10. Trade-offs and alternatives

Decision	Options	Chosen	Why	What we give up
Transport	Long polling; Server-Sent Events plus POST; WebSocket	WebSocket	Bidirectional, low overhead per frame, mature on all platforms	Stateful gateways and reconnect handling
Ordering	Client timestamps; global sequence; per-conversation sequence	Per-conversation sequence	Total order where users notice it, no global bottleneck	No order across conversations
Message store	Relational; document store; wide-column	Wide-column	Linear write scaling, efficient range reads per partition	Limited secondary indexes and transactions
Fan-out	Always on write; always on read; hybrid	Hybrid at 1 000 members	Low latency for normal groups, bounded cost for huge channels	Two code paths
Receipts	Per-message rows; watermarks	Watermarks	Writes scale with activity, not messages	Cannot show "read" on one older message independently
Encryption	TLS only; E2EE for 1:1; E2EE for all	E2EE for 1:1 in v1	Strong privacy for most sensitive chats with manageable scope	No server search or moderation on 1:1 content
[Decision]	[Options]	[Chosen]	[Why]	[Cost]

11. Delivery plan and risks

11.1 Milestones

Milestone	Scope	Exit criterion	Target date
M1	1:1 chat, single region, web client	Load test at 2x peak sends, zero lost messages	YYYY-MM-DD
M2	Groups, receipts, mobile push	Gap rate below 0.1% for 14 days in beta	YYYY-MM-DD
M3	Offline sync at scale, attachments	7-day offline device syncs in under 10 s	YYYY-MM-DD
M4	E2EE for 1:1	External cryptography review passed	YYYY-MM-DD

11.2 Rollout and rollback

[Example: Ship behind a per-user flag. Start with internal staff, then 1%, 10%, 50% and 100% of users, holding each step for at least 48 h while watching send latency, gap rate and undelivered backlog. Rollback disables the flag; messages already written stay readable through the history API.]

11.3 Risks

ID	Risk	Likelihood	Impact	Mitigation	Owner
R-01	Example: Reconnect storm after a gateway deploy overloads auth	Medium	High	Rolling deploys with connection draining; jittered reconnect	[@name]
R-02	Example: Hot broadcast channel saturates one shard	Medium	Medium	Hybrid fan-out; batch sequence allocation	[@name]
R-03	Example: Push provider throttles bursts	Medium	Medium	Collapse keys, per-app send rate control	[@name]
[R-nn]	[Risk]	[L / M / H]	[L / M / H]	[Mitigation]	[@name]

12. Appendices

12.1 Open questions

ID	Question	Owner	Due
OQ-01	Example: Do we keep history forever for paid workspaces?	[@name]	YYYY-MM-DD
OQ-02	Example: MLS or sender keys for group E2EE?	[@name]	YYYY-MM-DD
[OQ-nn]	[Question]	[@name]	YYYY-MM-DD

12.2 Glossary

Term	Meaning
`seq`	Server-assigned, gap-free, increasing number of a message within its conversation
Fan-out	Copying or signalling one message to every recipient device
Watermark	"Everything up to this `seq`" marker used for receipts and sync
Presence	Online or last-seen state of a user
E2EE	End-to-end encryption: only endpoints hold the keys
[Term]	[Meaning]

12.3 References

Reference	Use
RFC 6455 The WebSocket Protocol	Transport in 4.1
RFC 9420 The Messaging Layer Security (MLS) Protocol	Candidate for group E2EE
Apple Push Notification service and Firebase Cloud Messaging documentation	Push integration in 7.5
[Internal ADR or document]	[Use]

Standard and licence note

Based on common industry practice for real-time messaging system design and on the protocols cited in 12.3. This is a reference design to adapt: the numbers are illustrative assumptions, not measurements. Original text written for this collection. Template licence: MIT.