Video Conferencing System Design Template (WebRTC SFU)

System Design: [Video conferencing service name]

Reference design for a multiparty video conferencing service built on WebRTC: browser and native clients, a signaling service, selective forwarding units (SFUs) with simulcast, bandwidth estimation and layer selection, cascading across regions for large meetings, recording and optional end-to-end encryption. It builds on the W3C Recommendation "WebRTC: Real-Time Communication in Browsers" (13 March 2025), RTP (RFC 3550), RTP topologies (RFC 7667), simulcast (RFC 8853), congestion control feedback (RFC 8888) and SFrame (RFC 9605). It follows the structure of the System Design Document template, adds a 45-minute interview path, access-pattern and failure-mode tables and a cited "How real systems do it" box, and is pre-filled with a worked design to adapt. Replace every [bracketed] value and every row marked Example; values marked Example are assumptions, not benchmarks.

Document control

Approvals

Revision history

Contents

1. Overview
2. Requirements
3. Capacity estimation
4. API design
5. Data model
6. High-level design
7. Detailed design
8. Scalability and performance
9. Reliability, security and operations
10. Trade-offs and alternatives
11. Delivery plan and risks
12. Appendices

1. Overview

1.1 Problem statement

[Example: The current product runs small calls as a peer-to-peer mesh. Each participant uploads one copy of its video to every other participant, so a six-person call needs about 7.7 Mbps of upload per person (section 3.2), and calls above four people freeze on home connections. Customers also ask for recording, meetings of several hundred people and an end-to-end encrypted mode for sensitive calls.]

1.2 Proposed solution

[Example: Clients join through a signaling service over a WebSocket and send media to the nearest SFU over WebRTC (ICE, DTLS-SRTP). Each sender publishes three simulcast layers of its camera. The SFU forwards packets without decoding them and picks, for every receiver, the layer that fits the receiver's estimated bandwidth and screen layout. Meetings whose participants are in several regions use one SFU per region, linked by a cascade that carries each layer once between regions. Recording writes the raw forwarded tracks to object storage and composes a playable file after the meeting. An optional end-to-end encrypted mode wraps every frame with SFrame keys that the server never sees.]

1.3 Decisions requested from reviewers

1.4 45-minute interview path

Use this order when the document serves an interview or a short design review. The written sections below are the long form of each step.

Start the default deep dive with the naive answer, a mesh where every peer sends to every other peer, and show that upload grows with meeting size. Then introduce the MCU, which fixes upload but decodes and re-encodes every stream, and finally the SFU, which only forwards packets but needs the sender to offer several qualities so that each receiver can get one that fits.

2. Requirements

2.1 Core functional requirements

2.2 Non-functional requirements

2.3 Below the line (out of scope)

Name these explicitly so reviewers and interviewers do not pull the design toward them.

• [Example: broadcast to unlimited viewers through a CDN (see the live streaming design)]
• [Example: telephone dial-in through a PSTN gateway]
• [Example: live captions and translation (separate speech pipeline consuming audio tracks)]
• [Example: calendar integration and scheduling user interface]
• [Item deferred, with the release that will take it]

3. Capacity estimation

3.1 Assumptions

3.2 Why not a mesh

In a mesh each sender uploads its high layer and audio to every other participant:

Many home and mobile uplinks cannot sustain this, so a server in the media path is required for meetings above three or four people.

3.3 Upload and download per participant with an SFU

The sender uploads three layers once; the receive mix stays constant as the meeting grows, because the layout caps the number of tiles.

3.4 SFU fleet

Ingress alone would need 515 servers, so egress is the binding constraint. With 25% headroom for failover and uneven placement the fleet is about 870 SFU servers across all regions. With an average packet of about 1 100 bytes, egress is about 470 million packets per second, about 680 000 per server before headroom, so the forwarding path must avoid per-packet allocation and system calls where possible (batching sends, kernel bypass or large socket buffers).

3.5 Signaling and recording

• Joins: $P / d = 2 \times 10^{6} / 2\,700 \ \text{s} \approx 741$ joins/s on average at peak, about 7 400 joins/s in the minute after the hour when meetings start together. At about 10 signaling messages per join that is about 74 000 messages/s, small for a horizontally scaled WebSocket tier.
• Recording: if 5% of 4 million daily meetings are recorded, live composite recording at 2.064 Mbps (720p plus 64 kbps audio) writes about 139 TB per day (12.5 PB for 90 days) and needs about 16 700 compositors running at peak, about 33 000 vCPU at 2 vCPU each. Raw track recording of the 360p layers ($6 \times (0.7 \times 0.5 + 0.04) = 2.34$ Mbps per meeting) writes about 158 TB per day (14.2 PB for 90 days) but needs no live decoding.

3.6 Summary

4. API design

4.1 Interfaces

4.2 Join flow

The join token is a signed, short-lived credential (for example a JWT with a 60 s expiry) that names the meeting, the participant and the role. The signaling service validates it, picks an SFU (7.4) and returns ICE servers, including TURN credentials that expire with the session.

{"type": "joined", "meetingId": "mtg_7Qx2", "participantId": "p_19", "role": "host", "sfu": {"region": "eu-west", "node": "sfu-eu-w-114"}, "iceServers": [{"urls": ["turns:turn.eu-west.example.com:443?transport=tcp"], "username": "1730300000:p_19", "credential": "[short-lived]"}]}

4.3 Signaling messages

The layout message matters for capacity: a client that shows only the active speaker should not receive 24 thumbnails it will never draw.

5. Data model

5.1 Entities

Meetings and recordings are durable records; sessions, tracks and SFU assignments are live state that exists only while a meeting runs.

5.2 Storage choices

5.3 Access patterns

6. High-level design

Media and signaling take different paths. Media goes from client to SFU over UDP when possible, through TURN when a firewall blocks UDP.

7. Detailed design

The deep dives in this section, each with options, choice and reason:

7.1 Topology: mesh, MCU or SFU

RFC 7667 describes the RTP topologies; the three relevant here are:

The SFU wins because it adds almost no latency (no decode), keeps CPU cost low and lets every receiver choose its own layout. Its weakness, that the sender must offer several qualities, is solved by simulcast.

7.2 Simulcast

Each sender encodes its camera at several resolutions at once and sends them as separate RTP streams, identified by a restriction identifier (rid) negotiated in SDP as RFC 8853 describes. In the W3C WebRTC API the sender sets sendEncodings on addTransceiver, one entry per layer, with rid and scaleResolutionDownBy.

pc.addTransceiver(cameraTrack, {
  direction: "sendonly",
  sendEncodings: [
    { rid: "l", scaleResolutionDownBy: 4, maxBitrate: 150000 },
    { rid: "m", scaleResolutionDownBy: 2, maxBitrate: 500000 },
    { rid: "h", scaleResolutionDownBy: 1, maxBitrate: 1500000 }
  ]
});

The SFU tells a sender to pause a layer nobody is receiving (for example the 720p layer when nobody has that participant in the large tile), which saves the sender's uplink and CPU. Screen share uses one high-resolution, low frame rate stream instead, because text must stay sharp.

7.3 Bandwidth estimation and layer selection

Two estimates drive adaptation:

• Sender side: the sender's congestion controller estimates how much it can upload and sets the encoder bitrates. Browsers use delay-based and loss-based control (the Google Congestion Control approach, described in the expired draft-ietf-rmcat-gcc-02) driven by transport-wide feedback. RFC 8888 standardises an RTCP feedback format for congestion control.
• Receiver side at the SFU: the SFU sends packets to each receiver and estimates that receiver's downlink from feedback about arrival times and loss. It then picks layers.

The layer selection loop runs per receiver every few hundred milliseconds:

1. Start from the receiver's layout: one large tile, up to $k$ small tiles, the three loudest audio streams.
2. Audio first: audio is never dropped for video.
3. Give each small tile its lowest layer.
4. Give the large tile the highest layer that still fits the estimate.
5. If even the lowest layers do not fit, pause the small tiles and keep the large tile at its lowest layer; tell the client so it can show avatars.

Worked example with the ladder from 3.1 (Example):

A layer switch must happen on a key frame of the new layer, otherwise the receiver cannot decode. The SFU requests a key frame from the sender (RTCP picture loss indication) and switches when it arrives, and it rewrites sequence numbers and timestamps so the receiver sees one continuous stream. To avoid oscillation, step up only after the estimate has stayed above the next layer's bitrate plus 15% for a few seconds; step down at once.

Packet loss is handled per hop: the SFU retransmits lost packets to a receiver from its own short buffer (NACK), so a lossy receiver does not cause retransmissions from the sender to everyone. Audio relies on Opus forward error correction instead, because a retransmission arrives too late for speech.

7.4 Placement and cascading

• Placement: the first participant in a region creates the meeting's presence on the least-loaded SFU of that region (by egress, not session count). Later participants in the same region join the same node, so one meeting stays on one SFU per region while it fits.
• Cascade: when participants are in several regions, each region's SFU forwards to the other region's SFU only the layers that remote receivers need, once per layer and not once per receiver. A 40-person meeting split across two regions crosses the long-haul link with a few streams instead of dozens.
• Large meetings: a meeting above the capacity of one SFU spreads its receivers over several SFUs in the same region connected as a tree. View-only attendees beyond about 1 000 move to the live streaming path, which trades seconds of latency for CDN scale.

7.5 Recording

Two options were compared for FR-06:

Raw capture stores the 360p layer of each camera, the screen share and audio. Composition runs on batch capacity within the 30-minute target in FR-06. Participants are told that recording is on (visual indicator and an audible notice) before any packet is captured.

7.6 Security and end-to-end encryption

• Transport: WebRTC requires DTLS-SRTP, so every media hop is encrypted. The SFU terminates SRTP, so in standard mode the operator can access media on the server (needed for recording and transcription).
• E2EE mode: clients encrypt each encoded frame with SFrame (RFC 9605) before packetisation. Keys come from a group key agreement between clients, for example MLS (RFC 9420); the server only relays opaque handshake messages and sees epoch numbers. The SFU still routes packets and selects layers because RTP headers stay readable. Server recording, transcription and dial-in are disabled in this mode, and the UI shows a verification code participants can compare.
• Access: join tokens are short-lived and bound to a role; a waiting room and meeting lock prevent uninvited joins; TURN credentials expire with the session.

7.7 How real systems do it

How real systems do it (paraphrased, not copied). Discord's engineering post "How Discord Handles Two and Half Million Concurrent Voice Users using WebRTC" (Jozsef Vass, 10 September 2018) explains that all calls are routed through Discord's own servers rather than peer to peer, because peer-to-peer cost rises with participants and routing through servers hides user IP addresses. A home-grown C++ SFU forwards audio and video and uses receiver reports to tell senders how much bandwidth they have. Native clients use a customised WebRTC media engine with a much smaller signaling exchange than full SDP, while browsers use the standard API. Signaling services assign a voice server from those registered in a service discovery system, and when a voice server fails its health check clients reconnect to a new one. The post reports more than 850 voice servers in 13 regions. Read the original in 12.3; this box keeps only the general idea.

8. Scalability and performance

Latency budget within one region (target p95 250 ms glass to glass):

9. Reliability, security and operations

9.1 Failure modes

9.2 Security and abuse

9.3 Observability

9.4 Cost estimate

10. Trade-offs and alternatives

11. Delivery plan and risks

11.1 Milestones

11.2 Rollout and rollback

[Example: New SFU releases go to 1% of new meetings in one region, then 10%, then all, with automatic rollback when the freeze rate or reconnect rate rises by more than 20% against the previous release. Meetings already running stay on their node until they end; nodes are drained, never killed.]

11.3 Risks

12. Appendices

12.1 Open questions

12.2 Glossary

12.3 References

Standard and licence note

Based on the W3C WebRTC Recommendation (13 March 2025) and the IETF RFCs cited in 12.3, which are public specifications. The Discord engineering post is paraphrased in 7.7 only; no text, figures or code were copied apart from the short reported counts attributed to it, and all other numbers here are illustrative examples. WebRTC specifications and browser support change; check the current W3C and IETF pages before implementing. Original text written for this collection. Template licence: MIT.