System Design: [Live streaming platform name]

Reference design for a Twitch-style live streaming platform: RTMP, SRT and WHIP ingest, real-time transcoding into an adaptive bitrate ladder, low-latency delivery with LL-HLS (HTTP Live Streaming 2nd Edition draft, which updates RFC 8216) and low-latency DASH with CMAF chunks, an origin and edge tier that collapses millions of viewer requests, chat kept in step with video, a glass-to-glass latency budget, and a VOD archive. It follows the structure of the System Design Document template, adds a 45-minute interview path, and is pre-filled with a worked design to adapt. Replace every [bracketed] value and every row marked Example; numbers marked Example are assumptions to recompute.

Document control

Approvals

Revision history

Contents

1. Overview
2. Requirements
3. Capacity estimation
4. API design
5. Data model
6. High-level design
7. Detailed design
8. Scalability and performance
9. Reliability, security and operations
10. Trade-offs and alternatives
11. Delivery plan and risks
12. Appendices

1. Overview

1.1 Problem statement

[Example: The platform delivers live video with standard HLS at 10 to 15 seconds of latency, so viewers react to moments long after chat has discussed them and interactive features (polls, predictions) feel broken. Every channel is transcoded into a full ladder, although most channels have fewer than five viewers, and a single popular channel recently overloaded the origin when two million viewers joined within a minute.]

1.2 Proposed solution

[Example: Broadcasters connect to the nearest ingest edge over RTMP, Enhanced RTMP, SRT or WHIP. The edge authenticates the stream key and forwards the stream over the backbone to a transcoder in a regional origin. Channels above a viewer threshold get a full ladder produced by one decoder feeding several scalers and encoders; small channels get the source plus two low renditions. A packager writes CMAF segments with 0.5 s parts and publishes LL-HLS playlists and a low-latency DASH manifest from the same media. A mid-tier shield and edge caches collapse identical requests, so the origin sees load per edge, not per viewer. Chat and interactive events carry stream timestamps so clients can align them with the frame on screen. Sources are archived for VOD and transcoded on first view.]

1.3 Decisions requested from reviewers

1.4 45-minute interview path

Say early that live is different from VOD: segments exist for seconds before they are needed, so caching works by collapsing simultaneous requests, not by long retention.

2. Requirements

2.1 Core functional requirements

2.2 Non-functional requirements

2.3 Below the line (out of scope)

• [Example: sub-second interactive video (video calls, co-streaming rooms), which needs WebRTC delivery end to end]
• [Example: DRM for premium live events; see Video Streaming System Design (system-design/design-video-streaming)]
• [Example: clip editing and highlights]
• [Example: chat service internals; see Chat System Design (system-design/design-chat-system)]
• [Example: recommendation of live channels]
• [Item deferred, with the release that will take it]

3. Capacity estimation

3.1 Assumptions

3.2 Ingest

• 100 000 x 6 Mbit/s = 600 Gbit/s at peak. Consequence: ingest is terminated at regional edges close to broadcasters and carried on the backbone; a single central ingest site would put all of it on transit.

3.3 Transcoding

• Full ladders: 20 000 / 8 = 2 500 servers.
• Light ladders: 80 000 / 40 = 2 000 servers.
• With 30% headroom: 4 500 / 0.7 = about 6 430, so plan 6 500 servers.
• If every channel got a full ladder: 100 000 / 8 / 0.7 = about 17 900 servers. Consequence: the viewer-threshold policy cuts transcode capacity by about 64%; decode-once pipelines (7.1) cut it further.

3.4 Egress and origin fan-out

• 10 million x 3 Mbit/s = 30 Tbit/s at peak. Consequence: delivery needs several CDN providers or an owned edge; see Content Delivery Network System Design (system-design/design-cdn).
• For the 1 million viewer channel with 0.5 s parts, viewers make about 2 million playlist requests/s and 2 million part requests/s. If 500 edge servers each fetch every part of 6 renditions once: 500 x 6 / 0.5 s = 6 000 requests/s to the shield, and the shield sends 6 / 0.5 = 12 requests/s to the origin. Consequence: request collapsing at edge and shield is mandatory; the origin must never see per-viewer traffic.

3.5 Chat

• A channel with 1 million viewers and 1 000 messages/s would need 1 billion deliveries/s; at 100 bytes per message that is 100 KB/s, about 800 kbit/s of chat per viewer, more than a low video rendition. Consequence: cap the displayed rate per viewer (Example: 50 messages/s through sampling and slow mode), which brings one channel to 5 KB/s x 1 million = 5 GB/s, about 40 Gbit/s, fanned out by chat edges.

3.6 VOD archive

• Source only: 50 000 channels x 6 Mbit/s = 300 Gbit/s = 37.5 GB/s, times 86 400 s = about 3.24 PB/day.
• 14-day retention: about 45 PB. Consequence: archive the source only and produce the VOD ladder on first view; most past broadcasts are never watched, so pre-transcoding them would waste most of the compute.

3.7 Summary

4. API design

4.1 Control plane endpoints

4.2 Ingest and playback URLs

Broadcasters and players see only these URL shapes; the internal layout can change behind them.

rtmps://ingest-{region}.example.com/app/{stream_key}
srt://ingest-{region}.example.com:7000?streamid={stream_key}
https://ingest-{region}.example.com/whip/{channel_id}        (Bearer token)
https://live.example.com/{channel_id}/master.m3u8             (LL-HLS)
https://live.example.com/{channel_id}/manifest.mpd            (LL-DASH)

4.3 Errors and limits

5. Data model

5.1 Entities

Control-plane state is small; media is large and short-lived on the live path.

5.2 Storage choices

5.3 Access patterns

6. High-level design

7. Detailed design

7.1 Deep dive 1: ingest and transcoding ladder

Problem. Broadcasters use many encoders and unstable home networks, and transcoding every channel into a full ladder is the largest compute cost.

Choice. C, with D accepted when the broadcaster's encoder sends multiple tracks through Enhanced RTMP or WHIP simulcast.

Reason. Most channels have few viewers. One decoder feeds all scalers, and renditions of the same resolution share one scaler, which removes duplicate work. Keyframes are forced at the same timestamps in every rendition (Example: every 2 s) so players can switch at segment boundaries; source passthrough is used as the top rendition only when the source keyframe interval matches.

Light ladder: source, 480p30 and 160p30. Ingest protocol choice:

7.2 Deep dive 2: low-latency delivery and origin fan-out

Problem. Low latency needs small units of media, which multiplies requests, while one popular channel can draw a million viewers at once.

Choice. B and C from one set of CMAF media; A as automatic fallback for players without low-latency support.

Reason. One encode and one set of CMAF fragments serve both manifests. With LL-HLS, the packager publishes each 0.5 s part as soon as it is complete (EXT-X-PART), announces the next one in advance (EXT-X-PRELOAD-HINT), and holds playlist requests until the requested media sequence and part exist (blocking reload, advertised with CAN-BLOCK-RELOAD in EXT-X-SERVER-CONTROL). Delta playlist updates keep playlist responses small. These features are defined in the HTTP Live Streaming 2nd Edition draft (draft-pantos-hls-rfc8216bis); RFC 8216 itself has no low-latency mode. For DASH, the manifest signals that segments can be fetched before they are complete, and the CDN streams chunks as they arrive, following DASH-IF low-latency guidance.

Request collapsing makes this scale: when 10 000 viewers on one edge ask for the same part, the edge sends one request upstream and answers all of them from that response. Blocking playlist requests are collapsed the same way, keyed by the requested sequence and part.

7.3 Deep dive 3: latency budget

Problem. "3 seconds" is only reachable if every stage has a budget and the player buffer is set on purpose.

Options. Smaller parts (Example: 0.25 s) cut both the part wait and the hold-back but double request rates; shorter keyframe intervals ease channel switching but cost bitrate; a smaller buffer cuts latency but raises rebuffering on mobile.

Choice. 0.5 s parts, 2 s segments, keyframes every 2 s, part hold-back of 1.5 s, and catch-up playback (Example: up to 1.1x speed) when the player drifts beyond 5 s.

Reason. This keeps request rates at the levels in 3.4 and lands at the 3 s target at p50: the table uses the worst-case 500 ms part wait, and the average wait is half a part (250 ms), which brings the typical total to about 2.8 s. The draft requires PART-HOLD-BACK to be at least twice the part target duration and recommends at least three times, so 1.5 s matches the recommendation for 0.5 s parts; check the revision you pin.

7.4 Deep dive 4: chat and event sync

Problem. Chat travels in real time while video is 3 s (low latency) or 10 s (fallback) behind, so viewers see reactions before the moment, and a poll can open before the broadcaster announces it on screen.

Choice. A for chat, C for interactive events, B as an option for fallback-latency players.

Reason. With 3 s latency, chat feels live; aligning it would delay the broadcaster's conversation. Events matter at frame accuracy, so the packager stamps them with media time and inserts them as EXT-X-DATERANGE in playlists and as event messages in CMAF, and the player fires them when playback reaches that time. Players compute their own latency from EXT-X-PROGRAM-DATE-TIME against a synchronised clock and report it in telemetry. Chat delivery itself follows Chat System Design (system-design/design-chat-system), with batching every 200 ms and the displayed-rate cap from 3.5.

How real systems do it (paraphrased from Twitch Engineering, "Live Video Transmuxing/Transcoding: FFmpeg vs TwitchTranscoder, Part I", October 2017; inspiration only). The post describes broadcasters sending RTMP to Twitch and viewers watching HLS in several quality variants. It explains why running one independent encoder process per variant wastes work: the decoding step is identical for every variant, so a single decode should feed the scalers and encoders, and variants of the same resolution can share scaling. It stresses that keyframes must line up across variants for smooth switching, which a pure pass-through of the source cannot guarantee when the source keyframe interval varies, and it describes features such as frame rate reduction and metadata insertion that motivated an in-house transcoder. Check the post for its exact settings and results.

7.5 VOD archive

At broadcast start the packager also writes the source rendition to object storage. At the end, a job writes a VOD playlist that points at the archived source. On the first VOD view the ladder is transcoded faster than real time in parallel chunks, and the full VOD then follows Video Streaming System Design (system-design/design-video-streaming). Archived sources expire after 14 days unless the broadcaster saves the video.

8. Scalability and performance

Viewer count is computed from edge playback logs every 10 s and smoothed; exact counts would need per-viewer state the delivery path does not keep.

9. Reliability, security and operations

9.1 Failure modes

9.2 Security and abuse

9.3 Observability

9.4 Cost estimate

10. Trade-offs and alternatives

10.1 Decisions

10.2 Alternatives considered

• WebRTC for all viewers. Sub-second latency, but every viewer needs a stateful media session, which scales far worse through commodity CDNs. Keep it for co-streaming and interactive rooms.
• Managed live video service. Fastest to launch; becomes the main cost line at scale and limits ladder and latency tuning.

11. Delivery plan and risks

11.1 Milestones

11.2 Rollout and rollback

[Example: Low-latency playback is enabled per player version and per country with a flag. Rollback serves the standard playlist from the same media, which every player supports, so no re-encoding is needed.]

11.3 Risks

12. Appendices

12.1 Open questions

12.2 Glossary

12.3 References

Inspiration only; no text, diagrams or figures are copied.

Standard and licence note

Based on RFC 8216, the HTTP Live Streaming 2nd Edition Internet-Draft, RFC 9725 and the other public specifications listed in 12.3; the LL-HLS draft is not a published RFC and its details can change between revisions. The engineering write-ups are cited as inspiration only. This is a reference design to adapt, with illustrative numbers. Original text written for this collection; check the current official versions before you implement. Template licence: MIT.