System Design: [Content delivery network name]

Reference design for a content delivery network (CDN) that serves web assets, images, API responses and video segments for several internal tenants: request routing with DNS and anycast, a cache hierarchy of edge, shield and origin, pull-through and proactive fill, purge and invalidation, TLS at the edge, origin shielding, appliances embedded in ISP networks, cache key design and observability. It applies HTTP caching as specified in RFC 9111, with RFC 9213 (targeted cache control), RFC 9211 (Cache-Status) and RFC 5861 (stale content). It follows the structure of the System Design Document template, adds a 45-minute interview path, and is pre-filled with a worked design to adapt. Replace every [bracketed] value and every row marked Example; numbers marked Example are assumptions to recompute.

Document control

Approvals

Revision history

Contents

1. Overview
2. Requirements
3. Capacity estimation
4. API design
5. Data model
6. High-level design
7. Detailed design
8. Scalability and performance
9. Reliability, security and operations
10. Trade-offs and alternatives
11. Delivery plan and risks
12. Appendices

1. Overview

1.1 Problem statement

[Example: The company buys delivery from two commercial CDNs. Egress is the second largest infrastructure cost, purges take up to 15 minutes, cache hit ratio for images is 82% because each provider fills from origin separately, and video quality in three large markets suffers from congested transit links that neither provider controls.]

1.2 Proposed solution

[Example: Build an owned CDN for the largest tenants and keep a commercial CDN as overflow. About 200 points of presence (PoPs) announce shared anycast addresses for web and API traffic; video clients receive server-specific URLs from a steering service. Each PoP runs edge caches with consistent hashing inside the PoP; misses go to one regional shield per origin, which collapses duplicate fetches before they reach the origin. Popular video is filled proactively to edge and to appliances embedded in partner ISPs during off-peak hours. Purges use tags and soft invalidation and reach all PoPs within 5 seconds at p99. TLS terminates at the edge with ECDSA certificates, and every response carries a Cache-Status header for debugging.]

1.3 Decisions requested from reviewers

1.4 45-minute interview path

Separate the data plane (serving bytes) from the control plane (configuration, purge, steering) in the first diagram.

2. Requirements

2.1 Core functional requirements

2.2 Non-functional requirements

2.3 Below the line (out of scope)

• [Example: volumetric DDoS scrubbing at layers 3 and 4 beyond PoP capacity (contracted provider)]
• [Example: web application firewall rule authoring (separate product team)]
• [Example: edge compute for arbitrary tenant code]
• [Example: video packaging and DRM (see Video Streaming System Design (system-design/design-video-streaming))]
• [Example: selling the CDN to external customers]
• [Item deferred, with the release that will take it]

3. Capacity estimation

3.1 Assumptions

Check: 50 million requests/s x 250 KB = 12.5 TB/s = 100 Tbit/s, which matches the egress assumption.

3.2 Servers and storage

• Average per PoP: 100 Tbit/s / 200 = 500 Gbit/s; at 50 Gbit/s usable per server that is 10 servers, 2 000 in total, with large metros holding many times the average. Consequence: PoP sizes follow traffic, not a fixed template; capacity planning is per metro.
• Storage per average PoP: 10 x 100 TB = 1 PB, 20% of the catalog. Consequence: the edge cannot hold everything; hit ratio depends on popularity skew, so eviction policy and proactive fill for predictable content (7.3) matter.

3.3 Origin load

• With a 95% edge byte hit ratio, misses are 5 Tbit/s.
• Without a shield, each of 200 PoPs fetches a missing object independently: up to 200 origin fetches per new object and 5 Tbit/s at origins.
• With a regional shield at 80% hit ratio: 5 Tbit/s x 0.2 = 1 Tbit/s at origins, and each new object is fetched once per shield. Consequence: shielding is mandatory for every origin; tenants size origins for 1% of edge traffic, not 5%.
• Origin requests: 50 million x 0.05 x 0.2 = 500 000 requests/s across all tenants.

3.4 Purge

• Example: 1 000 purge requests/s at peak across tenants. Sending each to 2 000 servers directly would be 2 million messages/s. Consequence: purges fan out per PoP (200 messages each) and are broadcast inside the PoP; tags let one request cover thousands of objects.

3.5 TLS

• At Example: 10 requests per connection, 50 million requests/s means 5 million new connections/s. With 80% session resumption, 1 million full handshakes/s. At Example: 20 000 ECDSA P-256 signatures per core per second, that is 50 cores globally; with RSA-2048 at Example: 1 500 per core it would be about 670 cores. Consequence: serve ECDSA certificates by default, keep RSA only for old clients, and keep resumption tickets valid across servers in a PoP.

3.6 Summary

4. API design

4.1 Control plane endpoints

4.2 Data plane headers

The origin controls caching through standard headers; the edge reports what it did.

HTTP/1.1 200 OK
Cache-Control: public, max-age=60, stale-while-revalidate=30, stale-if-error=86400
CDN-Cache-Control: max-age=86400
Surrogate-Key: product-123 category-shoes
Vary: Accept-Encoding
Cache-Status: ExampleCDN-shield; fwd=miss; stored, ExampleCDN-edge; hit; ttl=84211

Cache-Control governs browsers; CDN-Cache-Control (RFC 9213) gives the CDN a longer lifetime that browsers ignore. stale-while-revalidate and stale-if-error (RFC 5861) allow serving stale content during refresh and origin errors. Surrogate-Key is a widely used, vendor-defined tag header, not an RFC; the edge strips it before the response leaves the CDN. Cache-Status (RFC 9211) lists each cache tier's result, ordered from the cache closest to the origin to the cache closest to the client, so each tier appends its own entry.

4.3 Errors and limits

5. Data model

5.1 Entities

The control plane is small and strongly consistent; cache indexes are local to each server.

5.2 Storage choices

5.3 Access patterns

6. High-level design

7. Detailed design

7.1 Deep dive 1: request routing

Problem. Clients must reach a nearby, healthy server with spare capacity, and must be moved away quickly when a PoP fails.

Choice. B for web, image and API hostnames; C for video sessions, with A as the backup path when anycast catchments become uneven.

Reason. Short web requests benefit from anycast's simplicity and fast failover. Video sessions are long and large, so choosing a server that already holds the file, and is close in network terms, pays off. Steering uses routes learned from ISPs and server inventory, not just geography.

7.2 Deep dive 2: cache hierarchy and fill

Problem. Each tier that misses independently multiplies origin load, while each extra tier adds latency on misses.

Choice. B, with request collapsing at both tiers: while a fetch for an object is in flight, later requests wait for it instead of starting new fetches. Inside a PoP, consistent hashing places each object on one owner server so the PoP stores one copy; objects above Example: 1 000 requests/s on one server are replicated to several servers in the PoP.

Reason. The shield turns 5 Tbit/s of misses into 1 Tbit/s at origins (3.3) and caps fetches per new object, which protects origins during launches. Large objects are fetched and cached in byte ranges (Example: 2 MB chunks) so a seek in a large file does not pull the whole file through the shield.

Fill modes:

7.3 Deep dive 3: ISP-embedded appliances and proactive fill

Problem. In some markets the bottleneck is the link between the ISP and the CDN, not the CDN itself; peak-hour video crosses congested transit.

Choice. A everywhere and B for the top markets by video traffic. Appliances receive popular video during off-peak fill windows, computed from predicted demand, and can fill from nearby appliances. The ISP decides which of its customers are routed to the appliances, and the steering service sends video sessions there.

Reason. For predictable catalogs, moving bytes at night and serving them locally at peak removes the congested link from the critical path, which benefits both the ISP and the viewers.

How real systems do it (paraphrased from Netflix, "Open Connect Overview", openconnect.netflix.com; inspiration only). Netflix describes Open Connect as its own CDN, started in 2011, built from purpose-built cache appliances placed in two ways: at internet exchange points, where they connect to ISPs through settlement-free peering, and inside ISP networks, where Netflix supplies the hardware and the ISP supplies power, space and connectivity and controls which customers are routed to them. The appliances store only video and image files, no member data, and do two jobs: report health, learned BGP routes and stored files to a control plane in the cloud, and serve files over HTTP or HTTPS. When a member presses play, cloud services check authorisation, pick the needed files, and a steering service chooses appliances and returns their URLs to the device. Most content is placed proactively during off-peak fill windows based on predicted viewing, and appliances can fill from each other. Check the overview for the current details.

7.4 Deep dive 4: purge and invalidation

Problem. Tenants need changes visible quickly, but deleting objects everywhere at once can send a thundering herd to the origin.

Choice. B as the default advice for static assets; D for content updates; C reserved for legal and security cases.

Reason. Soft purge keeps the stale copy available, so the first request revalidates with a conditional request (often a 304) while others are served stale within the tenant's stale-while-revalidate window. Purges travel through the distributor as an ordered log; each PoP confirms the sequence number it applied, which gives the 5 s p99 measurement and lets a PoP that was offline replay what it missed before it rejoins anycast.

7.5 Cache key design

The cache key decides correctness and hit ratio. The default key is scheme, normalised host, path and a sorted allow-list of query parameters.

8. Scalability and performance

Latency budget for a cache hit (target p75 50 ms time to first byte):

9. Reliability, security and operations

9.1 Failure modes

9.2 Security

9.3 Observability

9.4 Cost estimate

10. Trade-offs and alternatives

10.1 Decisions

10.2 Alternatives considered

• Commercial multi-CDN only. Lowest effort; at this scale egress pricing and lack of ISP placement were the deciding costs. Kept as overflow and as the 0.1x answer.
• Peer-to-peer delivery between viewers. Reduces egress for live events but adds client complexity and privacy concerns; not pursued.

11. Delivery plan and risks

11.1 Milestones

11.2 Rollout and rollback

[Example: Traffic is shifted per tenant and per country by weighted DNS between the owned CDN and commercial providers. Rollback moves the weight back within one DNS TTL (Example: 60 s).]

11.3 Risks

12. Appendices

12.1 Open questions

12.2 Glossary

12.3 References

Inspiration only for company write-ups; RFCs define the protocol behaviour.

Standard and licence note

Based on the IETF HTTP caching specifications RFC 9111, RFC 9213, RFC 9211 and RFC 5861 and the other RFCs listed in 12.3, and on common industry practice for content delivery networks. The Netflix overview is cited as inspiration only. This is a reference design to adapt, with illustrative numbers. Original text written for this collection; check the current official versions of the cited specifications. Template licence: MIT.