System Design: [Product name] recommendation system

Reference design for a large-scale recommendation system that ranks items (videos, products, articles or posts) for each user. It covers the multi-stage funnel of candidate generation, pre-ranking, ranking and re-ranking; embeddings and approximate nearest neighbour (ANN) retrieval; feature freshness; the training pipeline; online serving within a latency budget; exploration; A/B testing; and feedback loops and bias. It follows the structure of the System Design Document template and is pre-filled with a worked design to adapt. Replace every [bracketed] value and every row marked Example; values marked Example are assumptions, not benchmarks. This is an engineering format; recommender transparency and profiling rules depend on your jurisdiction and are not legal advice.

Document control

Approvals

Revision history

Contents

1. Overview
2. Requirements
3. Capacity estimation
4. API design
5. Data model
6. High-level design
7. Detailed design
8. Scalability and performance
9. Reliability, security and operations
10. Trade-offs and alternatives
11. Delivery plan and risks
12. Appendices

1. Overview

1.1 Problem statement

[Example: Our video app has 100 million daily users and a catalogue of 50 million items, with 400 000 new uploads a day. The home feed is ranked by a popularity score per category, so new creators get almost no exposure and returning users see the same items for days. Product wants longer sessions without growing clickbait complaints. A nightly batch writes a fixed list per user, so the feed ignores what the user watched five minutes ago.]

1.2 Proposed solution

[Example: A request-time funnel: several candidate sources (two-tower embedding retrieval through an ANN index, co-visitation, subscriptions, fresh uploads, trending) return about 1 500 candidates; a light pre-ranker cuts them to 300; a deep ranking model scores those with user, item and context features predicting several engagement and satisfaction targets; a re-ranker applies diversity, freshness, policy filters and exploration slots and returns a page of 20. User features update from a streaming pipeline within a minute. Every impression is logged with position and model versions; training is daily for ranking and weekly for retrieval; every change ships through an A/B test with guardrails.]

1.3 Decisions requested from reviewers

1.4 45-minute interview path

2. Requirements

2.1 Core functional requirements

2.2 Non-functional requirements and objective

2.3 Below the line (out of scope)

• [Example: search ranking (shares features, separate design)]
• [Example: ads selection and auction (blended by a separate mixer)]
• [Example: content moderation classifiers (consumed as item flags)]
• [Example: notification send-time optimisation (see Notification Service System Design (system-design/design-notification-service))]
• [Example: the model development record (see ML System Design Document (ai-data/ml-system-design))]
• [Item deferred, with the release that will take it]

3. Capacity estimation

3.1 Assumptions

3.2 Serving throughput

3.3 Index and storage

3.4 Training

3.5 Summary

4. API design

4.1 Endpoints

4.2 Request and response examples

GET /v1/feed?surface=home&cursor=c_19a
Authorization: Bearer [token]
X-Client-Context: {"device": "tv", "locale": "en-GB", "network": "wifi"}

HTTP/1.1 200 OK
Content-Type: application/json

{"request_id": "req_7b3f", "items": [{"item_id": "it_501", "impression_id": "imp_7b3f_01", "position": 1, "source": "two_tower"}, {"item_id": "it_88a", "impression_id": "imp_7b3f_02", "position": 2, "source": "explore"}], "next_cursor": "c_19b"}

• The client sends an impression event only when an item was actually visible (Example: 50% of the tile on screen for 1 s), carrying the impression_id. Requests do not count as impressions.
• request_id ties events to the model versions and features used, which the server logs at serving time.

5. Data model

5.1 Entities

Users and items have features and embeddings; requests produce impressions; impressions produce engagements. Training joins all four.

5.2 Storage choices

5.3 Access patterns

6. High-level design

7. Detailed design

7.1 Deep dive: candidate generation and ANN retrieval

Choice: two-tower retrieval plus heuristic sources, each with a quota. Reason: two-tower gives personalised recall across the long tail; heuristic sources guarantee freshness, subscriptions and robustness when the model or index fails.

Index options:

Choice: HNSW over 8-bit quantised vectors, rebuilt weekly with the retrieval model and updated incrementally hourly for new items. Reason: low latency at high recall, and incremental inserts meet NFR-04.

• The user tower and item embeddings must come from the same model version; the orchestrator pins a version per request and the index is swapped atomically.
• Train with in-batch and sampled negatives, correcting for item popularity in the sampling; otherwise popular items appear too often as negatives and end up under-recommended.
• New items with no engagement get embeddings from content features (title, tags, creator), so they are retrievable on day one.

How real systems do it (paraphrased from a public write-up; not copied)

The YouTube recommendations paper by Covington, Adams and Sargin describes a two-stage system. A candidate generation network uses a user's watch and search history plus context to narrow a corpus of millions of videos to a few hundred, framed as a very large classification over videos whose learned embeddings can be served with nearest neighbour search. A separate ranking network then scores those candidates with many more features, and is trained to predict expected watch time rather than clicks, so that misleading thumbnails are not rewarded. The authors also feed the age of each training example to the model so it can learn that users prefer fresh content, and they note that offline metrics did not always predict live A/B results. This template keeps the same split and the watch-time-style objective, and adds heuristic sources, a pre-ranker and exploration.

Source: Paul Covington, Jay Adams, Emre Sargin, "Deep Neural Networks for YouTube Recommendations", Proceedings of the 10th ACM Conference on Recommender Systems (RecSys 2016), pp. 191 to 198, https://doi.org/10.1145/2959100.2959190

7.2 Deep dive: feature freshness

Options: recompute everything in batch; streaming for all; tiered by value. Choice: tiered. Reason: session features carry most of the "react to what I just did" value (FR-03), while long-term features change slowly and are cheaper in batch. Log every feature value used at serving time with the request, so training reads what serving saw (AP-05) and train-serve skew is avoided.

7.3 Ranking, objective and re-ranking

• The ranker is a multi-task model predicting several targets per item: Example: probability of play, expected watch time, probability of like, probability of "not interested". The final score is a weighted blend owned by product (Q-01), versioned like code.
• Position is a training feature (set to a constant at serving) so the model learns item quality separately from position bias.
• The re-ranker enforces: policy filters (NFR-07), dedupe against recent impressions, at most Example 2 items per creator per page, a minimum share of fresh items, and exploration slots (7.4).

Choice: blend of watch time and predicted satisfaction, judged in experiments by long-term retention and guardrails. Reason: short-term proxies drift from user value; anchoring the experiments on retention keeps the proxy honest.

7.4 Exploration and cold start

Choice: 2 of 20 slots use Thompson sampling over new and under-exposed items, plus a fresh-item quota in candidate generation. Reason: new items need impressions to learn about them, and logging the exploration probability (propensity) gives unbiased data for evaluation. New users get a short onboarding of topic choices and a popularity prior by region, updated after each action through session features.

7.5 A/B testing and evaluation

• Offline: recall at K for retrieval on held-out next actions; normalised ranking metrics and calibration for the ranker; counterfactual estimates from logged propensities for re-ranking changes.
• Online: user-level randomisation; primary metric NFR-05; guardrails NFR-06; latency and error rate. Use A/B Test Plan (product/ab-test-plan) to fix hypothesis, sample size and duration before launch.
• Keep a long-term holdout (Example: 1% of users on a frozen baseline for a quarter) to measure cumulative effects that weekly tests miss.
• Interference: recommendations change what creators upload and what others see; for creator-side metrics consider cluster or time-split designs.

7.6 Feedback loops and bias

Monitor these with Model Monitoring Plan (ai-data/model-monitoring-plan), including distribution of impressions across creators and topics, not only model accuracy.

8. Scalability and performance

Latency budget for a feed page (p99 200 ms):

9. Reliability, security and operations

9.1 Failure modes

9.2 Security, abuse and privacy

9.3 Observability

10. Trade-offs and alternatives

11. Delivery plan and risks

11.1 Milestones

11.2 Risks

12. Appendices

12.1 Traceability

12.2 Open questions

12.3 Glossary

12.4 References

Standard and licence note

Based on common industry practice for large-scale recommender systems and on the public references in 12.4. Papers and company write-ups were used as inspiration only; the text, diagrams and numbers are original and illustrative. Rules on recommender transparency, profiling and minors differ by jurisdiction and change; check the current official texts. This is an engineering format, not legal advice. Template licence: MIT.