System Design: [News feed name]

Reference design for a social news feed: users follow other accounts and see a ranked, paginated feed of their posts. It covers hybrid fan-out (push for most authors, pull for accounts with very large followings), a ranking pipeline, feed caching, cursor pagination, media delivery and the consistency the product can tolerate. It follows the structure of the System Design Document template and is pre-filled with a worked design to adapt. Every value marked Example is an assumption: replace it and recompute.

Document control

Approvals

Revision history

Contents

1. Overview
2. Requirements
3. Capacity estimation
4. API design
5. Data model
6. High-level design
7. Detailed design
8. Scalability and performance
9. Reliability, security and operations
10. Trade-offs and alternatives
11. Delivery plan and risks
12. Appendices

1. Overview

1.1 Problem statement

[Example: The current feed is computed on every request by querying posts from all followed accounts. p99 latency is 2.4 s for users following more than 1 000 accounts, the database is at 80% CPU at peak, and the order is purely chronological, so engagement drops when users follow active accounts.]

1.2 Proposed solution

[Example: When an author with fewer than 10 000 followers posts, a fan-out service writes the post ID into each follower's precomputed feed in an in-memory store (push). Posts from accounts above the threshold are fetched at read time and merged (pull). A feed service combines both candidate sets, filters blocked and deleted content, ranks with a lightweight model and returns a page with an opaque cursor. Post bodies and media are hydrated from caches and a CDN.]

1.3 Decisions requested from reviewers

2. Requirements

2.1 Functional requirements

2.2 Non-functional requirements

2.3 Out of scope

• [Example: recommendations from accounts the user does not follow]
• [Example: ads insertion (separate design; reserved slots only)]
• [Example: comments and reactions storage]
• [Item deferred, with the release that will take it]

3. Capacity estimation

3.1 Assumptions

3.2 Throughput

Feed reads:

Posts:

Fan-out writes (feed inserts):

3.3 Why the celebrity case breaks push

One post from an account with $5 \times 10^{7}$ followers would require $5 \times 10^{7}$ inserts. At 420 000 inserts/s of spare capacity that single post would take about 2 minutes to fan out and would delay every other post behind it. This is why accounts above the threshold are pulled at read time.

3.4 Storage and memory

Posts per year:

Precomputed feeds in memory (all daily active users):

With one replica that is 9.6 TB of memory, for example about 150 nodes with 64 GB usable each. Reducing $L$ to 400 halves it; measure how deep users actually scroll before deciding.

Media (separate object store):

assuming 20% of posts carry media averaging 500 KB after compression.

3.5 Summary

4. API design

4.1 Endpoints

4.2 Request and response examples

A feed page returns hydrated items plus an opaque cursor for the next page:

GET /v1/feed?limit=20 HTTP/1.1
Authorization: Bearer [token]

{
  "items": [
    {"postId": "p_7a91", "author": {"id": "a_12", "name": "City Library"}, "text": "Late opening hours start Monday.", "media": [{"type": "image", "url": "https://media.example/p_7a91/0.webp"}], "createdAt": "2026-03-14T08:01:00Z", "reason": "followed"}
  ],
  "nextCursor": "eyJzbmFwIjoiczg4MiIsIm9mZiI6MjB9"
}

4.3 Errors, limits and versioning

• Rate limits: [Example: 50 posts per hour, 400 follows per day per user].
• Versioning: /v1 path prefix; feed item schema is additive only.

5. Data model

5.1 Entities

Posts, the follow graph and the per-user precomputed feed are the three core data sets; engagement events feed ranking.

5.2 Storage choices

6. High-level design

The write path (post, fan-out) and the read path (feed, rank, hydrate) share only the stores.

7. Detailed design

7.1 Feed read (critical path)

The feed service reads the pushed feed and pulled candidates in parallel, then ranks and hydrates only the page it returns.

If ranking times out (budget 60 ms), the service falls back to base_score order (recency with author affinity) so the user always gets a page.

7.2 Hybrid fan-out

• An account becomes a pull author when follower_count exceeds Example: 10 000, with hysteresis (back to push below 8 000) to avoid flapping.
• Push: on a post event, workers page through the author's followers (1 000 per batch) and insert (post_id, author_id, base_score) into each active follower's sorted set, trimming to $L$ entries.
• Pull: on read, the feed service fetches the latest posts of each pull author the user follows. Most users follow few such accounts; cap at Example: 200 pull authors per user, chosen by affinity.
• Inactive users (no session for 14 days) are skipped during fan-out. When they return, the feed is rebuilt from followees' recent posts in the background, and the first page is served by a pull-only path meanwhile.

7.3 Feed cache layout

Each user's feed is a sorted set keyed by feed:{user_id}, scored by a time-decayed base score so trimming keeps the most relevant recent items. Entries hold IDs only; bodies live in the post cache so edits and deletes take effect without rewriting feeds. A deletion event adds the post to a short-lived tombstone set consulted by the filter step, and a background job removes the entries.

7.4 Ranking pipeline

A typical score combines predicted actions with weights and an age decay:

where $h$ is the half-life, for example 6 hours. Weights $w_a$ are set by product goals and reviewed for integrity effects (for example down-weighting predicted outrage).

7.5 Pagination and consistency model

• The first page call stores the ranked list of up to 500 IDs as a snapshot for 30 minutes; the cursor encodes snapshot_id and offset. Later pages read the snapshot, so new posts never shift items and cause duplicates or skips.
• New content is surfaced through new-count, and pulling to refresh creates a new snapshot.
• Consistency: feeds are eventually consistent. A post appears in pushed feeds within seconds; follows take effect on the next snapshot; unfollows and blocks are enforced by the filter step at read time, so they apply immediately even before fan-out entries are cleaned up.
• Read-your-writes: the author's own new posts are injected into their first page from the post service, independent of fan-out lag.

7.6 Media

Clients upload media directly to object storage with pre-signed URLs; a processing job produces resized WebP/AVIF variants and video renditions, and the post becomes visible only when its media are ready. Feed items reference CDN URLs with content-hash names so they can be cached for a year.

8. Scalability and performance

Latency budget for a first feed page (target p99 250 ms):

9. Reliability, security and operations

9.1 Failure modes

9.2 Security and abuse

9.3 Observability

9.4 Cost estimate

10. Trade-offs and alternatives

11. Delivery plan and risks

11.1 Milestones

11.2 Rollout and rollback

[Example: Dual-write feeds for two weeks while the old query path serves traffic; compare outputs on 1% of requests. Switch reads per percentage of users; rollback switches reads back while dual writes continue.]

11.3 Risks

12. Appendices

12.1 Open questions

12.2 Glossary

12.3 References

Standard and licence note

Based on common industry practice for social feed system design. This is a reference design to adapt: the numbers are illustrative assumptions, not measurements. Original text written for this collection. Template licence: MIT.