System Design: [Full-text search service name]

Reference design for a full-text search engine: inverted index built from immutable segments, near-real-time ingestion running concurrently with queries, sharding by document or by time, replicas for throughput and availability, BM25-style ranking with a second-phase reranker, the boundary with autocomplete, and an offline and online relevance evaluation loop. It follows the structure of the System Design Document template, adds a 45-minute interview path, access-pattern and failure-mode tables, and is pre-filled with a worked design to adapt. Replace every [bracketed] value and every row marked Example; values marked Example are assumptions, not benchmarks.

Document control

Approvals

Revision history

Contents

1. Overview
2. Requirements
3. Capacity estimation
4. API design
5. Data model
6. High-level design
7. Detailed design
8. Scalability and performance
9. Reliability, security and operations
10. Trade-offs and alternatives
11. Delivery plan and risks
12. Appendices

1. Overview

1.1 Problem statement

[Example: Users search 2 billion documents (posts, comments and help articles) through a relational database LIKE query on titles only. Queries time out above a few million rows, body text is not searchable, results are ordered by date only, and a new post takes up to an hour to appear because a nightly job copies data into a secondary table.]

1.2 Proposed solution

[Example: A change stream from the primary database feeds an indexing pipeline that analyses text (tokenise, normalise, stem per language) and writes into a document-partitioned inverted index. Each shard is a set of immutable segments plus one small in-memory segment that receives new documents and becomes searchable within one second. Queries go to a broker that fans out to one copy of every shard, gathers top results, and passes the best candidates to a reranking step. Each shard has three copies across zones. A judged query set and online experiments measure every ranking change.]

1.3 Decisions requested from reviewers

1.4 45-minute interview path

Use this order when the document serves an interview or a short review.

Draw the inverted index early (term to postings list) and explain why it makes keyword lookup cheap. Most weak answers stop at "use Elasticsearch"; strong answers explain segments, refresh and fan-out.

2. Requirements

2.1 Core functional requirements

2.2 Non-functional requirements

2.3 Below the line (out of scope)

• [Example: typing suggestions and query completion (see 7.5 and the autocomplete design)]
• [Example: semantic or vector search (planned as a hybrid extension, see 10)]
• [Example: web crawling; documents arrive from our own databases only]
• [Example: cross-language retrieval]
• [Item deferred, with the release that will take it]

3. Capacity estimation

3.1 Assumptions

3.2 Index size and shard count

Consequence: 40 shards, sized so a lost copy recovers in minutes, not hours. Plan for growth by choosing the shard count for the two-year corpus or by using time-based indices (7.3) so new data goes to new shards.

3.3 Write rate

Consequence: the write rate is moderate, but every write must become visible within 1 s, so the cost is in frequent refreshes and in merging the many small segments they create, not in raw throughput.

3.4 Query fan-out and replicas

With document partitioning every query visits one copy of every shard:

That is 2 copies per shard for throughput; a third copy covers a zone failure. Consequence: 3 copies per shard (120 shard copies, 6 TB of index). At 10 times the query rate we add replicas, not shards; at 10 times the corpus we add shards.

3.5 Summary

4. API design

4.1 Endpoints

4.2 Request and response examples

A search request with a filter and a cursor, and its response:

{"q": "refund \"late delivery\"", "filters": {"type": ["post", "article"], "lang": "en", "created_after": "2026-01-01"}, "sort": "relevance", "page_size": 20, "cursor": null}

{"total_estimate": 18420, "partial": false, "results": [{"id": "doc_7c1e", "score": 12.84, "title": "[Title]", "snippet": "our <em>refund</em> policy for <em>late delivery</em> orders", "created_at": "2026-09-14T10:02:00Z"}], "next_cursor": "[opaque cursor]", "index_version": "posts-v12"}

4.3 Errors and limits

Deep pagination uses a cursor (search after the last sort value), not offsets: an offset of 10 000 forces every shard to rank 10 020 results.

5. Data model

5.1 Entities

The source database owns documents; the index is a derived, rebuildable view.

5.2 Index mapping

5.3 Access patterns

6. High-level design

7. Detailed design

7.1 Deep dive: the inverted index and segments

An inverted index maps each term to a postings list: the documents that contain it, with term frequency and, for phrase queries, positions. Query evaluation intersects (AND) or unions (OR) postings lists, so cost follows the length of the lists touched, not the size of the corpus.

Choice: immutable segments, the model used by Lucene and its derivatives. Reason: a segment never changes after it is written, so it can be compressed, cached and read without locks; updates are a delete mark plus a new version in a newer segment.

7.2 Deep dive: real-time ingestion against concurrent queries

The hard part is making a document searchable within 1 s while thousands of queries read the same shard.

Choice: periodic refresh every 1 s for posts and 15 min for help articles, with a transaction log for durability between refreshes. Reason: it meets NFR-02 with a standard engine. A custom single-writer design is only worth building when sub-second freshness at very high write rates is the product (see 7.8).

Version numbers from the source make ingestion idempotent: an operation with a version lower than the indexed one is ignored, so replays after a failure are safe.

7.3 Deep dive: sharding by document, term or time

Choice: by document within each index, plus time-based indices for the message corpus. Reason: term partitioning makes every document write a distributed write; document partitioning keeps writes local and scales query throughput with replicas. Time indices let old weeks be merged to one segment, moved to cheaper nodes and dropped by retention without deletes.

Control the fan-out tail: per-shard timeout (Example 110 ms, the shard budget in section 8), return partial results with a flag, and pick the copy with the shortest queue (adaptive replica selection) rather than round robin.

7.4 Deep dive: ranking

The first phase scores every matching document cheaply on each shard; the second phase re-scores a few hundred candidates with richer signals.

BM25 is the standard first-phase text score. For query terms $q_i$, document $D$ with length $|D|$ and average length $avgdl$:

$k_1$ controls how quickly repeated terms stop adding score and $b$ controls length normalisation; Lucene's defaults are $k_1 = 1.2$ and $b = 0.75$. Tune them only against the judged set.

Choice: two phase. Reason: the reranker sees only 200 candidates per query, so its cost is bounded and independent of corpus size. Note that IDF is computed per shard; with hash partitioning shard statistics are close to global, but small shards or time indices can skew them. Use global statistics if scores vary visibly between shards.

7.5 Autocomplete boundary

Typing suggestions and full search look similar but have different contracts: suggestions answer every keystroke in tens of milliseconds from a precomputed prefix structure built from popular queries, while full search evaluates the whole query against the corpus. Keep them separate services. Search contributes its query logs to the suggestion pipeline described in Search Autocomplete System Design (system-design/design-search-autocomplete), and suggestions submit a full query here when selected.

7.6 Relevance evaluation

Sample judged queries from logs across head, torso and tail. Record the engine, mapping and model version with every evaluation run so results can be compared over time.

7.7 Deletes, access control and reindexing

• Deletes mark the document in a per-segment bitmap and disappear from results at the next refresh; space is reclaimed by merges.
• Access filters run on the shard using acl_principals; the API also checks access when it loads documents from the source, so a stale ACL in the index cannot leak a document. Total estimates, facet counts and snippets still come from the index, so ACL changes travel on the same 1 s path as document changes and snippets are shown only for documents that pass the load-time check.
• Mapping changes that alter analysis need a full reindex: build a new index from the source, replay the change stream from the build start, then switch the alias. Keep the old index until the new one passes evaluation.

7.8 How real systems do it

How real systems do it (paraphrased, not copied). Busch, Gade, Larson, Lok, Luckenbill and Lin describe Earlybird, the retrieval engine behind Twitter's real-time search (ICDE 2012). It is built on Lucene but organises postings for traversal from newest to oldest, keeps several segments per server of which only one accepts new documents at a time, converts a full segment into a read-optimised form, and uses a single-writer, many-reader concurrency model with targeted memory barriers so indexing and queries run at once without locks. Front-end servers fan queries out to the partitions and merge results. The lesson for this design: when freshness is the product, make the write path single-threaded per segment and let readers proceed without coordination. Read the paper in 12.3; this box keeps only the general idea.

8. Scalability and performance

Latency budget for a search (target p99 200 ms):

9. Reliability, security and operations

9.1 Failure modes

9.2 Security and privacy

9.3 Observability

9.4 Cost estimate

10. Trade-offs and alternatives

11. Delivery plan and risks

11.1 Milestones

11.2 Rollout and rollback

[Example: Search reads through an alias. New indices are built in parallel, evaluated offline, then shadow-queried with production traffic for 3 days. Switching the alias is the release; switching it back is the rollback.]

11.3 Risks

12. Appendices

12.1 Open questions

12.2 Glossary

12.3 References

Standard and licence note

Based on common industry practice for inverted-index search engines and on the published research cited in 12.3. The Earlybird paper and other engineering write-ups are inspiration only; no text, diagrams or numbers were copied, and all figures here are illustrative examples. Check the current documentation of your search engine for defaults and limits. Original text written for this collection. Template licence: MIT.