# System Design: [File sync service name]

> Reference design for a cloud file sync service in the style of consumer
> and business "sync folder" products: a desktop and mobile client keeps a
> local folder in step with the cloud and with the user's other devices. It
> covers content-defined chunking (as introduced by the Low-Bandwidth File
> System, SOSP 2001), content-addressed deduplication and its privacy side
> channel, the commit protocol between metadata and block storage, change
> journals and push notifications, and conflict handling with a three-tree
> model. It follows the structure of the System Design Document template,
> adds a 45-minute interview path, access-pattern and failure-mode tables and
> a cited "How real systems do it" box that paraphrases public engineering
> write-ups, and is pre-filled with a worked design to adapt. Replace every
> `[bracketed]` value and every row marked *Example*; values marked *Example*
> are assumptions, not benchmarks.

<!--
Guidance: How to use this template
- Write down the invariant first: the service must never lose or silently
  overwrite a user's file. Every design choice below is judged against it.
- Separate metadata (names, folders, versions) from content (chunks) early;
  they have different sizes, access patterns and consistency needs.
- Fill 3.1 with real usage data: files per user and the edit rate drive
  metadata sharding; stored bytes drive the blob store.
- Decide conflict behaviour (7.4) with product and support before coding;
  users judge the product by what happens when two devices disagree.
-->

## Document control

<!-- Guidance: A design is a proposal until it is approved; record who decided and when. -->

| Field | Value |
|---|---|
| Document ID | [SDD-SYNC-001] |
| System / feature | [File sync and sharing] |
| Version | [0.1] |
| Status | [Draft / In review / Approved / Implemented / Superseded] |
| Owner | [@owner] |
| Author(s) | [@author] |
| Reviewers | [@reviewer-1], [@reviewer-2] |
| Last updated | YYYY-MM-DD |
| Classification | [Internal] |
| Difficulty | Advanced |
| Clients in scope | [Windows, macOS, Linux desktop; iOS and Android on-demand] |
| Related | [Object storage design], [Notification service design], [ADR: chunking scheme] |

### Approvals

| Role | Name | Decision | Date |
|---|---|---|---|
| Tech lead | [@name] | [Approved / Changes requested] | YYYY-MM-DD |
| Staff / principal engineer | [@name] | [Decision] | YYYY-MM-DD |
| Client platform lead | [@name] | [Decision] | YYYY-MM-DD |
| SRE / storage | [@name] | [Decision] | YYYY-MM-DD |
| Security and privacy | [@name] | [Decision] | YYYY-MM-DD |

## Revision history

<!-- Guidance: Note what changed in the design, not the wording. -->

| Version | Date | Author | Change summary |
|---|---|---|---|
| 0.1 | YYYY-MM-DD | [@name] | First draft for design review |
| 0.2 | YYYY-MM-DD | [@name] | *Example:* Switched from fixed 4 MB blocks to content-defined chunks after the edit-pattern study |
| 0.3 | YYYY-MM-DD | [@name] | *Example:* Limited deduplication to one account after the privacy review |
| 1.0 | YYYY-MM-DD | [@name] | *Example:* Approved |

## Contents

<!-- Guidance: Keep aligned with the headings. -->

1. [Overview](#1-overview)
2. [Requirements](#2-requirements)
3. [Capacity estimation](#3-capacity-estimation)
4. [API design](#4-api-design)
5. [Data model](#5-data-model)
6. [High-level design](#6-high-level-design)
7. [Detailed design](#7-detailed-design)
8. [Scalability and performance](#8-scalability-and-performance)
9. [Reliability, security and operations](#9-reliability-security-and-operations)
10. [Trade-offs and alternatives](#10-trade-offs-and-alternatives)
11. [Delivery plan and risks](#11-delivery-plan-and-risks)
12. [Appendices](#12-appendices)

## 1. Overview

<!-- Guidance: One page that a busy reviewer can read alone: problem, proposed solution in a paragraph, and the decisions that need their attention. -->

### 1.1 Problem statement

[*Example:* Our first sync client uploads whole files on every save and polls
the server every 60 seconds. A one-line change to a 200 MB design file
re-uploads 200 MB, laptops on hotel Wi-Fi fall hours behind, and when two
devices edit the same file the later upload silently replaces the earlier
one. Support logs about 400 "lost edit" tickets a month.]

### 1.2 Proposed solution

[*Example:* The client splits files into content-defined chunks (average
1 MB, bounded 256 KB to 4 MB) and identifies each chunk by its SHA-256 hash.
A save becomes a commit of the file's ordered chunk list to the metadata
service; the server answers with the chunks it does not already have in the
account, the client uploads only those to the block service, and the commit
completes with an optimistic-concurrency check on the file's parent
revision. Each committed change is appended to a per-namespace journal, and
a notification service tells the user's other online devices to pull
changes since their cursor. Conflicting edits never overwrite: the losing
version is saved beside the original as a conflicted copy.]

### 1.3 Decisions requested from reviewers

| ID | Question | Recommended option | Section |
|---|---|---|---|
| Q-01 | *Example:* Fixed blocks or content-defined chunks? | Content-defined, average 1 MB | 7.1 |
| Q-02 | *Example:* Deduplicate across all users or per account? | Per account, to close the cross-user side channel | 7.2 |
| Q-03 | *Example:* How do devices learn about changes? | Persistent connection per device plus journal cursor; poll as fallback | 7.3 |
| Q-04 | *Example:* What happens on a conflicting edit? | Keep both; the later commit becomes a conflicted copy | 7.4 |
| [Q-nn] | [Question] | [Option] | [Section] |

### 1.4 45-minute interview path

Use this order when the document serves an interview or a short design
review. The written sections below are the long form of each step.

| Minutes | Cover | Output on the whiteboard |
|---|---|---|
| 0 to 5 | Scope: upload, sync across devices, share a folder, version history; below-the-line items (2.3) | Four user actions |
| 5 to 10 | Estimates: stored bytes, commits per second, connected devices, metadata rows (section 3) | Four numbers with consequences |
| 10 to 15 | Interface: commit with chunk list, upload chunk, list changes since cursor (section 4) | Three calls |
| 15 to 25 | High-level design: client, metadata service, block service, object storage, notifications (section 6) | One diagram |
| 25 to 40 | One deep dive: chunking and the commit protocol (default, 7.1 and 7.5), or conflict handling (7.4), or notifications at scale (7.3) | Options, choice, reason |
| 40 to 45 | Failures (partial upload, metadata shard down), the 10x row in section 10 | Two failure rows |

Start the default deep dive with the naive answer, upload the whole file on
every change, and show the cost of a small edit to a large file. Then try
fixed-size blocks and show that inserting one byte shifts every later block.
Finish with content-defined chunking and the "commit, need chunks, upload,
commit again" protocol.

## 2. Requirements

<!-- Guidance: Split core requirements from below-the-line items so the design stays focused. Non-functional targets need numbers and a way to measure them. -->

### 2.1 Core functional requirements

| ID | Requirement | Priority |
|---|---|---|
| FR-01 | *Example:* Sync additions, edits, renames, moves and deletes between the cloud and every linked device | Must |
| FR-02 | *Example:* Upload only changed content after a small edit to a large file | Must |
| FR-03 | *Example:* Propagate a change to the user's other online devices quickly | Must |
| FR-04 | *Example:* Never silently overwrite a concurrent edit; keep both versions | Must |
| FR-05 | *Example:* Share a folder with other users with view or edit rights | Must |
| FR-06 | *Example:* Keep file versions and deleted files restorable for 30 days | Must |
| FR-07 | *Example:* Resume interrupted uploads and downloads | Must |
| FR-08 | *Example:* Selective sync and files-on-demand placeholders on desktop | Should |
| [FR-nn] | [Requirement] | [Must / Should / Could] |

### 2.2 Non-functional requirements

| ID | Quality | Target | Measured as |
|---|---|---|---|
| NFR-01 | Durability | *Example:* No committed file version lost; 11 nines object durability from the storage layer | Storage provider figures plus audit scrubbing |
| NFR-02 | Sync latency | *Example:* Small change visible on another online device within 5 s (p95) | Synthetic two-device canary |
| NFR-03 | Bandwidth efficiency | *Example:* A 1 KB insertion in a 200 MB file uploads at most 8 MB | Client benchmark |
| NFR-04 | Correctness | *Example:* Zero silent overwrites; every conflict produces a conflicted copy | Randomised simulation tests (7.6) |
| NFR-05 | Availability | *Example:* Metadata API 99.95% monthly; clients keep working offline | API success rate |
| NFR-06 | Client footprint | *Example:* Idle CPU under 1%; hashing throttled on battery | Client telemetry (opt-in) |
| [NFR-nn] | [Quality] | [Target] | [Measurement] |

### 2.3 Below the line (out of scope)

Name these explicitly so reviewers and interviewers do not pull the design
toward them.

- [*Example:* real-time co-editing inside documents (see the collaborative editing design)]
- [*Example:* full-text search over file contents]
- [*Example:* client-side end-to-end encryption (would disable server-side deduplication and previews)]
- [*Example:* the object storage system itself (see `system-design/design-object-storage`)]
- [Item deferred, with the release that will take it]

## 3. Capacity estimation

<!-- Guidance: Back-of-the-envelope numbers decide the shape of the design. Show inputs, formulas and results; round generously and say which way you rounded. -->

### 3.1 Assumptions

| Input | Symbol | Value | Source |
|---|---|---|---|
| Registered active users | $U$ | *Example:* 10 million | Product plan |
| Daily active users | $D$ | *Example:* 4 million | Product plan |
| Linked devices per user | $v$ | *Example:* 2.5 | Pilot data |
| Average stored bytes per user | $u$ | *Example:* 12 GB | Pilot data |
| Saving from per-account deduplication | $g$ | *Example:* 20% | Pilot data |
| Erasure-coding overhead | $o$ | *Example:* 1.4x | Storage layer |
| File commits per daily active user per day | $c$ | *Example:* 20 | Pilot data |
| New bytes uploaded per commit (after dedupe) | $n$ | *Example:* 0.6 MB | Pilot data |
| Other devices that download each change | $m$ | *Example:* 1.5 | Pilot data |
| Files per user | $F_u$ | *Example:* 8 000 | Pilot data |
| Share of a daily user's devices online at peak | $w$ | *Example:* 0.6 | Pilot data |

### 3.2 Storage

$$
S_{logical} = U \times u = 10^{7} \times 12 \ \text{GB} = 120 \ \text{PB}, \qquad
S_{raw} = S_{logical} \times (1 - g) \times o = 120 \times 0.8 \times 1.4 = 134.4 \ \text{PB}
$$

Consequence: content belongs in an erasure-coded object store; the sync
service never stores bytes itself, only chunk references.

### 3.3 Commit and transfer rate

$$
C = D \times c = 4 \times 10^{6} \times 20 = 8 \times 10^{7} \ \text{commits/day}, \qquad
\bar{r} = \frac{8 \times 10^{7}}{86\,400} \approx 926/\text{s}, \qquad r_{peak} = 3\bar{r} \approx 2\,778/\text{s}
$$

| Flow | Formula | Per day | Rate |
|---|---|---|---|
| Upload | $C \times n$ | 48 TB | 4.4 Gbit/s average, 13.3 Gbit/s at peak |
| Download | $C \times n \times m$ | 72 TB | 6.7 Gbit/s average |
| Change notifications | $C \times m$ | 120 million | About 4 170/s at peak |

Consequence: uploads and downloads go straight between clients and the
block service in the nearest region; metadata servers never proxy bytes.

### 3.4 Metadata

$$
N_{files} = U \times F_u = 10^{7} \times 8\,000 = 8 \times 10^{10}, \qquad
M = 8 \times 10^{10} \times 300 \ \text{B} = 24 \ \text{TB}
$$

With an average of 1.6 chunks per file (most files are one small chunk,
large files have many), files reference about 128 billion chunks, or about
102 billion unique chunks after the 20% deduplication saving; a chunk index
entry of 64 B makes the chunk index about 6.6 TB. A commit
costs about 5 metadata writes (file row, version row, chunk references,
journal entry, quota), so peak writes are about 13 900/s. Consequence: shard
metadata by namespace from day one; 64 shards hold about 375 GB and take
about 217 writes/s each at peak.

### 3.5 Connected devices

$$
N_{conn} = D \times v \times w = 4 \times 10^{6} \times 2.5 \times 0.6 = 6 \times 10^{6}
$$

At 100 000 idle persistent connections per notification server that is 60
servers, or 90 with 1.5x headroom. Consequence: notifications are a separate
tier from the metadata API.

### 3.6 Summary

| Quantity | Estimate | Design consequence |
|---|---|---|
| Stored content | 120 PB logical, 134.4 PB raw | Erasure-coded object storage |
| Commits | About 2 800/s at peak | Metadata sharded by namespace |
| Transfer | 48 TB up, 72 TB down per day | Direct client-to-block-service transfers |
| Metadata | 80 billion file rows, 24 TB | 64 shards to start |
| Connections | 6 million concurrent | Dedicated notification tier, 90 servers |

## 4. API design

<!-- Guidance: Define the contract first: resources, methods, request and response shapes, errors, idempotency, pagination, rate limits and versioning. Use RFC 9457 problem details for errors. -->

### 4.1 Interfaces

| Interface | Operation | Purpose |
|---|---|---|
| Metadata API | `POST /v1/namespaces/{ns}/commits` | Commit a file version as an ordered chunk list |
| Block API | `PUT /v1/chunks/{sha256}` (or a pre-signed URL) | Upload one chunk; idempotent by hash |
| Block API | `GET /v1/chunks/{sha256}` | Download one chunk |
| Metadata API | `GET /v1/namespaces/{ns}/changes?cursor=` | List journal entries after a cursor, paged |
| Notification | `WS /v1/notify` with namespace cursors | Push "namespace changed" signals |
| Metadata API | `POST /v1/namespaces/{ns}/moves` | Atomic move or rename by file ID |
| Sharing API | `POST /v1/shares` | Mount a shared namespace for another user |

### 4.2 Commit request and responses

The client sends the path, the stable file ID, the revision it edited (its
parent) and the chunk hashes in order. The commit is idempotent through the
`Idempotency-Key` header so a retried commit never creates two versions.

```json
{"file_id": "f_5Q2M8X", "path": "/Projects/brochure.indd", "parent_rev": 41, "size": 209715200, "chunks": ["9a1f3c", "77be02", "c4d815"], "client_mtime": "2026-10-30T08:12:44Z", "device_id": "dev_mac_02"}
```

If any chunk is unknown in the account, the server answers `409` with the
missing hashes, and the client uploads them and retries:

```json
{"type": "https://api.example.com/problems/need-chunks", "title": "Chunks missing", "status": 409, "missing": ["c4d815"]}
```

If `parent_rev` is no longer the latest revision, the server answers with a
conflict problem that carries the current revision, and the client follows
7.4. The chunk hashes above are shortened for the example; real requests
carry full 64-character SHA-256 values.

## 5. Data model

<!-- Guidance: Choose storage by access pattern, not by habit. Show entities, keys, partitioning and the queries each store must answer. -->

### 5.1 Entities

A namespace is a user's root or a shared folder; it is the unit of
sharding, journaling and permission.

```mermaid
erDiagram
  NAMESPACE {
    string ns_id PK
    string owner_account_id
    bigint latest_journal_id
  }
  FILE {
    string ns_id PK
    string file_id PK
    string path
    bigint current_rev
    bool deleted
  }
  FILE_VERSION {
    string file_id PK
    bigint rev PK
    bigint size
    string device_id
    datetime committed_at
  }
  VERSION_CHUNK {
    string file_id PK
    bigint rev PK
    int seq PK
    string chunk_hash FK
  }
  CHUNK {
    string account_id PK
    string chunk_hash PK
    int size
    int ref_count
    string blob_location
  }
  JOURNAL_ENTRY {
    string ns_id PK
    bigint journal_id PK
    string file_id
    string op
  }
  NAMESPACE ||--o{ FILE : contains
  FILE ||--|{ FILE_VERSION : "has versions"
  FILE_VERSION ||--|{ VERSION_CHUNK : "made of"
  CHUNK ||--o{ VERSION_CHUNK : "referenced by"
  NAMESPACE ||--o{ JOURNAL_ENTRY : "records"
```

### 5.2 Storage choices

| Data | Store | Key | Main queries | Why |
|---|---|---|---|---|
| Namespaces, files, versions, journal | *Example:* Sharded relational database, 64 shards | `ns_id` | Commit transaction, list changes after cursor | One namespace's commit is a single-shard transaction |
| Chunk index | Sharded key-value store | (`account_id`, `chunk_hash`) | Which chunks exist, reference counts | Point lookups, high volume |
| Chunk content | Erasure-coded object storage | Opaque blob key | Get and put by key | Durable and cheap at petabyte scale |
| Device cursors | Notification tier memory plus client local database | `device_id`, `ns_id` | Who to notify | Rebuilt from clients on reconnect |
| Client state | Local embedded database on each device | Path and file ID | Remote, local and synced trees (7.4) | Works offline |

### 5.3 Access patterns

| Access pattern | Frequency | Key or index that serves it | Partition key rationale |
|---|---|---|---|
| Commit a file version | About 2 800/s at peak | `ns_id` shard, row for (`ns_id`, `file_id`) | A namespace never spans shards, so the commit is one local transaction |
| Find missing chunks | Every commit | Chunk index by (`account_id`, `chunk_hash`) | Account scope matches the dedupe scope (7.2) |
| List changes since cursor | Every notification, every reconnect | Journal (`ns_id`, `journal_id`) range | Monotonic journal IDs per namespace make the cursor a single number |
| Download a chunk | 72 TB/day | Blob key from the chunk index | Object storage spreads keys itself |
| Restore a deleted file | Rare | (`ns_id`, `file_id`) with `deleted=true`, versions kept 30 days | Same shard as the live file |
| Fan out a change to devices | About 4 170/s at peak | Subscriptions by `ns_id` on notification servers | Shared folders notify every member's devices through one key |

## 6. High-level design

<!-- Guidance: One diagram that shows every component on the request paths and where state lives. Name components by responsibility. -->

Metadata and content take separate paths: small, consistent writes to the
metadata service; large, idempotent transfers to the block service.

```mermaid
flowchart LR
  subgraph client["Client device"]
    watch[File watcher] --> chunker[Chunker and hasher]
    chunker --> planner[Sync planner: three trees]
    planner --> localdb[(Local state DB)]
  end
  planner -->|"commit, list changes"| meta[Metadata service]
  planner -->|"upload and download chunks"| block[Block service]
  meta --> mdb[(Metadata DB, sharded by namespace)]
  meta --> cidx[(Chunk index)]
  block --> obj[(Object storage, erasure-coded)]
  block --> cidx
  meta -->|"namespace changed"| bus[[Change events]]
  bus --> notify[Notification service]
  notify -->|"push over WebSocket"| planner
  gc[Chunk garbage collector] --> cidx
  gc --> obj
```

| Component | Responsibility | Scales by |
|---|---|---|
| File watcher | Detect local changes from operating system events, with periodic rescans | One per device |
| Chunker and hasher | Split files into content-defined chunks and hash them | Device CPU, throttled |
| Sync planner | Compare remote, local and synced trees and plan uploads, downloads and conflicts | One per device |
| Metadata service | Commits, journal, moves, permissions | Stateless instances over 64 shards |
| Block service | Authenticated chunk transfer, hash verification | Stateless instances per region |
| Notification service | Hold device connections and push change signals | Connections, 100 000 per server |
| Chunk garbage collector | Delete chunks with zero references after the retention period | Batch workers |

## 7. Detailed design

<!-- Guidance: Go deep on the parts that are new, risky or contested. Show the sequence of calls and the failure behaviour of each step. -->

The deep dives in this section, each with options, choice and reason:

| Deep dive | Options considered | Choice | Reason | Section |
|---|---|---|---|---|
| Chunking | Whole file; fixed-size blocks; rsync-style rolling diff; content-defined chunks | Content-defined chunks, average 1 MB | Insertions change only nearby chunks | 7.1 |
| Deduplication scope | None; per account; global | Per account | Most saving, no cross-user side channel | 7.2 |
| Change notification | Polling; long polling; persistent connection | Persistent connection plus journal cursor | Seconds of latency without polling cost | 7.3 |
| Conflicts | Last writer wins; lock files; keep both | Keep both with a conflicted copy | No silent data loss | 7.4 |

### 7.1 Chunking

| Scheme | Small insertion in a 200 MB file | Notes |
|---|---|---|
| Whole file | 200 MB uploaded | Simple, wasteful |
| Fixed 4 MB blocks | 50 blocks; an insertion at 100 MB shifts every later boundary, so 25 blocks (100 MB) change | Simple, parallel hashing; good for appends and in-place edits only |
| Content-defined chunks, average 1 MB | About 200 chunks; boundaries move with the content, so about 2 chunks (about 2 MB) change | Boundaries chosen by a rolling hash over a small window |

Content-defined chunking slides a rolling hash (for example a Rabin
fingerprint) over a small window of bytes and declares a boundary wherever
the hash matches a chosen bit pattern. Because boundaries depend on content,
not on offsets, an insertion only affects the chunks around it. Minimum and
maximum sizes (here 256 KB and 4 MB) stop pathological inputs, such as long
runs of zeros, from producing tiny or huge chunks. Each chunk is hashed with
SHA-256; the hash is its identity everywhere in the system.

### 7.2 Deduplication and its side channel

When chunks are deduplicated across all users, an upload that completes
instantly tells the uploader that someone already stored that content. Harnik,
Pinkas and Shulman-Peleg (IEEE Security and Privacy, 2010) showed that this
lets an attacker test whether a specific file, or a file with a guessed
secret in it, exists in other users' storage. This design deduplicates within
one account only: versions of the same file, copies and moves are the bulk of
the saving (the 20% in 3.1), and no answer about other users' content ever
leaves the server. Shared folders deduplicate within the namespace owner's
account.

### 7.3 Change journal and notifications

Every commit appends an entry with the next `journal_id` to its namespace's
journal in the same transaction as the file update. A device keeps one cursor
(the last `journal_id` applied) per namespace. The notification service only
sends "namespace X changed"; the device then calls `list changes since
cursor`. Notifications are therefore hints: a lost or duplicated signal costs
at most one extra call, and correctness rests entirely on the journal. Clients
that cannot keep a WebSocket open fall back to long polling, and every client
also pulls on reconnect and every 10 minutes as a safety net.

### 7.4 Conflict handling with three trees

The client keeps three trees in its local database: the **remote** tree (what
the server last reported), the **local** tree (what is on disk) and the
**synced** tree (the last state where both agreed). The synced tree is the
merge base: comparing each side with it shows which side changed.

| Remote vs. synced | Local vs. synced | Action |
|---|---|---|
| Unchanged | Changed | Upload (commit with `parent_rev` from synced) |
| Changed | Unchanged | Download and apply |
| Changed | Changed, same content hash | Mark synced, nothing to transfer |
| Changed | Changed, different content | Conflict: keep remote as the file, save local as a conflicted copy |
| Deleted | Changed | Edit wins: re-upload as a new file at the same path |
| Changed | Deleted | Edit wins: download the remote version again |

Moves and renames use the stable `file_id`, so a rename on one device and an
edit on another merge cleanly. The server enforces the same rule: a commit
whose `parent_rev` is stale is rejected, never applied over the newer
revision. The conflicted copy is named
`[name] (conflicted copy, [device], YYYY-MM-DD).[ext]` so users can find it.

```mermaid
stateDiagram-v2
  [*] --> Synced
  Synced --> LocalChanged: local edit detected
  Synced --> RemoteChanged: journal entry received
  LocalChanged --> Uploading: commit with parent_rev
  Uploading --> Synced: commit accepted
  Uploading --> Conflict: parent_rev stale
  RemoteChanged --> Synced: download applied
  RemoteChanged --> Conflict: local edit before apply
  Conflict --> Synced: remote kept, local saved as conflicted copy
```

### 7.5 Commit protocol

```mermaid
sequenceDiagram
  participant A as Client A
  participant M as Metadata service
  participant B as Block service
  participant N as Notification service
  participant C as Client B
  A->>A: chunk and hash changed file
  A->>M: POST commit (parent_rev 41, chunk list)
  M-->>A: 409 need chunks [c4d815]
  A->>B: PUT chunk c4d815
  B->>B: verify SHA-256 matches name
  B-->>A: 201 stored
  A->>M: POST commit again (same Idempotency-Key)
  M->>M: check parent_rev, write version 42 and journal entry
  M-->>A: 200 committed rev 42
  M->>N: namespace changed
  N->>C: push signal
  C->>M: GET changes since cursor
  M-->>C: rev 42 with chunk list
  C->>B: GET missing chunks
```

The block service verifies that uploaded bytes hash to the name they were
uploaded under, so a buggy or hostile client cannot poison the chunk store.
Chunks uploaded but never committed are deleted by the garbage collector
after 24 hours.

### 7.6 Testing sync correctness

Sync bugs are rare interleavings: a rename during an upload, a delete during
a download, a crash between two steps. Run the planner with a deterministic
scheduler against a simulated file system and server, generate random
operation sequences from a seed, and check invariants after each step (no
file lost, trees converge, no silent overwrite). A failing seed reproduces
the bug exactly.

### 7.7 How real systems do it

> **How real systems do it** (paraphrased, not copied). In "Streaming File
> Synchronization" (Dropbox tech blog, Nipunn Koorapati, 10 July 2014),
> Dropbox describes splitting files into 4 MB blocks identified by SHA-256
> hashes. A client commits a file's list of block hashes to a metadata
> server, which replies with the blocks it still needs; the client uploads
> those to a block server and commits again. The post explains how letting
> other devices start fetching blocks before the commit finishes improved
> multi-device sync time. In "Rewriting the heart of our sync engine" (Sujay
> Jayakar, 9 March 2020), Dropbox explains why it rebuilt its desktop sync
> engine in Rust: files gained stable identifiers, moves became atomic, and
> most logic moved onto one deterministic control thread so that
> pseudorandom simulation tests could reproduce any failure from its seed. A
> companion post, "Testing sync at Dropbox" (Isaac Goldberg, 20 April 2020),
> describes the remote, local and synced trees used as a merge base, the
> model adopted in 7.4. This design differs on one point: it uses
> content-defined chunks instead of fixed blocks to cut re-uploads after
> insertions. Read the originals in 12.3.

## 8. Scalability and performance

<!-- Guidance: Show how the design meets the numbers in section 3 and where it will break first. -->

| Concern | Approach | Limit before redesign |
|---|---|---|
| Metadata writes | 64 shards by namespace | *Example:* about 2 000 writes/s per shard, 9x current peak |
| Very large shared namespaces | Journal paging; split hot namespaces onto dedicated shards | *Example:* namespaces above 50 million files |
| Chunk transfers | Parallel chunk uploads (4 per file), resumable by chunk | Client uplink |
| Notifications | 100 000 connections per server; subscriptions by namespace | Fan-out for namespaces shared with thousands of members |
| Client hashing | Rehash only files whose size or modification time changed | Very large files on slow disks |

Sync latency budget for a small change between two online devices (NFR-02,
5 s p95):

| Step | Budget |
|---|---|
| File watcher debounce | 1.0 s |
| Chunk and hash (small file) | 0.2 s |
| Commit, need chunks, upload, commit | 1.2 s |
| Notification push | 0.5 s |
| List changes and download | 1.0 s |
| Headroom | 1.1 s |

## 9. Reliability, security and operations

<!-- Guidance: Name the failure modes with their blast radius and recovery time, the threats, and the signals operators watch. Each mitigation should be testable. -->

### 9.1 Failure modes

| Component | Failure | Detection | Blast radius | Mitigation | RTO |
|---|---|---|---|---|---|
| Metadata shard | Primary fails | Commit error rate per shard | Namespaces on one of 64 shards cannot commit; clients queue locally | Synchronous replica failover; clients retry with idempotency keys | *Example:* 30 s |
| Block service | Region outage | Transfer error rate | Uploads and downloads in that region | Clients switch to another region; object storage is multi-zone | *Example:* 5 min |
| Notification tier | Server crash | Connection count drop | Devices on that server miss pushes | Reconnect with backoff, pull since cursor on reconnect | *Example:* 1 min |
| Client | Crash mid-upload | Not needed on server | One device's pending changes | Local state DB is transactional; resume per chunk | *Example:* next start |
| Garbage collector | Deletes a referenced chunk | Scrubber finds missing blob | Files that use the chunk | Two-phase delete (mark, wait 7 days, delete); restore from versioned bucket | *Example:* 4 h |
| Client release | Bug corrupts local trees | Spike in conflicted copies or rescans | Users on the new version | Staged rollout, kill switch, full rescan rebuilds trees | *Example:* 24 h |
| [Component] | [Failure] | [Signal] | [Scope] | [Mitigation] | [Time] |

### 9.2 Security and privacy

| Threat | Control |
|---|---|
| Learning other users' content through deduplication | Per-account dedupe scope (7.2) |
| Chunk poisoning (bytes that do not match the hash) | Block service verifies SHA-256 on upload |
| Stolen device keeps syncing | Device unlinking revokes tokens and can wipe the local folder at next contact |
| Over-sharing | Share links expire by default; audit log of share changes |
| Data at rest | Encryption at rest with keys per storage unit; TLS in transit |
| Malware in shared folders | Version history and bulk restore after mass changes; alert on mass encryption patterns |
| [Threat] | [Control] |

### 9.3 Observability

| Signal | Metric or log | Alert threshold |
|---|---|---|
| Sync latency | Canary pair end-to-end time | Above 5 s p95 for 10 min |
| Commit conflicts | Stale `parent_rev` rejections per 1 000 commits | Doubling vs. 7-day baseline |
| Conflicted copies created | Per client version | Spike after a release |
| Need-chunks ratio | Commits answered with missing chunks | Sudden drop (dedupe broken) or rise |
| Orphan chunks | Uploaded but uncommitted after 24 h | Above 1% of uploads |
| Scrubber errors | Blobs failing checksum | Any |

### 9.4 Cost estimate

| Item | Unit cost | Quantity | Monthly cost |
|---|---|---|---|
| Object storage | [$/PB-month] | 134.4 PB raw | [$] |
| Egress for downloads | [$/TB] | About 2 160 TB (72 TB x 30) | [$] |
| Metadata database | [$/shard-month] | 64 shards with replicas | [$] |
| Notification servers | [$/instance-month] | 90 | [$] |
| **Total** | | | **[$]** |

## 10. Trade-offs and alternatives

<!-- Guidance: For each major decision list the options, what each costs and why you chose one. Reviewers will propose the alternatives anyway; answer them here. -->

| Decision | Options | Chosen | Why | What we give up |
|---|---|---|---|---|
| Chunking | Fixed 4 MB; content-defined | Content-defined, average 1 MB | Insertions upload about 2 MB instead of 100 MB in the 7.1 example | About 4x more chunk references than 4 MB blocks |
| Dedupe scope | Global; per account | Per account | Closes the side channel | Lower saving on popular shared files |
| Consistency | Eventual everywhere; strong per namespace | Strong per namespace, journal ordered | Simple client logic, atomic moves | Cross-namespace moves need a two-step protocol |
| Notifications | Polling; push plus cursor | Push plus cursor | Seconds, not minutes | 6 million connections to run |
| Conflicts | Last writer wins; locks; keep both | Keep both | No silent loss | Users must merge conflicted copies |
| What changes at 10x / 0.1x | 10x (about 27 800 commits/s, 60 million connections): grow to about 640 shards or a distributed SQL store keyed by namespace, regional notification tiers, and per-namespace journals split by subtree for the largest shared folders. 0.1x (about 280 commits/s): one replicated relational database, a single notification cluster and a managed object store; polling every 30 s is acceptable | Review at each doubling of commits | Metadata shards and connections grow first | [Note] |
| [Decision] | [Options] | [Chosen] | [Why] | [Cost] |

## 11. Delivery plan and risks

<!-- Guidance: How the design reaches production safely, and what could stop it. -->

### 11.1 Milestones

| Milestone | Scope | Exit criterion | Target date |
|---|---|---|---|
| M1 | Metadata service, block service, desktop client with fixed chunking | Simulation tests green for 10 million seeds | YYYY-MM-DD |
| M2 | Content-defined chunking and per-account dedupe | NFR-03 met on benchmark files | YYYY-MM-DD |
| M3 | Notifications and journal cursors | NFR-02 met on canary pairs | YYYY-MM-DD |
| M4 | Shared folders and mobile files-on-demand | Beta users report zero lost edits for 30 days | YYYY-MM-DD |

### 11.2 Rollout and rollback

[*Example:* Release the client to 1% of users, then 10%, 50% and 100%, each
stage held for 7 days and gated on the conflicted-copy rate and rescan rate.
The server keeps accepting the old commit format for two client versions, so
a client rollback needs no server change.]

### 11.3 Risks

| ID | Risk | Likelihood | Impact | Mitigation | Owner |
|---|---|---|---|---|---|
| R-01 | *Example:* A client bug deletes files across devices | Low | High | Mass-delete detection pauses sync and asks the user; 30-day restore | [@name] |
| R-02 | *Example:* Large shared namespaces overload one shard | Medium | Medium | Dedicated shards for top namespaces; journal paging | [@name] |
| R-03 | *Example:* Hashing drains laptop batteries | Medium | Low | Throttle on battery; skip unchanged size and time | [@name] |
| R-04 | *Example:* File systems differ in case sensitivity and path rules | High | Medium | Normalise names, detect case-only conflicts, per-OS tests | [@name] |
| [R-nn] | [Risk] | [L / M / H] | [L / M / H] | [Mitigation] | [@name] |

## 12. Appendices

<!-- Guidance: Supporting detail, glossary and open questions. -->

### 12.1 Open questions

| ID | Question | Owner | Due |
|---|---|---|---|
| OQ-01 | *Example:* Offer end-to-end encrypted folders, accepting no dedupe or previews there? | [@name] | YYYY-MM-DD |
| OQ-02 | *Example:* How long are versions kept on paid plans? | [@name] | YYYY-MM-DD |
| [OQ-nn] | [Question] | [@name] | YYYY-MM-DD |

### 12.2 Glossary

| Term | Meaning |
|---|---|
| Namespace | A root folder (a user's own or a shared folder) that is the unit of journaling and permission |
| Chunk | A piece of file content identified by its SHA-256 hash |
| Content-defined chunking | Choosing chunk boundaries from the content with a rolling hash, so edits move few boundaries |
| Journal | Ordered list of changes per namespace; the cursor is the last applied entry |
| Synced tree | The client's record of the last state where local and remote agreed; the merge base |
| Conflicted copy | A second file holding the losing side of a concurrent edit |
| Parent revision | The revision a client edited, checked at commit for optimistic concurrency |
| [Term] | [Meaning] |

### 12.3 References

| Reference | Use |
|---|---|
| Muthitacharoen A., Chen B., Mazieres D., "A low-bandwidth network file system", SOSP 2001, pp. 174-187, [doi:10.1145/502034.502052](https://doi.org/10.1145/502034.502052) | Content-defined chunking with Rabin fingerprints and minimum and maximum chunk sizes, 7.1 |
| Harnik D., Pinkas B., Shulman-Peleg A., "Side Channels in Cloud Services: Deduplication in Cloud Storage", IEEE Security and Privacy 8(6), pp. 40-47, 2010, [doi:10.1109/MSP.2010.187](https://doi.org/10.1109/MSP.2010.187) | Dedupe side channel, 7.2 |
| Koorapati N., ["Streaming File Synchronization"](https://dropbox.tech/infrastructure/streaming-file-synchronization), Dropbox tech blog, 10 July 2014 | 7.7 |
| Jayakar S., ["Rewriting the heart of our sync engine"](https://dropbox.tech/infrastructure/rewriting-the-heart-of-our-sync-engine), Dropbox tech blog, 9 March 2020 | 7.6, 7.7 |
| Goldberg I., ["Testing sync at Dropbox"](https://dropbox.tech/infrastructure/-testing-our-new-sync-engine), Dropbox tech blog, 20 April 2020 | Three trees in 7.4 |
| Collaborative Editing System Design (`system-design/design-collaborative-editing`) | Real-time co-editing, out of scope here |
| Notification Service System Design (`system-design/design-notification-service`) | Push delivery patterns |
| Distributed Key-Value Store System Design (`system-design/design-distributed-kv-store`) | Chunk index store |
| Photo Sharing System Design (`system-design/design-photo-sharing`) | Blob storage and upload patterns |
| Object Storage System Design (`system-design/design-object-storage`) | Erasure-coded store behind the block service |
| Distributed File System Design (`system-design/design-distributed-file-system`) | Server-side file system alternative to client sync |
| Back-of-Envelope Estimation and Capacity Model (`system-design/back-of-envelope-estimation`) | Method for section 3 |

## Standard and licence note

Based on common industry practice for file synchronisation and on the
published papers and engineering write-ups cited in 12.3. The Dropbox posts
are paraphrased in 7.7 only; no text, figures or code were copied, the
fictional service and every number in this template are illustrative
examples computed for this design, and nothing here describes any company's
current system. Original text written for this collection. Template
licence: MIT.