# System Design: [Commerce platform name] cart, checkout and inventory

> Reference design for the cart, checkout and inventory core of an online
> store or multi-merchant commerce platform. It covers cart storage, price
> and tax quotes with clear boundaries to pricing, tax and promotion
> services, stock reservation across warehouses, oversell prevention, the
> payment handoff, the order state machine, and flash-sale throttling with a
> checkout queue. It follows the structure of the System Design Document
> template and is pre-filled with a worked design to adapt. Replace every
> `[bracketed]` value and every row marked *Example*; values marked
> *Example* are assumptions, not benchmarks. This is an engineering format,
> not tax, consumer-protection or payment-compliance advice.

<!--
Guidance: How to use this template
- Write the invariants in 2.2 first: "units sold never exceed sellable
  stock plus the approved backorder allowance", "a customer is charged the
  total shown on the final review screen", "every paid order is fulfilled
  or refunded". Every reservation and payment decision protects them.
- Decide where a reservation starts: at add-to-cart, at checkout start or
  at payment. The choice drives contention, abandoned-stock waste and the
  customer experience more than any database choice.
- Keep pricing, tax and promotions behind one quote boundary. Checkout
  should never recompute prices on its own.
- Ticket and Hotel Booking System Design (`system-design/design-ticket-booking`)
  covers identified inventory (seats). This template covers counted
  inventory (SKUs in warehouses) and the order lifecycle after payment.
-->

## Document control

<!-- Guidance: A design is a proposal until it is approved; record who decided and when. -->

| Field | Value |
|---|---|
| Document ID | [SDD-CHK-001] |
| System / feature | [Cart, checkout and inventory] |
| Version | [0.1] |
| Status | [Draft / In review / Approved / Implemented / Superseded] |
| Owner | [@owner] |
| Author(s) | [@author] |
| Approvers / reviewers | [@reviewer-1], [@reviewer-2] |
| Last updated | YYYY-MM-DD |
| Classification | [Internal] |
| Selling model | [Single store / marketplace / multi-tenant platform] |
| Related | [PRD], [Payment ledger design], [Fraud design], [Tax engine contract], [Threat model] |

### Approvals

| Role | Name | Decision | Date |
|---|---|---|---|
| Tech lead | [@name] | [Approved / Changes requested] | YYYY-MM-DD |
| Staff / principal engineer | [@name] | [Decision] | YYYY-MM-DD |
| Merchandising and pricing | [@name] | [Decision] | YYYY-MM-DD |
| Fulfilment and warehouse | [@name] | [Decision] | YYYY-MM-DD |
| Payments | [@name] | [Decision] | YYYY-MM-DD |
| SRE / operations | [@name] | [Decision] | YYYY-MM-DD |

## Revision history

<!-- Guidance: Note what changed in the design, not the wording. -->

| Version | Date | Author | Change summary |
|---|---|---|---|
| 0.1 | YYYY-MM-DD | [@name] | First draft for design review |
| 0.2 | YYYY-MM-DD | [@name] | *Example:* Moved reservation from add-to-cart to checkout start |
| 0.3 | YYYY-MM-DD | [@name] | *Example:* Added edge checkout throttle for flash sales |
| 1.0 | YYYY-MM-DD | [@name] | *Example:* Approved |

## Contents

<!-- Guidance: Keep aligned with the headings. -->

1. [Overview](#1-overview)
2. [Requirements](#2-requirements)
3. [Capacity estimation](#3-capacity-estimation)
4. [API design](#4-api-design)
5. [Data model](#5-data-model)
6. [High-level design](#6-high-level-design)
7. [Detailed design](#7-detailed-design)
8. [Scalability and performance](#8-scalability-and-performance)
9. [Reliability, security and operations](#9-reliability-security-and-operations)
10. [Trade-offs and alternatives](#10-trade-offs-and-alternatives)
11. [Delivery plan and risks](#11-delivery-plan-and-risks)
12. [Appendices](#12-appendices)

## 1. Overview

<!-- Guidance: One page that a busy reviewer can read alone: problem, proposed solution, decisions needed, and the interview path. -->

### 1.1 Problem statement

[*Example:* We run 40 000 stores on one platform. During a sneaker drop
last spring one store sold 1 400 pairs it did not have, because stock was
decremented after payment and two checkout pods read the same count. The
same drop sent write traffic to the shared order database that slowed
checkout for every store on that shard for 25 minutes. Promotions are
computed in three places, so the cart, the review page and the receipt
sometimes show different totals, and support refunds the difference by
hand.]

### 1.2 Proposed solution

[*Example:* A quote service is the only place that computes a total: it
calls pricing, promotions and tax and returns a signed, versioned quote.
Checkout reserves stock for the quoted lines with a conditional decrement
and a 10-minute expiry when the customer starts checkout. Payment is
authorised against the quote; the order is created in `placed` and moves
through an explicit state machine to fulfilment. Stock is held per SKU and
location in an inventory service partitioned by SKU. Flash sales are
recognised ahead of time; a throttle at the edge admits shoppers into
checkout at the rate the hot SKU partition and the payment provider can
absorb, and everyone else waits on a cached queue page.]

### 1.3 Decisions requested from reviewers

| ID | Question | Recommended option | Section |
|---|---|---|---|
| Q-01 | *Example:* When to reserve stock? | At checkout start, 10-minute TTL, re-validated at payment | 7.2 |
| Q-02 | *Example:* Oversell prevention mechanism? | Conditional decrement on SKU-location row; sharded counters for drop SKUs | 7.3 |
| Q-03 | *Example:* Payment order? | Authorise, create order, commit reservation, capture at shipment | 7.4 |
| Q-04 | *Example:* Flash-sale control? | Edge throttle with signed pass, enabled per store or SKU | 7.5 |
| Q-05 | *Example:* Price lock duration? | Quote valid 30 minutes; recompute and show changes after expiry | 7.1 |
| [Q-nn] | [Question] | [Option] | [Section] |

### 1.4 45-minute interview path

<!-- Guidance: In an interview, the hard part is oversell prevention under contention and the flash-sale spike, not the cart. Spend the deep-dive time there. -->

| Minute | Cover | Notes |
|---|---|---|
| 0 to 5 | Scope: single store or platform, physical goods, number of warehouses | Name the oversell invariant |
| 5 to 10 | Core requirements; below the line (search, recommendations, returns) | Agree reservation timing as an open question |
| 10 to 15 | Estimates with consequences (3.5) | Show that normal traffic is easy and the drop is not |
| 15 to 20 | API and data model with access patterns | Cart, quote, reservation, order |
| 20 to 30 | High-level design, then deep dive 1: oversell prevention (7.3) | Compare locks, conditional writes, sharded counters, queue per SKU |
| 30 to 40 | Deep dive 2: flash-sale throttling (7.5) or order state machine (7.4) | Pick throttling if the interviewer mentions drops or launches |
| 40 to 45 | Failure modes, 10x and 0.1x | Close on payment handoff and reconciliation |

## 2. Requirements

<!-- Guidance: Split the core from what is explicitly below the line, and attach a number to every quality. -->

### 2.1 Core functional requirements

| ID | Requirement | Priority |
|---|---|---|
| FR-01 | *Example:* Add, update and remove items in a cart for signed-in and guest shoppers | Must |
| FR-02 | *Example:* Show a quote with item prices, discounts, shipping, tax and total | Must |
| FR-03 | *Example:* Reserve stock when checkout starts and release it on expiry or abandonment | Must |
| FR-04 | *Example:* Choose shipping address and method; select fulfilment location | Must |
| FR-05 | *Example:* Pay through the payment service; place the order exactly once | Must |
| FR-06 | *Example:* Track order status through fulfilment, shipment, delivery and cancellation | Must |
| FR-07 | *Example:* Limit quantity per customer for flagged SKUs | Must |
| FR-08 | *Example:* Throttle checkout and queue shoppers during flash sales | Must |
| FR-09 | *Example:* Allow pre-orders and backorders with explicit customer consent | Should |
| [FR-nn] | [Requirement] | [Must / Should / Could] |

### 2.2 Non-functional requirements and invariants

| ID | Quality | Target | Measured as |
|---|---|---|---|
| NFR-01 | No oversell | *Example:* Units committed per SKU-location never exceed on-hand plus approved backorder | Nightly audit; real-time negative stock alert |
| NFR-02 | Price integrity | *Example:* Captured amount equals the accepted quote total | Daily reconciliation of quotes to captures |
| NFR-03 | No lost paid order | *Example:* Every authorised payment maps to one order or one void | Reconciliation with payment service |
| NFR-04 | Checkout latency | *Example:* Place-order p99 at or below 1.2 s excluding 3-D Secure | Service timing |
| NFR-05 | Isolation | *Example:* A flash sale in one store raises p99 of other stores by no more than 10% | Per-tenant latency dashboards |
| NFR-06 | Availability | *Example:* 99.95% for cart and checkout | SLO |
| NFR-07 | Reservation hygiene | *Example:* Expired reservations released within 30 s p99 | Sweeper lag |
| [NFR-nn] | [Quality] | [Target] | [Measurement] |

### 2.3 Below the line (out of scope)

- [*Example:* product search and recommendations (see Recommendation System Design (`system-design/design-recommendation-system`))]
- [*Example:* pricing algorithms and promotion authoring (consumed through the quote boundary)]
- [*Example:* tax rule content (consumed from a tax engine)]
- [*Example:* returns and exchanges workflow]
- [*Example:* the payment ledger (see Payment Ledger System Design (`system-design/design-payment-ledger`))]
- [*Example:* warehouse management and picking]
- [Item deferred, with the release that will take it]

## 3. Capacity estimation

<!-- Guidance: Every number must end in a consequence for the design. -->

### 3.1 Assumptions

| Input | Value | Source |
|---|---|---|
| Orders per day (normal) | *Example:* 3 million | Platform analytics |
| Cart writes per order | *Example:* 8 | Funnel data (adds, updates, removals) |
| Checkout starts per order | *Example:* 2.5 | Conversion from checkout start to order |
| Peak to average ratio (normal day) | *Example:* 4 | Evening peak |
| Drop: shoppers arriving in first minute | *Example:* 600 000 | Store forecast |
| Drop: units available | *Example:* 20 000, limit 1 per customer | Merchant |
| Payment provider authorisations per second available to the platform | *Example:* 2 000 | Provider contract |

### 3.2 Normal traffic

| Quantity | Calculation | Result | Consequence |
|---|---|---|---|
| Average orders/s | 3 million / 86 400 | about 35/s | Order writes are small |
| Peak orders/s | 35 x 4 | about 140/s | One sharded order store with headroom |
| Peak cart writes/s | 140 x 8 | about 1 100/s | Cart in a key-value store, not the order database |
| Peak checkout starts (reservations)/s | 140 x 2.5 | 350/s | Reservations spread across many SKUs; little contention |

### 3.3 Flash-sale traffic

| Quantity | Calculation | Result | Consequence |
|---|---|---|---|
| Arrival rate first minute | 600 000 / 60 | 10 000 shoppers/s | Cannot all enter checkout: throttle at the edge |
| Orders to sell out | 20 000 units / 1 per order | 20 000 orders | Sale is decided in seconds to minutes |
| Admission rate if payment is the ceiling for this store | *Example:* 25% of 2 000 auth/s reserved for one store | 500 orders/s | Sell-out in about 40 s at full admission |
| Writes on the drop SKU row | 500 reservation attempts/s, retries x 1.5 | 750/s on one row | One row with locks is too slow: sharded stock counters (7.3) |

### 3.4 Storage

| Quantity | Calculation | Result | Consequence |
|---|---|---|---|
| Orders per year | 3 million x 365 | about 1.1 billion | Partition orders by store and month |
| Order size with lines | *Example:* 4 KB | about 4.4 TB/year | Archive closed orders after 2 years to cheaper storage |
| SKU-location rows | *Example:* 50 million SKUs x 1.5 locations | 75 million rows | Inventory fits a sharded store partitioned by SKU |

### 3.5 Summary

| Quantity | Estimate | Design consequence |
|---|---|---|
| Normal checkout | 140 orders/s | Simple; correctness, not scale, drives design |
| Drop arrivals | 10 000 shoppers/s | Edge throttle and cached queue page |
| Hot SKU writes | 750/s on one SKU | Sharded counters or single-writer queue per SKU |
| Payment ceiling | 2 000 auth/s platform-wide | Per-store admission share protects other tenants (NFR-05) |

## 4. API design

<!-- Guidance: Define the contract first: idempotency for every state change, quotes as immutable objects, and explicit error types. -->

### 4.1 Endpoints

| Method and path | Purpose | Idempotent |
|---|---|---|
| `PUT /v1/carts/{id}/lines/{sku}` | Set quantity of a SKU | Yes (set semantics) |
| `GET /v1/carts/{id}` | Cart with indicative prices | Yes (read) |
| `POST /v1/checkouts` | Start checkout from a cart: create quote and reservations | Yes, with `Idempotency-Key` |
| `PATCH /v1/checkouts/{id}` | Set address, shipping method; returns a new quote version | Yes (set semantics) |
| `POST /v1/checkouts/{id}/place-order` | Authorise payment and create order for quote version N | Yes, with `Idempotency-Key` |
| `GET /v1/orders/{id}` | Order and status history | Yes (read) |
| `POST /v1/orders/{id}/cancel` | Cancel if state allows | Yes, with `Idempotency-Key` |

### 4.2 Request and response examples

```http
POST /v1/checkouts/chk_31b9/place-order
Idempotency-Key: 0d9e6f12-7a44-4c1b-8e3f-5b2a9c7d1e08
Authorization: Bearer [token]
Content-Type: application/json

{"quote_id": "q_31b9_v3", "payment_method": "pm_tok_8f2", "accepted_total": {"value": 12999, "currency": "USD"}}
```

```http
HTTP/1.1 201 Created
Content-Type: application/json

{"order_id": "ord_7e21", "status": "placed", "total": {"value": 12999, "currency": "USD"}}
```

The client sends the total it displayed. If the server's quote total
differs, place-order fails with 409 `quote-changed` and returns the new
quote, so the customer is never charged an amount they did not see (NFR-02).

### 4.3 Errors

Errors use RFC 9457 problem details.

| Status | Problem type | Meaning |
|---|---|---|
| 409 | `quote-changed` | Price, tax or promotion changed; new quote attached |
| 409 | `out-of-stock` | Reservation failed for listed lines; alternatives attached |
| 410 | `reservation-expired` | Checkout took longer than the TTL |
| 402 | `payment-declined` | Provider declined; reservation kept until TTL for retry |
| 429 | `checkout-queued` | Flash-sale throttle; body carries the queue page URL |
| 403 | `quantity-limit` | Per-customer limit reached |

## 5. Data model

<!-- Guidance: Choose storage by access pattern. Show entities, keys, partitioning, the access-pattern table and the invariant each structure enforces. -->

### 5.1 Entities

The cart is indicative and cheap; the quote, reservation and order are
authoritative. Stock is counted inventory, one row per SKU and location.

```mermaid
erDiagram
  CART ||--o{ CART_LINE : has
  CHECKOUT ||--|{ QUOTE : "versions"
  CHECKOUT ||--o{ RESERVATION : holds
  RESERVATION }o--|| STOCK : "decrements"
  ORDER ||--|{ ORDER_LINE : has
  ORDER ||--|| QUOTE : "priced by"
  ORDER ||--o{ ORDER_EVENT : "state history"
  CART {
    string cart_id PK
    string customer_id
    datetime updated_at
  }
  CART_LINE {
    string cart_id PK
    string sku PK
    int qty
  }
  QUOTE {
    string quote_id PK
    string checkout_id FK
    bigint total
    string currency
    string pricing_version
    string tax_version
    datetime valid_until
  }
  STOCK {
    string sku PK
    string location_id PK
    int on_hand
    int reserved
    int backorder_allowed
    int version
  }
  RESERVATION {
    string reservation_id PK
    string checkout_id FK
    string sku
    string location_id
    int qty
    string status
    datetime expires_at
  }
  ORDER {
    string order_id PK
    string store_id
    string status
    string payment_ref
  }
  ORDER_LINE {
    string order_id PK
    int line_no PK
    string sku
    int qty
    bigint unit_price
  }
  ORDER_EVENT {
    string order_id PK
    int seq PK
    string from_status
    string to_status
  }
```

### 5.2 Storage choices and invariants

| Data | Store | Partition key | Invariant enforced by |
|---|---|---|---|
| Carts | *Example:* Key-value store with TTL 30 days | `cart_id` | None (indicative only) |
| Quotes | Document store, immutable versions | `checkout_id` | Signed hash of lines, prices and versions |
| Stock | Sharded relational or key-value store with conditional writes | `sku` | `reserved + qty <= on_hand + backorder_allowed` in the update predicate |
| Reservations | Same shard as the stock row | `sku` | Status machine; expiry swept |
| Orders, lines, events | Sharded relational | `store_id` then `order_id` | Allowed transitions table (7.4); outbox in same transaction |
| Idempotency records | Same shard as orders | `store_id`, key | Unique constraint |

### 5.3 Access patterns

| ID | Query | Served by | Partition key rationale |
|---|---|---|---|
| AP-01 | *Example:* Read cart | `cart_id` lookup | Single key; cart never joins |
| AP-02 | *Example:* Reserve qty of SKU at best location | Conditional update on `stock(sku, location_id)` | All locations of one SKU on one shard so location fallback is local |
| AP-03 | *Example:* Release expired reservations | Index on `(expires_at)` per shard | Sweeper scans its own shard |
| AP-04 | *Example:* Place order and record payment reference | Insert into `order` on store shard | Store-level reports and isolation (NFR-05) |
| AP-05 | *Example:* Customer's order history | Secondary index `customer_id` in a read model | Avoids cross-shard scans on the primary |
| AP-06 | *Example:* Stock level for product page | Cached availability projection | Product page never touches the stock row |

## 6. High-level design

<!-- Guidance: One diagram that shows every component on the request paths and where state lives. -->

```mermaid
flowchart LR
  shopper([Shopper]) --> edge[CDN and edge throttle]
  edge --> queue[Cached queue page]
  edge --> cartapi[Cart API]
  edge --> chk[Checkout API]
  cartapi --> carts[(Cart store)]
  chk --> quote[Quote service]
  quote --> price[Pricing]
  quote --> promo[Promotions]
  quote --> tax[Tax engine]
  chk --> inv[Inventory service]
  inv --> stock[(Stock shards by SKU)]
  chk --> pay[Payment service]
  pay --> psp[[Payment provider]]
  chk --> ord[Order service]
  ord --> orders[(Order shards by store)]
  orders --> outbox[[Outbox relay]]
  outbox --> bus[[Event log]]
  bus --> ful[Fulfilment]
  bus --> avail[Availability projection]
  bus --> notify[Notifications]
```

| Component | Responsibility | Scales by |
|---|---|---|
| Edge throttle | Admit shoppers into checkout per store or SKU; issue signed pass | Edge workers |
| Quote service | Single source of totals; signs quotes | Horizontally; caches pricing |
| Inventory service | Reserve, commit, release stock | SKU shards |
| Order service | Create order, run state machine, publish events | Store shards |
| Payment service | Authorise, capture, void, refund | Horizontally; provider limits |
| Availability projection | Product-page stock indicators | Cache |

## 7. Detailed design

<!-- Guidance: Two to four deep dives on the hard problems, each with options, choice and reason. -->

### 7.1 Quote boundary: pricing, tax and promotions

- The quote service owns the arithmetic: list price from pricing, discounts
  from promotions in a fixed evaluation order, shipping, then tax on the
  discounted amounts according to the tax engine's rules.
- A quote is immutable. Any change (address, shipping, quantity) creates a
  new version. The quote records the versions of the price list, promotion
  set and tax rules it used.
- Promotions with limited redemptions (first 1 000 uses) are reserved like
  stock at checkout start and committed at order placement.
- Rounding happens once, in the quote, per the tax engine's rule (per line
  or per invoice). No other service rounds.

**Options:** compute totals in checkout; compute in each service and sum;
central quote. **Choice:** central quote. **Reason:** one place to test and
audit totals, and NFR-02 becomes a comparison of two numbers.

### 7.2 Deep dive: when to reserve stock

| Option | Benefit | Cost |
|---|---|---|
| At add-to-cart | Shopper sees guaranteed stock early | Abandoned carts lock stock for hours; easy denial of inventory by bots |
| At checkout start (TTL) | Short holds tied to intent; fair at drop time | Shopper can lose items between cart and checkout |
| At payment authorisation | No waste | Many shoppers pay for items that are gone; refunds and anger |
| At order placement without hold | Simplest | Oversell under concurrency unless the decrement is conditional |

**Choice:** reserve at checkout start with a *Example* 10-minute TTL,
extendable once during 3-D Secure, and re-validate the reservation at
place-order. **Reason:** it binds stock to real intent, bounds waste and
lets the cart stay a cheap, non-authoritative store.

### 7.3 Deep dive: oversell prevention under contention

| Option | How it works | Fits |
|---|---|---|
| Row lock (`SELECT ... FOR UPDATE`) | Serialise reservations on the SKU row | Low traffic only; lock queue under drops |
| Conditional decrement | `UPDATE stock SET reserved = reserved + :q WHERE sku = :s AND location_id = :l AND reserved + :q <= on_hand + backorder_allowed` | Normal traffic: one statement, no read-then-write gap |
| Sharded counters | Split a drop SKU's stock into K buckets; reserve from a random bucket, try the next on failure | Drop SKUs: parallel writes, small leftover fragmentation |
| Single-writer queue per SKU | All reservations for a SKU go through one ordered consumer | Extreme contention; adds queue latency |
| In-memory counter with durable log | Decrement in an in-memory store, append to log, reconcile | Very high rates; needs careful recovery |

**Choice:** conditional decrement by default; sharded counters (*Example*
K = 8) for SKUs flagged as drops. **Reason:** 750 writes/s on one row
(3.3) exceeds a safe row update rate, while 8 buckets bring each to about
95 writes/s. When most buckets are empty, a final pass merges remaining
units into one bucket so the last items still sell.

```sql
UPDATE stock_bucket
SET reserved = reserved + :qty, version = version + 1
WHERE sku = :sku AND bucket = :b
  AND reserved + :qty <= on_hand;
-- 0 rows: try next bucket; after K misses return 409 out-of-stock.
```

### 7.4 Deep dive: payment handoff and order state machine

```mermaid
stateDiagram-v2
  [*] --> placed: authorised and reservation committed
  placed --> fulfilling: allocated to location
  fulfilling --> shipped: carrier handoff, capture payment
  shipped --> delivered: carrier event
  placed --> cancelled: customer or fraud review, void authorisation
  fulfilling --> cancelled: stock missing at pick, void or refund
  delivered --> returned: return accepted, refund
  cancelled --> [*]
  returned --> [*]
  delivered --> [*]
```

Place-order sequence:

```mermaid
sequenceDiagram
  participant C as Checkout API
  participant Q as Quote
  participant I as Inventory
  participant P as Payment
  participant O as Order
  C->>Q: verify quote version and signature
  C->>I: verify reservations active
  C->>P: authorise (key checkout:authorise)
  alt authorised
    P-->>C: auth ref
    C->>O: create order placed (key checkout:order)
    O->>I: commit reservations (reserved to sold)
    O-->>C: order ID
  else declined or timeout
    P-->>C: declined or unknown
    C->>P: if unknown, query by key before deciding
  end
```

- **Options for capture:** at order placement; at shipment; split capture
  per shipment. **Choice:** capture at shipment for physical goods, at
  placement for digital goods. **Reason:** the shopper is never charged for
  items that cannot ship, and cancellations before shipment are voids, not
  refunds. Check the authorisation validity period of your card networks and
  re-authorise if fulfilment is slow.
- An order that cannot commit its reservations after authorisation
  (expired meanwhile) attempts to re-reserve; on failure it voids the
  authorisation and returns 410.
- Every transition writes an `order_event` and an outbox row in one
  transaction; illegal transitions are rejected by a transitions table.

### 7.5 Deep dive: flash-sale throttling

| Option | Benefit | Cost |
|---|---|---|
| Autoscale only | No user-facing queue | Databases and payment provider do not autoscale in seconds |
| Rate limit with 429 | Simple | Shoppers hammer refresh; unfair |
| Edge throttle with cached queue page and signed pass | Origin sees only admitted shoppers; isolates other stores | Queue fairness must be designed |
| Full virtual waiting room with randomised positions | Fair at opening | More infrastructure (see Ticket and Hotel Booking System Design (`system-design/design-ticket-booking`)) |

**Choice:** edge throttle with a cached queue page and a signed pass, plus
randomised positions for drops announced in advance. **Reason:** it caps
checkout writes per store at the admission rate in 3.3, keeping other
tenants within NFR-05, and the queue page costs the origin nothing.

> **How real systems do it** *(paraphrased from a public write-up; not copied)*
>
> Shopify's engineering blog describes how flash sales on a shared
> multi-tenant platform produced bursts of checkout writes large enough to
> hurt other stores on the same database shard. Rather than keep enough
> capacity running for the worst sale, the team put scripted logic into its
> Nginx load balancers (OpenResty with Lua): a leaky-bucket style throttle
> admitted a fixed number of checkout requests per short interval, sent the
> rest to a queue page cached at the load balancer that polled for entry,
> and gave admitted shoppers a signed cookie so they were not throttled again
> during their session. The post is candid that random polling made waits
> unfair for some shoppers, which led to a later redesign. This template
> keeps the edge throttle and signed pass and adds ordered or randomised
> positions to address fairness.
>
> Source: Emil Stolarsky, "Surviving Flashes of High-Write Traffic Using
> Scriptable Load Balancers (Part I)", Shopify Engineering, 3 February 2017,
> https://shopify.engineering/surviving-flashes-of-high-write-traffic-using-scriptable-load-balancers-part-i

### 7.6 Multi-location allocation

The inventory service picks a location at reservation time using a simple
rule (*Example:* nearest location with full quantity, else split shipment
if the store allows it). Fulfilment may reallocate later; reallocation moves
the reservation between rows of the same SKU on the same shard.

## 8. Scalability and performance

<!-- Guidance: Show how the design meets the numbers in section 3 and where it will break first. -->

| Concern | Approach | Limit before redesign |
|---|---|---|
| Hot SKU | Sharded counters; single-writer queue as fallback | *Example:* about 300 writes/s per bucket in load test |
| Tenant isolation | Per-store admission share at the edge; order shards by store | Largest store's share of the payment ceiling |
| Cart writes | Key-value store, no transactions | Store throughput |
| Product page stock | Availability projection, 2-second cache | Projection lag |
| Quote computation | Cache price lists and promotion sets by version | Tax engine latency |

Latency budget for place-order (p99 target 1.2 s):

| Step | Budget |
|---|---|
| Quote and reservation verification (60 ms) and fraud check (100 ms) in parallel | 100 ms |
| Payment authorisation | 600 ms |
| Order creation and reservation commit | 80 ms |
| Network and headroom | 420 ms |

## 9. Reliability, security and operations

<!-- Guidance: Name the failure modes, the threats and the signals operators watch. -->

### 9.1 Failure modes

| ID | Component | Failure | Detection | Blast radius | Mitigation | RTO |
|---|---|---|---|---|---|---|
| FM-01 | Payment provider | Timeout | Step timeout | Orders in flight | Query by key; void if order not created | *Example:* 10 min |
| FM-02 | Inventory shard | Failover | Error rate per shard | SKUs on one shard | Edge pauses admission for affected drops; retries with keys | *Example:* 2 min |
| FM-03 | Reservation sweeper | Stalled | Expired-but-reserved count | Stock appears sold out | Lazy expiry in predicate; restart | *Example:* 5 min |
| FM-04 | Tax engine | Unavailable | Error rate | All new quotes | Cached rules for known addresses; block checkout otherwise | *Example:* 15 min |
| FM-05 | Order outbox | Stops | Outbox age | Fulfilment and emails delayed | Restart; consumers dedupe | *Example:* 10 min |
| FM-06 | Edge throttle | Misconfigured rate | Admission vs order rate | One store or platform | Per-store caps with safe defaults; config review | *Example:* 5 min |
| [FM-nn] | [Component] | [Failure] | [Signal] | [Scope] | [Mitigation] | [RTO] |

Score these in a Software FMEA (`ops/fmea-software`).

### 9.2 Security and abuse

| ID | Threat | Control |
|---|---|---|
| SEC-01 | Bots buying drop stock | Edge bot controls, per-customer, per-card and per-address limits, randomised queue |
| SEC-02 | Inventory denial through reservations | Reservations per session and account limited; short TTL |
| SEC-03 | Price tampering in client | Server-side quote with signature; client total only compared |
| SEC-04 | Promotion abuse | Redemption reservation; per-customer limits; fraud scoring |
| SEC-05 | Card testing at checkout | Rate limits, fraud scoring (see Payment Fraud Detection System Design (`system-design/design-fraud-detection`)) |
| [SEC-nn] | [Threat] | [Control] |

### 9.3 Observability

| Signal | Metric | Alert threshold |
|---|---|---|
| Oversell | Negative available stock per SKU-location | Any |
| Quote mismatch | `quote-changed` rate at place-order | Above 2% |
| Reservation waste | Share of reservations expiring | Above 70% outside drops |
| Paid order integrity | Authorisations without orders after 10 min | Any |
| Tenant isolation | p99 checkout for non-drop stores during a drop | Over NFR-05 |

## 10. Trade-offs and alternatives

<!-- Guidance: List the options for each major decision, the cost of the choice, and how the design changes at 10x and 0.1x. -->

| Decision | Options | Chosen | Why | What we give up |
|---|---|---|---|---|
| Reservation timing | Cart; checkout; payment | Checkout start with TTL | Intent-bound, bounded waste | Items can vanish before checkout |
| Oversell control | Locks; conditional; buckets; queue | Conditional plus buckets for drops | Parallel and exact | Bucket fragmentation logic |
| Capture timing | Placement; shipment | Shipment (physical) | No charge without goods | Re-authorisation for slow fulfilment |
| Totals | Distributed; central quote | Central quote | One auditable number | Quote service is critical path |
| Drop control | Autoscale; 429; edge throttle | Edge throttle with pass | Protects tenants | Shoppers wait |
| **At 10x (1 400 orders/s, 100 000 shoppers/s at drops)** | More shards; queue per SKU | Single-writer queue per drop SKU, multiple payment providers | Bucket retries and payment ceiling dominate | More moving parts, provider routing |
| **At 0.1x (14 orders/s)** | One database | One relational database, conditional decrement only | Contention is rare | Must re-shard if a drop business grows |
| [Decision] | [Options] | [Chosen] | [Why] | [Cost] |

## 11. Delivery plan and risks

<!-- Guidance: How the design reaches production safely, and what could stop it. -->

### 11.1 Milestones

| Milestone | Scope | Exit criterion | Target date |
|---|---|---|---|
| M1 | Quote service as single source of totals | Zero quote-to-capture mismatches for 30 days | YYYY-MM-DD |
| M2 | Reservations at checkout with conditional decrement | No oversell in a 5 000 concurrent-shopper test | YYYY-MM-DD |
| M3 | Order state machine with outbox | All transitions covered by tests; reconciliation clean | YYYY-MM-DD |
| M4 | Edge throttle and drop SKUs with buckets | Rehearsal drop at 2x forecast with NFR-05 met | YYYY-MM-DD |

### 11.2 Rollout and rollback

[*Example:* Enable reservations per store cohort behind a flag. Rollback
returns the cohort to decrement-at-order with conditional writes, which is
still oversell-safe. Drops are only run on stores already migrated.]

### 11.3 Risks

| ID | Risk | Likelihood | Impact | Mitigation | Owner |
|---|---|---|---|---|---|
| R-01 | *Example:* Warehouse on-hand counts are wrong, so reservations promise phantom stock | Medium | High | Cycle counts, safety stock per SKU, Data Contract (`ai-data/data-contract`) with WMS feed | [@name] |
| R-02 | *Example:* Tax engine rounding differs from quote assumption | Low | Medium | Contract tests with engine per jurisdiction | [@name] |
| R-03 | *Example:* Long fulfilment exceeds authorisation validity | Medium | Medium | Re-authorise job; capture at placement for preorders with consent | [@name] |
| [R-nn] | [Risk] | [L / M / H] | [L / M / H] | [Mitigation] | [@name] |

## 12. Appendices

<!-- Guidance: Supporting detail, traceability, glossary and references. -->

### 12.1 Traceability

| Requirement | Design element | Verification |
|---|---|---|
| NFR-01 | 7.3 conditional decrement and buckets | Concurrency test; nightly audit |
| NFR-02 | 7.1 quote; 4.2 accepted total | Quote-capture reconciliation |
| NFR-03 | 7.4 place-order sequence | Fault injection on payment timeout |
| NFR-05 | 7.5 per-store admission | Drop rehearsal with tenant latency |
| [ID] | [Element] | [Test] |

### 12.2 Open questions

| ID | Question | Owner | Due |
|---|---|---|---|
| OQ-01 | *Example:* Should guest carts reserve during drops at all? | [@name] | YYYY-MM-DD |
| OQ-02 | *Example:* Do we show the queue position or only an estimate? | [@name] | YYYY-MM-DD |
| [OQ-nn] | [Question] | [@name] | YYYY-MM-DD |

### 12.3 Glossary

| Term | Meaning |
|---|---|
| Quote | Immutable, signed price calculation for a checkout version |
| Reservation | Temporary claim on stock with an expiry |
| Oversell | Committing more units than sellable stock |
| Drop | Pre-announced sale of scarce stock with a traffic spike |
| Sharded counter | Stock for one SKU split into buckets to allow parallel writes |
| Authorisation | Hold on customer funds before capture |

### 12.4 References

| Reference | Use |
|---|---|
| Emil Stolarsky, "Surviving Flashes of High-Write Traffic Using Scriptable Load Balancers (Part I)", Shopify Engineering, 2017 | Edge throttle pattern (7.5); paraphrased |
| RFC 9457 Problem Details for HTTP APIs | Error format (4.3) |
| Ticket and Hotel Booking System Design (`system-design/design-ticket-booking`) | Holds, waiting room, identified inventory |
| Payment Ledger System Design (`system-design/design-payment-ledger`) | Authorisation, capture and ledger |
| Distributed Message Queue System Design (`system-design/design-message-queue`) | Outbox and event delivery |
| Rate Limiter System Design (`system-design/design-rate-limiter`) | Throttle algorithms |
| Resilience and Load Shedding Design (`system-design/resilience-load-shedding-design`) | Shedding under overload |

## Standard and licence note

Based on common industry practice for commerce checkout and inventory
systems and on the public references in 12.4. Company blogs were used as
inspiration only; the text, diagrams and numbers are original and
illustrative. Tax, consumer-protection and card-network rules vary by
jurisdiction and change; check the current official rules. This is an
engineering format, not legal advice. Template licence: MIT.