# System Design: [Booking system name] > Reference design for a hotel room and event ticket booking platform. It > covers the inventory model for rooms (date-based allotments) and seats > (individually identified), holds with a time to live, overbooking policy, > concurrency control with optimistic and pessimistic locking and conditional > writes, idempotency keys, payment integration through a saga with > compensation, flash-sale traffic with a virtual waiting room, the split > between the search path and the booking path, consistency boundaries, > failure handling and fraud controls. It follows the structure of the System > Design Document template and is pre-filled with a worked design to adapt. > Replace every `[bracketed]` value and every row marked *Example*; values > marked *Example* are assumptions. This is an engineering format, not legal, > consumer-protection or payment-compliance advice. ## Document control | Field | Value | |---|---| | Document ID | [SDD-BOOK-001] | | System / feature | [Hotel and event booking] | | Version | [0.1] | | Status | [Draft / In review / Approved / Implemented / Superseded] | | Author(s) | [@author] | | Reviewers | [@reviewer-1], [@reviewer-2] | | Last updated | YYYY-MM-DD | | Classification | [Internal] | | Related | [PRD], [Payment and ledger design], [Fraud policy], [Threat model] | ### Approvals | Role | Name | Decision | Date | |---|---|---|---| | Tech lead | [@name] | [Approved / Changes requested] | YYYY-MM-DD | | Staff / principal engineer | [@name] | [Decision] | YYYY-MM-DD | | Revenue management (overbooking) | [@name] | [Decision] | YYYY-MM-DD | | Payments and fraud | [@name] | [Decision] | YYYY-MM-DD | | SRE / operations | [@name] | [Decision] | YYYY-MM-DD | | Security | [@name] | [Decision] | YYYY-MM-DD | ## Revision history | Version | Date | Author | Change summary | |---|---|---|---| | 0.1 | YYYY-MM-DD | [@name] | First draft for design review | | 0.2 | YYYY-MM-DD | [@name] | *Example:* Replaced row locks with conditional writes for seat holds | | 0.3 | YYYY-MM-DD | [@name] | *Example:* Added virtual waiting room for onsales above 10 000 seats | | 1.0 | YYYY-MM-DD | [@name] | *Example:* Approved | ## Contents 1. [Overview](#1-overview) 2. [Requirements](#2-requirements) 3. [Capacity estimation](#3-capacity-estimation) 4. [API design](#4-api-design) 5. [Data model](#5-data-model) 6. [High-level design](#6-high-level-design) 7. [Detailed design](#7-detailed-design) 8. [Scalability and performance](#8-scalability-and-performance) 9. [Reliability, security and operations](#9-reliability-security-and-operations) 10. [Trade-offs and alternatives](#10-trade-offs-and-alternatives) 11. [Delivery plan and risks](#11-delivery-plan-and-risks) 12. [Appendices](#12-appendices) ## 1. Overview ### 1.1 Problem statement [*Example:* We sell hotel rooms from 4 000 partner properties and tickets for concerts and sports events. Last season a stadium onsale sold 312 seats twice because two application servers read the same "available" row before either wrote. The search page timed out for 18 minutes during the same onsale because availability queries ran against the booking database. Customers whose payment failed kept their seats locked for 30 minutes, and resale bots bought 40% of the best seats in the first two minutes.] ### 1.2 Proposed solution [*Example:* Inventory lives in an inventory service that owns a strongly consistent store partitioned by event or property. A booking is a saga: hold inventory with a conditional write and a 10-minute expiry, authorise payment, confirm the hold, capture payment, issue tickets or a hotel confirmation. Every step is idempotent and has a compensating action. Search reads a denormalised availability index fed by inventory change events and is allowed to be a few seconds stale; the booking path always re-checks the source of truth. Large onsales go through a virtual waiting room that admits users at the rate checkout can sustain. Bot and fraud controls run at the waiting room and again before payment.] ### 1.3 Decisions requested from reviewers | ID | Question | Recommended option | Section | |---|---|---|---| | Q-01 | *Example:* Lock strategy for seats? | Conditional write (compare-and-set on seat status and version), no long-held row locks | 7.3, 10 | | Q-02 | *Example:* Hold duration? | 10 minutes for tickets, 15 minutes for hotels, extendable once by 5 minutes during 3-D Secure | 7.2 | | Q-03 | *Example:* Overbooking? | Hotels only, per property and night, capped by revenue management; never for seated events | 7.5 | | Q-04 | *Example:* Payment order? | Authorise after hold, capture after confirm; void on any failure | 7.6 | | Q-05 | *Example:* Waiting room threshold? | Mandatory for onsales forecast above 5 times normal checkout capacity | 7.7 | | [Q-nn] | [Question] | [Option] | [Section] | ## 2. Requirements ### 2.1 Functional requirements | ID | Requirement | Priority | |---|---|---| | FR-01 | *Example:* Search hotels by destination, dates and guests; search events by artist, venue and date | Must | | FR-02 | *Example:* Show a seat map with live seat status for reserved-seating events | Must | | FR-03 | *Example:* Hold selected seats or rooms for a limited time while the customer pays | Must | | FR-04 | *Example:* Book, pay and receive a confirmation and tickets or a hotel voucher | Must | | FR-05 | *Example:* Cancel within the rate's policy and refund automatically | Must | | FR-06 | *Example:* Limit tickets per customer per event (for example 6) | Must | | FR-07 | *Example:* Virtual waiting room with fair, randomised admission at onsale start | Must | | FR-08 | *Example:* Hotel overbooking within approved limits, with walk (relocation) workflow | Should | | FR-09 | *Example:* General-admission (unnumbered) inventory with counted capacity | Should | | FR-10 | *Example:* Waitlist for sold-out events, offered when holds expire or tickets are returned | Could | | [FR-nn] | [Requirement] | [Must / Should / Could] | ### 2.2 Non-functional requirements and invariants | ID | Quality | Target | Measured as | |---|---|---|---| | NFR-01 | No double sale (identified inventory) | *Example:* Zero seats sold twice | Nightly audit of tickets per seat | | NFR-02 | Allotment respected (counted inventory) | *Example:* Sold room nights never exceed allotment plus approved overbooking | Nightly audit per property and night | | NFR-03 | No silent loss | *Example:* Every captured payment maps to exactly one confirmed booking or one refund | Daily reconciliation | | NFR-04 | Search latency | *Example:* p95 at or below 300 ms at 20 000 searches/s | Synthetic and real-user monitoring | | NFR-05 | Hold latency | *Example:* p99 at or below 250 ms for a hold of up to 8 seats | Service timing | | NFR-06 | Availability (booking path) | *Example:* 99.95% monthly; onsale windows treated as critical | SLO | | NFR-07 | Search freshness | *Example:* Availability index lag p99 at or below 5 s | Event age at index write | | NFR-08 | Fairness | *Example:* Waiting room position independent of arrival time before onsale opens | Audit of queue assignment | | [NFR-nn] | [Quality] | [Target] | [Measurement] | ### 2.3 Out of scope - [*Example:* dynamic pricing algorithms (consumed as a price service)] - [*Example:* secondary-market resale and ticket transfer (separate design)] - [*Example:* the double-entry ledger (see Payment Ledger System Design (`system-design/design-payment-ledger`))] - [*Example:* channel manager integrations with property management systems beyond the allotment feed] - [Item deferred, with the release that will take it] ## 3. Capacity estimation ### 3.1 Assumptions | Input | Symbol | Value | Source | |---|---|---|---| | Seats in the largest onsale | $S$ | *Example:* 50 000 | Venue capacity | | Average tickets per order | $t$ | *Example:* 2.5 | Past onsales | | Users arriving in the first 10 minutes | $U$ | *Example:* 2 000 000 | Marketing forecast | | Sustainable checkouts per second | $C$ | *Example:* 300 orders/s | Payment provider contract and load test | | Share of admitted users who complete an order | $k$ | *Example:* 0.6 | Past onsales | | Peak hotel and event searches per second | $Q$ | *Example:* 20 000 | Traffic forecast | | Hold TTL (tickets) | $T_h$ | *Example:* 600 s | 7.2 | ### 3.2 Onsale throughput and admission rate Orders needed to sell out: $$ O = \frac{S}{t} = \frac{50\,000}{2.5} = 20\,000 \ \text{orders} $$ At the sustainable checkout rate the sale takes about $$ T_{sell} = \frac{O}{C} = \frac{20\,000}{300} \approx 67 \ \text{s} $$ The waiting room admits users at the rate checkout can absorb, allowing for users who browse and leave: $$ A = \frac{C}{k} = \frac{300}{0.6} = 500 \ \text{users/s} $$ Most of the 2 million users will not get tickets. Admitting at 500 users/s, only about $500 \times 67 \approx 33\,500$ users are admitted before the event sells out; the rest must be told quickly and honestly (7.7), not left in a queue that cannot succeed. ### 3.3 Holds and contention Active holds are bounded by inventory, not by users: at most $S = 50\,000$ seat holds exist at once. Each hold attempt touches 1 to 8 seat rows in the partition of one event, so the hottest partition sees roughly $$ W = A \times k_{try} \times \bar{s} = 500 \times 1.5 \times 2.5 \approx 1\,875 \ \text{seat writes/s} $$ with $k_{try} = 1.5$ hold attempts per admitted user (*Example:* some retry after losing a seat) and $\bar{s} = 2.5$ seats per attempt. That rate fits one well-provisioned partition but not one row lock queue, which is why 7.3 uses short conditional writes rather than long transactions. ### 3.4 Search Search at $Q = 20\,000$ queries/s is served from the availability index and cache. With a cache hit ratio of 0.9, the index sees about 2 000 queries/s. The booking store sees no search traffic at all. ### 3.5 Hotel inventory size | Item | Calculation | Result | |---|---|---| | Inventory rows (counted) | 4 000 properties x 8 room types x 365 nights | 11.7 million rows | | Row size | *Example:* 200 bytes | about 2.3 GB, before indexes and replicas | | Booking records per year | *Example:* 6 million bookings x 2 KB | about 12 GB per year | ### 3.6 Summary | Quantity | Estimate | Design consequence | |---|---|---| | Onsale checkouts | 300 orders/s | Payment capacity is the real ceiling; waiting room meters to it | | Admission rate | 500 users/s | Waiting room with token issuance | | Hot partition writes | about 1 900 seat writes/s | Conditional writes, no row locks held across network calls | | Search | 20 000 queries/s | Separate read path, cache plus index | | Hotel inventory | about 12 million rows | Fits one sharded relational cluster | ## 4. API design ### 4.1 Endpoints | Method and path | Purpose | Idempotent | |---|---|---| | `GET /v1/events/{id}/availability` | Seat map or section counts (from index) | Yes (read) | | `GET /v1/hotels/search?dest=&checkin=&checkout=&guests=` | Hotel availability and prices (from index) | Yes (read) | | `POST /v1/holds` | Hold seats or room nights; returns hold ID and expiry | Yes, with `Idempotency-Key` | | `DELETE /v1/holds/{id}` | Release a hold early | Yes | | `POST /v1/bookings` | Convert a hold into a booking and start payment | Yes, with `Idempotency-Key` | | `GET /v1/bookings/{id}` | Booking status (pending, confirmed, failed, cancelled) | Yes (read) | | `POST /v1/bookings/{id}/cancel` | Cancel under the rate or event policy | Yes, with `Idempotency-Key` | | `POST /v1/queue/{onsale}/join` | Join the waiting room; returns a queue token | Yes, per device and account | ### 4.2 Request and response examples A hold request names the inventory and carries the admission token during metered onsales; the response returns the server-set expiry and the price that the booking will be charged. ```http POST /v1/holds Idempotency-Key: 8f1c2a7e-4d55-4b0e-9c61-2b1f0e9d7a10 Authorization: Bearer [token] Content-Type: application/json {"event_id": "evt_2291", "seats": ["A-12-07", "A-12-08"], "queue_token": "qt_9c1e4b"} ``` ```http HTTP/1.1 201 Created Content-Type: application/json {"hold_id": "hld_77a1", "status": "held", "seats": ["A-12-07", "A-12-08"], "expires_at": "YYYY-MM-DDThh:mm:ssZ", "price": {"amount": 18000, "currency": "EUR"}} ``` ### 4.3 Idempotency and errors - The server stores the key, a hash of the request body and the response for 24 hours. The same key with the same body returns the stored response; the same key with a different body returns 422. - A hold that loses a race returns 409 with the seats that are no longer available, so the client can offer alternatives without reloading the map. - An expired hold returns 410 on `POST /v1/bookings`. - Errors use RFC 9457 problem details with a stable `type` URI. | Status | Problem type | Meaning | |---|---|---| | 409 | `seat-unavailable` | One or more requested seats were taken; body lists them | | 410 | `hold-expired` | Hold TTL passed before booking | | 422 | `idempotency-key-reused` | Same key, different request | | 429 | `queue-required` | Onsale is metered; join the waiting room | | 403 | `purchase-limit` | Per-customer ticket limit reached | ## 5. Data model ### 5.1 Entities Seats are identified inventory (one row per seat per event). Hotel rooms are counted inventory (one row per property, room type and night). ```mermaid erDiagram EVENT ||--o{ SEAT : has PROPERTY ||--o{ ROOM_NIGHT : has HOLD ||--o{ HOLD_ITEM : contains HOLD_ITEM }o--|| SEAT : "reserves (identified)" HOLD_ITEM }o--|| ROOM_NIGHT : "reserves (counted)" BOOKING ||--|| HOLD : "created from" BOOKING ||--o{ PAYMENT_ATTEMPT : has EVENT { string event_id PK datetime onsale_at int per_customer_limit } SEAT { string event_id PK string seat_id PK string status string hold_id datetime held_until int version } PROPERTY { string property_id PK string timezone } ROOM_NIGHT { string property_id PK string room_type PK date night PK int allotment int overbook_limit int held int sold int version } HOLD { string hold_id PK string customer_id string status datetime expires_at } HOLD_ITEM { string hold_id PK string item_ref PK int quantity } BOOKING { string booking_id PK string hold_id FK string status string idempotency_key } PAYMENT_ATTEMPT { string attempt_id PK string booking_id FK string psp_reference string status } ``` ### 5.2 Storage choices and invariants | Data | Store | Partition key | Invariant enforced by | |---|---|---|---| | Seats | *Example:* Distributed SQL or a key-value store with conditional writes | `event_id` | `UPDATE ... WHERE status = 'available' AND version = ?` | | Room nights | Relational, sharded by property | `property_id` | `held + sold + qty <= allotment + overbook_limit` in the update predicate | | Holds, bookings | Relational, same shard as the inventory they reference where possible | `event_id` or `property_id` | Foreign keys and status state machine | | Idempotency records | Same database as bookings | key | Unique constraint on key | | Availability index | Search engine plus cache | destination, event | Eventually consistent; never used to decide a sale | | Outbox | Same database as bookings | none | Written in the booking transaction | ## 6. High-level design Two paths share the inventory events but not the store: search is fast and stale by seconds, booking is exact. ```mermaid flowchart LR user([Customer]) --> edge[CDN and bot protection] edge --> wr[Virtual waiting room] edge --> search[Search API] search --> cache[(Availability cache)] cache --> idx[(Availability index)] wr -->|admission token| book[Booking API] book --> inv[Inventory service] inv --> invdb[(Inventory store, partitioned)] book --> orch[Booking saga orchestrator] orch --> inv orch --> pay[Payment service] pay --> psp[[Payment provider]] orch --> fraud[Fraud scoring] orch --> fulfil[Ticketing and confirmation] invdb --> outbox[[Outbox relay]] outbox --> bus[[Event bus]] bus --> idx bus --> notify[Notifications] sweeper[Hold expiry sweeper] --> invdb ``` | Component | Responsibility | Scales by | |---|---|---| | Waiting room | Queue, randomise, admit at a metered rate, issue signed admission tokens | Stateless edge workers plus a counter store | | Search API | Query index and cache; never writes | Horizontally | | Booking API | Validate tokens and limits, call the saga | Horizontally | | Inventory service | Holds, releases, confirms with conditional writes | Partitions (event or property) | | Saga orchestrator | Run booking steps and compensations durably | Workflow workers | | Payment service | Authorise, capture, void, refund through the PSP | Horizontally; PSP limits apply | | Hold expiry sweeper | Release expired holds and publish events | One worker per partition range | ## 7. Detailed design ### 7.1 Inventory model - **Identified inventory (seats)**: each seat has a status (`available`, `held`, `sold`, `blocked`), the hold that owns it and a version. A seat moves `available -> held -> sold`, or `held -> available` on expiry. - **Counted inventory (room nights, general admission)**: one row holds `allotment`, `held`, `sold` and `overbook_limit`. A hold increments `held` only when the predicate still holds after the increment. - **Multi-night stays** must hold every night or none. All nights of one room type at one property live in one partition, so one transaction covers them. ```mermaid stateDiagram-v2 [*] --> available available --> held: hold (conditional write) held --> available: expiry or release held --> sold: confirm after payment authorised sold --> available: cancellation and refund available --> blocked: venue or ops block blocked --> available: unblock ``` ### 7.2 Holds and TTL - A hold has `expires_at` set by the server. Clients see a countdown but the server clock decides. - Expiry is enforced twice: lazily (a hold past `expires_at` is treated as released by every read and by the conditional write predicate) and actively (the sweeper releases it and emits `inventory.released` so search and the waitlist learn about it). - A hold can be extended once while a 3-D Secure challenge is in progress, so a slow bank page does not lose the customer's seat during authorisation. - *Example* TTLs: tickets 10 minutes, hotels 15 minutes, extension 5 minutes. ### 7.3 Concurrency control | Approach | How it works | Use here | |---|---|---| | Pessimistic locking | `SELECT ... FOR UPDATE` on the rows, then update, then commit | Only inside one short database transaction, never across a payment call | | Optimistic locking | Read version, update `WHERE version = v`; retry on zero rows updated | Seat map edits by venue staff; low-contention admin flows | | Conditional write (compare-and-set) | Single statement whose predicate is the business rule | All customer holds: seats and counted inventory | | Serialised per partition | Queue all writes for one event through one worker | Fallback for extreme single-section contention | Seat hold, all or nothing for the requested seats: ```sql UPDATE seat SET status = 'held', hold_id = :hold_id, held_until = :expires_at, version = version + 1 WHERE event_id = :event_id AND seat_id IN (:seat_ids) AND (status = 'available' OR (status = 'held' AND held_until < now())); -- In the same transaction: if affected rows < requested seats, roll back and return 409. ``` Counted inventory hold for one night: ```sql UPDATE room_night SET held = held + :qty, version = version + 1 WHERE property_id = :p AND room_type = :rt AND night = :n AND held + sold + :qty <= allotment + overbook_limit; ``` Holding locks while calling a payment provider is forbidden: a slow PSP would turn into a lock queue across the whole event. The hold itself is the long-lived "lock", expressed as data with an expiry. ### 7.4 Idempotency Every mutating call carries an `Idempotency-Key`. The booking API writes the key, request hash and a `processing` marker in the same transaction as the hold or booking row; a concurrent duplicate waits or receives 409 `request-in-progress`. Saga steps carry their own step keys (`booking_id:authorise`, `booking_id:capture`) so retries after a crash do not authorise or capture twice. The payment provider receives the step key as its own idempotency key. ### 7.5 Overbooking policy Overbooking applies to counted hotel inventory only, never to seated events. Revenue management sets `overbook_limit` per property, room type and night from historical no-show and late cancellation rates. With allotment $N$, bookings $n$ and an independent show probability $q$, the expected number of guests who arrive is $nq$, and the risk of more arrivals than rooms is $$ P(\text{walk}) = P(X > N), \qquad X \sim \text{Binomial}(n, q) $$ *Example:* $N = 200$, no-show rate 8% so $q = 0.92$, and $n = 210$ (limit 10): mean arrivals $210 \times 0.92 = 193.2$, standard deviation $\sqrt{210 \times 0.92 \times 0.08} \approx 3.93$. With a normal approximation and continuity correction, $z = (200.5 - 193.2)/3.93 \approx 1.86$, so the probability of walking at least one guest is about 3% (the exact binomial value is about 2.4%; compute it exactly when the spreadsheet allows). Revenue management approves the acceptable walk risk; the system enforces the resulting limit, records every overbooked night and triggers the walk workflow (relocation, compensation, notification) when arrivals exceed rooms. ### 7.6 Booking saga and compensation The saga is orchestrated by a durable workflow so each step survives process crashes. Steps and compensations: | Step | Action | Compensation if a later step fails | |---|---|---| | S1 | Hold inventory (7.3) | Release hold | | S2 | Fraud score and purchase limit check | None (read only) | | S3 | Authorise payment | Void authorisation | | S4 | Confirm inventory: `held -> sold` for this hold only | Return to `available` and publish release | | S5 | Capture payment | Refund | | S6 | Issue tickets or hotel confirmation; notify the property | Cancel tickets; send cancellation to property | ```mermaid sequenceDiagram participant C as Customer participant B as Booking API participant O as Saga orchestrator participant I as Inventory participant F as Fraud participant P as Payment participant T as Ticketing C->>B: POST /bookings (hold_id, Idempotency-Key) B->>O: start saga O->>F: score order F-->>O: allow O->>P: authorise (key booking:authorise) alt authorised P-->>O: authorised O->>I: confirm hold (held to sold) alt confirmed I-->>O: sold O->>P: capture (key booking:capture) O->>T: issue tickets O-->>C: booking confirmed else hold expired meanwhile I-->>O: hold-expired O->>P: void authorisation O-->>C: booking failed, seats released end else declined or timeout P-->>O: declined O->>I: release hold O-->>C: payment failed end ``` A payment timeout is unknown, not failed: the orchestrator queries the PSP by idempotency key before deciding to void or proceed. Capture after confirm means a customer is never charged for a seat they did not get; the cost is that an authorisation can occasionally be voided after the customer saw a success screen for authorisation, which the UI avoids by showing success only after S4. ### 7.7 Flash sales and the virtual waiting room - Users who arrive before onsale time get a random queue position when the sale opens, so refreshing early gives no advantage (NFR-08). Users arriving after opening join the back of the queue. - The room admits users at rate $A$ (3.2), adjusted every 10 seconds by observed checkout latency and payment error rate. - Admission issues a signed, short-lived token bound to account and device. The booking API rejects holds without a valid token during metered sales. - When remaining inventory minus active holds reaches zero for a section, the room tells queued users that the section is sold out and offers the waitlist instead of keeping them waiting. - The waiting room page is static and served from the CDN, so the origin sees only the token and counter calls. ### 7.8 Search path versus booking path | Concern | Search path | Booking path | |---|---|---| | Store | Availability index and cache | Inventory store | | Freshness | Seconds (NFR-07) | Exact | | Failure mode | Show stale or approximate availability | Reject the hold with 409 and suggest alternatives | | Scaling | Replicas and cache | Partitions | | Load during onsale | Absorbed by CDN for seat maps of hot sections | Metered by waiting room | The index is fed by inventory events from the outbox. A seat map shows "likely available"; the hold is the only authoritative check. ### 7.9 Consistency boundaries - Strongly consistent inside one inventory partition: one event's seats, or one property's room nights. - A booking spanning two partitions (a hotel plus a concert package) is a saga across two holds, not a distributed transaction. Either both holds succeed or the first is released. - Booking and payment states converge through the saga and daily reconciliation with the PSP (NFR-03); the ledger is the financial source of truth. ## 8. Scalability and performance | Concern | Approach | Limit before redesign | |---|---|---| | Hot event partition | Conditional writes; sub-partition by section for stadiums | *Example:* about 5 000 seat writes/s per partition in load test | | Payment provider throughput | Waiting room meters to contracted rate; second PSP as overflow | Contracted orders/s | | Search spikes | CDN for seat-map tiles, cache with 2-second TTL for hot events | Index replicas | | Hold expiry storms | Sweeper spreads releases; lazy expiry in predicates | Sweeper lag p99 under 5 s | | Hotel inventory growth | Shard by property; archive past nights | Shard count | Latency budget for `POST /v1/holds` (target p99 250 ms): | Step | Budget | |---|---| | Token and limit validation | 10 ms | | Conditional write and commit | 40 ms | | Idempotency record and outbox write (same transaction) | included above | | Response and network | 30 ms | | Headroom for retries on contention | 170 ms | ## 9. Reliability, security and operations ### 9.1 Failure modes | ID | Failure | Impact | Detection | Mitigation | |---|---|---|---|---| | FM-01 | Payment provider timeout | Unknown payment state | Saga step timeout | Query by idempotency key; void if not authorised | | FM-02 | Saga worker crash mid-booking | Booking stuck in `pending` | Age of pending bookings | Durable workflow resumes from last completed step | | FM-03 | Sweeper stopped | Seats stuck as held | Expired-but-held count | Lazy expiry in predicates keeps selling correct; alert and restart | | FM-04 | Index lag | Search shows sold seats | Event age at index | Booking path re-checks; banner on seat map | | FM-05 | Inventory partition failover | Holds fail for one event | Error rate per partition | Waiting room pauses admission for that event | | FM-06 | Duplicate client submit | Double booking attempt | Idempotency conflicts metric | Idempotency key returns first result | | [FM-nn] | [Failure] | [Impact] | [Signal] | [Mitigation] | See Software FMEA (`ops/fmea-software`) to score these, and Chaos Experiment Plan (`ops/chaos-experiment`) to test FM-01 and FM-02 before the first large onsale. ### 9.2 Security, abuse and fraud | ID | Threat | Control | |---|---|---| | SEC-01 | Bots buying inventory for resale | Edge bot management, waiting room randomisation, token bound to account and device, purchase limits per account, payment instrument and address | | SEC-02 | Inventory hoarding through holds without payment | Holds per account limited; repeated expiry without purchase lowers admission priority | | SEC-03 | Card testing on the booking endpoint | Rate limits per account and IP, fraud score before authorisation, 3-D Secure | | SEC-04 | Token replay or forgery | Signed tokens with expiry, one use per hold | | SEC-05 | Account takeover to use stored cards | Step-up authentication for new device plus stored card | | SEC-06 | Card data exposure | Hosted payment fields from the PSP; no card data on our servers | | [SEC-nn] | [Threat] | [Control] | Model the remaining threats in a Threat Model (`security/threat-model`). ### 9.3 Observability | Signal | Metric | Alert threshold | |---|---|---| | Double sale audit | Seats with more than one ticket | Any | | Hold conflicts | 409 share of hold attempts per event | Above 30% for 5 min (UX problem, not a fault) | | Saga age | Bookings pending over 2 min | Above 0.5% | | Checkout rate vs admission rate | Orders/s against admitted users/s | Ratio drift above 20% | | Payment errors | Decline and timeout rate by PSP | Doubling vs. baseline | | Index lag | p99 event age | Above 5 s | ### 9.4 Cost estimate | Item | Unit cost | Quantity | Monthly cost | |---|---|---|---| | Inventory and booking database | [USD per node-month] | *Example:* 6 nodes | [USD] | | Search index and cache | [USD per node-month] | *Example:* 8 nodes | [USD] | | Waiting room (edge compute, requests) | [USD per million requests] | [Requests] | [USD] | | Workflow engine | [USD per month] | 1 cluster | [USD] | | **Total** | | | **[USD]** | ## 10. Trade-offs and alternatives | Decision | Options | Chosen | Why | What we give up | |---|---|---|---|---| | Seat concurrency | Row locks across checkout; optimistic retry; conditional write with hold rows | Conditional write plus TTL hold | Short transactions, no lock queues during onsales | Hold state must be expired and swept | | Payment timing | Charge first, then reserve; reserve, authorise, capture | Reserve, authorise, confirm, capture | Never charge without inventory | Authorisation voids for expired holds | | Cross-service booking | Two-phase commit; saga | Saga with compensations | Partitions and PSP cannot join a 2PC | Temporary intermediate states visible | | Onsale control | Autoscale everything; rate limit; waiting room | Waiting room metered to checkout capacity | Payment capacity cannot autoscale | Users wait; queue must be fair and honest | | Search freshness | Query inventory store; index with lag | Index with lag | Protects booking store from search load | Seat map can be seconds stale | | Overbooking | None; per property limit | Per property, room type and night, hotels only | Recovers no-show revenue within approved risk | Walk workflow and customer cost | | [Decision] | [Options] | [Chosen] | [Why] | [Cost] | ## 11. Delivery plan and risks ### 11.1 Milestones | Milestone | Scope | Exit criterion | Target date | |---|---|---|---| | M1 | Inventory service, conditional holds, sweeper | Zero double sales in a 10 000 concurrent-user load test | YYYY-MM-DD | | M2 | Booking saga with payment authorise, capture, void | Chaos test of PSP timeout passes; reconciliation clean for 14 days | YYYY-MM-DD | | M3 | Search index from outbox events | Index lag p99 under 5 s at forecast load | YYYY-MM-DD | | M4 | Waiting room for onsales | Rehearsal onsale at 2 times forecast with fair admission audit | YYYY-MM-DD | | M5 | Hotel overbooking with walk workflow | Revenue management sign-off | YYYY-MM-DD | ### 11.2 Rollout and rollback [*Example:* Migrate one venue at a time; the old and new inventory never own the same event. Run the first large onsale on the new system with a rehearsal the week before using synthetic users. Rollback for an event before onsale is a configuration switch; after onsale, the event stays on the system that sold it.] ### 11.3 Risks | ID | Risk | Likelihood | Impact | Mitigation | Owner | |---|---|---|---|---|---| | R-01 | *Example:* PSP throughput lower than contracted during peak | Medium | High | Rehearsal with PSP; second provider; waiting room adapts admission | [@name] | | R-02 | *Example:* Bots defeat device binding | High | Medium | Layered controls; post-sale cancellation of abusive orders | [@name] | | R-03 | *Example:* Overbooking limits too aggressive for a property | Low | High | Weekly review of walks; automatic limit reduction after a walk | [@name] | | R-04 | *Example:* Consumer law on displayed queue times and pricing | Medium | Medium | Legal review of waiting room and checkout copy | [@name] | | [R-nn] | [Risk] | [L / M / H] | [L / M / H] | [Mitigation] | [@name] | ## 12. Appendices ### 12.1 Traceability | Requirement | Design element | Verification | |---|---|---| | NFR-01 | 7.3 conditional seat write | Load test with competing clients; nightly audit | | NFR-02 | 7.3 counted predicate, 7.5 limits | Nightly allotment audit | | NFR-03 | 7.6 saga, 7.4 step keys | Daily PSP reconciliation | | NFR-04 | 7.8 search path | Load test at 20 000 queries/s | | NFR-08 | 7.7 randomised admission | Queue assignment audit | | [ID] | [Element] | [Test] | ### 12.2 Open questions | ID | Question | Owner | Due | |---|---|---|---| | OQ-01 | *Example:* Should accessible seats be held back until 24 hours before onsale closes? | [@name] | YYYY-MM-DD | | OQ-02 | *Example:* Do partner hotels accept overbooking on non-refundable rates? | [@name] | YYYY-MM-DD | | [OQ-nn] | [Question] | [@name] | YYYY-MM-DD | ### 12.3 Glossary | Term | Meaning | |---|---| | Allotment | Number of rooms of a type a property makes available for a night | | Hold | Temporary reservation of inventory with an expiry | | Identified inventory | Inventory where each unit is distinct (a seat) | | Counted inventory | Inventory tracked as a quantity (rooms of a type, general admission) | | Saga | Sequence of local transactions with compensating actions instead of a distributed transaction | | Walk | Relocating a hotel guest who cannot be accommodated because of overbooking | | Waiting room | Queue in front of the booking path that admits users at a controlled rate | ### 12.4 References | Reference | Use | |---|---| | RFC 9457 Problem Details for HTTP APIs | Error format in 4.3 | | IETF draft "The Idempotency-Key HTTP Header Field" | Idempotency semantics in 7.4 (draft; pin the revision) | | Garcia-Molina and Salem, "Sagas", ACM SIGMOD 1987 | Saga pattern in 7.6 | | Payment Ledger System Design (`system-design/design-payment-ledger`) | Payment states and ledger | | Event Sourcing and CQRS Design (`architecture/event-sourcing-cqrs`) | Read model for the availability index | | System Design Document (`system-design/system-design-document`) | General structure | | Capacity Plan (`ops/capacity-plan`) | Onsale capacity planning | ## Standard and licence note Based on common industry practice for inventory, reservation and ticketing systems and on the references in 12.4; the Idempotency-Key header field is an IETF draft, not a published RFC. This is a reference design to adapt, with illustrative numbers; it is a format, not legal, consumer-protection or payment-compliance advice. Check the current version of every cited specification. Original text written for this collection. Template licence: MIT.