System Design: [Fraud platform name] real-time payment fraud detection

Reference design for scoring card and account-to-account payments for fraud at authorisation time. It covers the decision API and its latency budget, features computed from the transaction and from streaming aggregates, the online and offline feature stores, rules combined with a machine-learning score, decisions (allow, challenge, review, block), manual review queues, label delay, feedback loops and selection bias, and model monitoring. It follows the structure of the System Design Document template and is pre-filled with a worked design to adapt. Replace every [bracketed] value and every row marked Example; values marked Example are assumptions, not benchmarks. This is an engineering format, not legal, card-network or supervisory advice.

Document control

Approvals

Revision history

Contents

1. Overview
2. Requirements
3. Capacity estimation
4. API design
5. Data model
6. High-level design
7. Detailed design
8. Scalability and performance
9. Reliability, security and operations
10. Trade-offs and alternatives
11. Delivery plan and risks
12. Appendices

1. Overview

1.1 Problem statement

[Example: We process payments for 120 000 merchants. Fraud losses rose to 0.11% of volume last year while our false decline rate is estimated at 2%, which merchants report as lost sales. Today's controls are 900 static rules evaluated in the payment service; nobody knows which rules still work. Features such as "cards used on this device in the last hour" are computed by a nightly batch, so a fraud ring testing stolen cards is visible only the next morning. The review team receives 40 000 cases a day and works them in arrival order.]

1.2 Proposed solution

[Example: A fraud decision service is called synchronously in the authorisation path with a 100 ms p99 budget. It joins request features with aggregates kept fresh by a streaming pipeline in an online feature store, runs a gradient-boosted model and a versioned rule set, and returns a decision with reasons. Thresholds map scores to allow, 3-D Secure challenge, review or block per merchant risk profile. Review cases are prioritised by expected loss. Labels from chargebacks, fraud reports and review outcomes flow back into an offline store with point-in-time correct features; models retrain on a fixed cadence and ship through shadow and canary stages under monitoring.]

1.3 Decisions requested from reviewers

1.4 45-minute interview path

2. Requirements

2.1 Core functional requirements

2.2 Non-functional requirements and decision economics

Decision economics (Example): a missed fraud costs the amount plus a chargeback fee and network penalties; a false decline costs the merchant margin and a share of customer lifetime value. The threshold for blocking sits where the expected fraud cost of allowing equals the expected cost of declining a good payment; a challenge band sits between allow and block where 3-D Secure shifts liability or adds friction.

2.3 Below the line (out of scope)

• [Example: anti-money-laundering transaction monitoring (separate system, different labels)]
• [Example: account opening and identity verification]
• [Example: chargeback dispute management]
• [Example: model training research and feature discovery notebooks]
• [Example: authorisation routing and the payment ledger (see Payment Ledger System Design (system-design/design-payment-ledger))]
• [Item deferred, with the release that will take it]

3. Capacity estimation

3.1 Assumptions

3.2 Throughput

3.3 Storage

3.4 Review capacity

3.5 Summary

4. API design

4.1 Endpoints

4.2 Request and response examples

POST /v1/decisions
Content-Type: application/json
X-Request-Deadline-Ms: 90

{"payment_id": "pay_6f31", "merchant_id": "m_2201", "amount": {"value": 18900, "currency": "USD"}, "card_fingerprint": "cf_a91e", "device_id": "dv_77c0", "ip": "[ip]", "email_hash": "eh_19b2", "channel": "ecommerce"}

HTTP/1.1 200 OK
Content-Type: application/json

{"decision": "challenge", "score": 0.62, "reasons": ["card_new_device", "ip_country_mismatch"], "model_version": "gbm-2026-10-01", "rule_set_version": "rs-418", "fallback": false}

• If the deadline passes, the caller applies its configured fallback (7.1) and records fallback: true with the later outcome call.
• Reason codes are stable identifiers documented for merchants; they never reveal thresholds or rule logic.

5. Data model

5.1 Entities

Entities are the things fraud attaches to: cards, devices, emails, IP addresses, accounts and merchants. Features are aggregates keyed by one entity or a pair of entities over a window.

5.2 Storage choices

5.3 Access patterns

6. High-level design

7. Detailed design

7.1 Latency budget and fallback

Fallback options: fail open (allow), fail closed (block), fail to challenge, or fall back to rules only. Choice: rules-only path inside the service when features time out; caller-side fail open below a small amount and fail to challenge above it when the service itself is down. Reason: blocking all payments during an outage costs more than a short window of weaker screening, but large amounts deserve friction.

7.2 Deep dive: streaming features and train-serve parity

Choice: streaming aggregates for windows up to 24 hours, batch for longer windows, and log every served vector. Reason: card-testing and account-takeover attacks unfold in minutes, so velocity features must be fresh; logging served vectors removes the most common parity bug, where training recomputes features differently from serving.

• Feature definitions live in one registry and generate both the stream job and the offline backfill.
• Offline training joins use event time, never processing time, so a payment only sees aggregates of events that happened before it.
• Late events update aggregates but are flagged; the parity job compares a sample of served values against recomputed values daily (NFR-07).
• Every feature has an owner and a Data Contract (ai-data/data-contract) with the upstream producer.

7.3 Deep dive: rules plus model

Choice: model score plus a rule layer. Rules have three kinds: policy (always block sanctioned or blocked instruments), overrides (merchant allow-lists with expiry) and tactical (temporary rules against an active attack, expiring in Example 14 days unless renewed). Reason: the model handles ranking; humans need a lever that works before the next retrain. Every rule shows its backtest and its live hit and precision rate, and rules with no measured value are retired.

How real systems do it (paraphrased from a public write-up; not copied)

Stripe's guide to machine learning for fraud describes how its Radar product turns each payment into a vector of features drawn from the payment and from activity seen across many businesses, trains models on labelled history, and judges them with precision, recall and score distributions rather than a single accuracy number. It treats the choice of threshold as a business trade-off between blocking fraud and turning away good customers, lets users add their own rules and see how a rule would have performed on past payments before turning it on, and uses manual review as a way to handle the uncertain middle. It also stresses retraining as fraud patterns change. This design mirrors that split: model for ranking, rules with backtests for policy, review for the middle band.

Source: Stripe, "A primer on machine learning for fraud detection", Stripe Guides, last updated 15 December 2021, https://stripe.com/guides/primer-on-machine-learning-for-fraud-protection

7.4 Manual review queues

• Cases enter only from the review band and only above an amount floor; the band is widened or narrowed so the forecast case count matches staffed capacity (3.4).
• Priority is expected loss: score times amount, adjusted by merchant tier. Cases older than the SLA escalate.
• The payment is held (authorised, not captured, or held in a pending state) while under review. If the SLA expires, the default action is set per merchant (release or cancel).
• Analysts see the reasons, the entity graph (other payments sharing card, device or email) and prior decisions. Their verdict becomes a label with source review.
• A random sample of decided cases is re-reviewed weekly to measure analyst agreement; low agreement points to unclear guidelines, not bad analysts.

7.5 Deep dive: label delay, feedback loops and selection bias

Labels arrive late and only for some payments:

Problems and choices:

• Delay: a model trained today learns from payments at least one maturity window old. Choice: train on matured data for the main model; monitor recent weeks with early labels and with label-free signals (score distribution, feature drift, decline rate). Reason: waiting for full labels is the only unbiased option; early labels catch breakage sooner.
• Selection bias: blocked payments never get chargebacks, so their labels are missing and the next model learns from a filtered world. Options: ignore (bias grows with each retrain); infer labels for blocked payments (circular); allow a small random exploration sample of would-be-blocked payments below an amount cap and weight it in training and evaluation. Choice: exploration sample with a strict loss budget approved by the risk owner (Example: 0.1% of blocks, amount at most USD 50). Reason: it is the only source of unbiased estimates of false decline rate (NFR-05) and of the block band's true fraud rate.
• Feedback loops with rules: a tactical rule that blocks an attack removes those payments from future training labels; record which rule fired so training can include or exclude them deliberately.
• Adversarial adaptation: fraudsters probe thresholds. Do not expose scores to merchants at full precision and rotate tactical rules.

7.6 Model lifecycle and monitoring

Monitoring signals, owners and thresholds belong in Model Monitoring Plan (ai-data/model-monitoring-plan); describe the model in a Model Card (ai-data/model-card). If the operator is a US banking organisation, the fraud model may fall under supervisory model risk management guidance (SR letter 26-2, "Revised Guidance on Model Risk Management", 17 April 2026); confirm scope with model risk management and record validation in Model Validation Report (ai-data/model-validation-report).

8. Scalability and performance

9. Reliability, security and operations

9.1 Failure modes

9.2 Security and privacy

9.3 Observability

10. Trade-offs and alternatives

11. Delivery plan and risks

11.1 Milestones

11.2 Risks

12. Appendices

12.1 Traceability

12.2 Open questions

12.3 Glossary

12.4 References

Standard and licence note

Based on common industry practice for payment fraud scoring and on the public references in 12.4. Company guides were used as inspiration only; the text, diagrams and numbers are original and illustrative. SR letter 26-2 is US supervisory guidance aimed at banking organisations; check its current text and scope before relying on it. Card-network and data protection rules vary and change; this is an engineering format, not legal advice. Template licence: MIT.