System Design: [Configuration and feature flag service name]

Reference design for a dynamic configuration and feature flag service: the system that lets engineers change runtime behaviour (flags, limits, thresholds, targeting rules) across thousands of servers and millions of client devices without a deploy. It covers configuration as code, validation, targeting rules, staged and percentage rollout, consistent bucketing, fan-out distribution and SDK caching, kill switches, audit and the consistency of evaluation. The SDK interface follows the concepts of the OpenFeature specification (release v0.9.0, July 2026; sections carry their own stability status). It follows the structure of the System Design Document template and is pre-filled with a worked design to adapt. Replace every [bracketed] value and every row marked Example; all numbers are illustrative assumptions, not benchmarks.

Document control

Approvals

Revision history

Contents

1. Overview
2. Requirements
3. Capacity estimation
4. API design
5. Data model
6. High-level design
7. Detailed design
8. Scalability and performance
9. Reliability, security and operations
10. Trade-offs and alternatives
11. Delivery plan and risks
12. Appendices

1. Overview

1.1 Problem statement

[Example: Teams use four different mechanisms for runtime switches: environment variables, a key-value store read on every request, a vendor SDK on mobile and hard-coded constants. A typo in a JSON value last quarter disabled checkout in one region for 47 minutes because nothing validated the change and it reached every server at once. Nobody can answer "what changed in the last hour" across systems, and 61% of flags have no owner.]

1.2 Proposed solution

[Example: One service stores all configuration and flag definitions as versioned, schema-validated objects in a repository. Every change goes through review, automated validation and a staged rollout that starts on a canary population and can roll back automatically. A distribution tier pushes changes to an agent on each host, which caches them on disk; server SDKs read from the local agent and evaluate rules in process. Browsers and mobile apps call a stateless evaluation endpoint that returns already evaluated values for the caller's context, so targeting rules never leave the server side. Percentage rollouts use deterministic hashing so a user stays in the same bucket as the percentage grows. Every change is audited and every flag has an owner and an expiry date.]

1.3 Decisions requested from reviewers

1.4 45-minute interview path

2. Requirements

2.1 Core functional requirements

2.2 Non-functional requirements

2.3 Below the line (out of scope)

• [Example: experiment analysis and statistics (experimentation platform consumes exposure data from this service)]
• [Example: secrets such as API keys and passwords (secret manager; this service must refuse values that look like secrets)]
• [Example: infrastructure configuration such as Kubernetes manifests (GitOps pipeline)]
• [Example: distribution of large binary artefacts such as ML models above 1 MB (artefact store; this service distributes a pointer and checksum)]

3. Capacity estimation

3.1 Assumptions

3.2 Storage

60,000 objects x 2 KB = 120 MB for the full current state.

Consequence: the whole state fits in memory of the distribution tier, but not sensibly in every process; servers subscribe by namespace (Example: average 500 objects, about 1 MB per process).

3.3 Fan-out

One change reaches 5% of 500,000 processes = 25,000 processes. At 2 KB each that is 50 MB per change; at the peak of 50 changes per minute it is 2.5 GB per minute, about 42 MB/s.

Consequence: pushing from one origin to 500,000 processes is the problem. Push once per host to a local agent (25,000 agents, so 1,250 agents per change instead of 25,000 processes) through regional distribution proxies; processes read from the agent over a local socket or a memory-mapped file.

3.4 Client refresh rate

20 million devices / 300 s = about 66,700 requests/s. Each request evaluates 200 flags; at Example: 2 microseconds per evaluation that is 66,700 x 200 x 2 microseconds = 26.7 CPU seconds per second, about 27 cores.

Consequence: remote evaluation for clients is affordable (about 54 cores with 2x headroom across regions), so there is no need to ship rules to devices. Responses are personalised, so a CDN cannot cache them; instead the endpoint returns a hash of the evaluated result and clients send it back in a known_hash field, so an unchanged result returns a short "not modified" body. (Do not rely on If-None-Match here: on a POST, RFC 9110 answers a failed condition with 412, not 304.)

3.5 Server evaluations and exposure data

2 million requests/s x 10 evaluations = 20 million evaluations/s, all in process.

Consequence: evaluation CPU is negligible, but logging every evaluation is not (20 million events/s). SDKs aggregate counts per flag, variant and minute, and emit individual exposure events only for flags linked to an experiment.

3.6 Summary

4. API design

4.1 Interfaces

4.2 SDK contract

Server SDKs expose an OpenFeature-style interface: an evaluation API with typed getters, an evaluation context (targeting key plus attributes), providers that implement the actual lookup, and hooks that run before and after evaluation. The detailed result carries the flag key, value, variant, reason and, when evaluation fails, an error code. Use the specification's reason values (for example STATIC, DEFAULT, TARGETING_MATCH, SPLIT, CACHED, DISABLED, STALE, ERROR) so dashboards can explain each result. The default value is always passed by the caller and returned on any error; evaluation never throws into application code.

// Example: server-side use through an OpenFeature-style client
ctx := openfeature.NewEvaluationContext("user-8d1f", map[string]any{
    "country": "DE", "plan": "pro", "app_version": "5.12.0",
})
details, _ := client.BooleanValueDetails(reqCtx, "checkout.new-tax-engine", false, ctx)
// details.Value, details.Variant, details.Reason (e.g. "SPLIT"), details.ErrorCode

4.3 Client evaluation request

POST /v1/evaluate HTTP/1.1
Content-Type: application/json

{"context": {"targetingKey": "user-8d1f", "platform": "ios", "app_version": "5.12.0", "country": "DE"}, "flags": ["*client"], "known_hash": "ev-3f9a"}

The response lists key, value and variant per flag, plus the snapshot version used, so a client can report which version it saw. The OpenFeature project also publishes a remote evaluation protocol (OFREP), defined in the open-feature/protocol repository and described in Appendix C of the specification; consider adopting it for this endpoint so standard client providers can call it.

4.4 Errors

Authoring API errors use Problem Details (RFC 9457). Validation failures return 422 with one entry per failed rule, so the author sees every problem at once.

5. Data model

5.1 Entities

Definitions are immutable once approved; a change creates a new version and a rollout that moves populations from the old version to the new one.

5.2 Storage choices

5.3 Access patterns

6. High-level design

7. Detailed design

7.1 Deep dive 1: distribution and propagation

Choice: C. Reason: it cuts the fan-out by the number of processes per host, removes the network from evaluation entirely and gives every host a durable cache. Agents also poll every Example: 5 minutes with the last version they hold, so a missed stream message is repaired without manual action. Reconnects use jittered exponential backoff.

The sequence for one approved change:

Large values (above Example: 1 MB) are not pushed inline: the snapshot holds a URL and checksum, and agents fetch the blob from the artefact store or from peers.

7.2 Deep dive 2: targeting and consistent bucketing

Rules are evaluated top to bottom; the first match wins:

1. If the flag is killed, return the off variant (reason DISABLED).
2. Individual targets (allow lists for testers).
3. Segment and attribute rules (country in [DE, FR] and app_version >= 5.12.0).
4. Percentage split for the remaining population.
5. Default variant.

Choice: C. Reason: no storage and no network, and increasing a rollout from 10% to 20% keeps the first 10% in. Bucket rule:

bucket = hash32(flag.salt + "." + context.targetingKey) mod 10000
in_rollout = bucket < percentage * 100      # 25.5% -> bucket < 2550

• Use one hash function across all SDKs (Example: MurmurHash3 32-bit) and publish conformance vectors so every language agrees.
• Give each flag its own salt so the same users are not always first into every rollout; reuse a salt deliberately when two flags must align.
• If the targeting key is missing, return the default with error code TARGETING_KEY_MISSING rather than bucketing on an empty string, which would put every anonymous user in one bucket.

7.3 Deep dive 3: safe change and kill switches

Choice: C, with A reserved for kill switches. Reason: the outage in 1.1 was a valid-looking change applied everywhere at once; staged exposure plus automatic rollback limits a bad change to the canary population.

Stages (Example): internal hosts, 1% of hosts in one region, one region, all regions. Each stage bakes for Example: 5 minutes. The controller compares error rate, latency and crash rate of hosts on the new version against hosts still on the old one and rolls back automatically on a regression above threshold.

Validation before any stage:

• JSON schema per namespace (types, ranges, required fields).
• Rule linting: unreachable rules, overlapping segments, percentage totals.
• Reference checks: segments and variants that exist.
• Secret scanning: reject values that look like credentials.
• Optional custom validators owned by the consuming team, run in CI.

Kill switches are pre-declared off variants. A kill sets the flag version to "off" through a priority channel that skips bake time, is limited to users with the kill permission, and is always audited.

7.4 Deep dive 4: consistency of evaluation

Choice: per-namespace snapshot versions with request-level pinning. Reason: global strong consistency would put a consensus round on every change and would not stop clients from holding stale values anyway; making staleness bounded and visible is enough for flags.

How real systems do it (paraphrased from Tang et al., "Holistic Configuration Management at Facebook", SOSP 2015; see 12.4). The paper describes a stack in which configuration is written as code, compiled and validated before it is committed, and stored in version control, so every change is reviewed and reversible. Changes are tested automatically on a small set of servers before wide release. A separate gating tool evaluates rollout and targeting rules inside the application, and mobile apps receive configuration through their own mechanism. Distribution is layered: a replicated store feeds proxies on each server, and large configurations travel through a hybrid of subscription and peer-to-peer transfer. The authors report that configuration errors were a major cause of incidents, which motivated treating config changes with the same rigour as code. Lesson for this design: invest first in validation, canary testing and per-host caching, not in a faster database.

7.5 Flag lifecycle

A weekly report lists expired flags by owner; the code scanner flags references to archived keys in pull requests.

8. Scalability and performance

Propagation budget for a normal change (target p99 30 s after the stage starts):

9. Reliability, security and operations

9.1 Failure modes

9.2 Security and privacy

Review the privacy impact with the Privacy Threat Model Template (security/privacy-threat-model) when context attributes include personal data.

9.3 Observability

9.4 Cost estimate

10. Trade-offs and alternatives

11. Delivery plan and risks

11.1 Milestones

11.2 Rollout and rollback

[Example: Migrate one namespace at a time. During migration the SDK provider reads both the old mechanism and the new service and logs mismatches; switch the provider to the new service once mismatches are zero for 7 days.]

11.3 Risks

12. Appendices

12.1 Open questions

12.2 Traceability

12.3 Glossary

12.4 References

Standard and licence note

Based on the OpenFeature specification (release v0.9.0, sections with their own stability status) for the SDK interface and on RFC 9457 for errors. The Facebook paper is used as inspiration only and is paraphrased, not copied. All numbers are illustrative examples. Check the current version and section status of the specification at openfeature.dev before you implement. Original text written for this collection. Template licence: MIT.