Key Management and Secrets Service Design Template (KMS)

System Design: [Key management and secrets service name]

Reference design for an internal key management service (KMS) and secrets service: a root of trust in hardware security modules (HSMs), a key hierarchy, envelope encryption with data keys, key versions and rotation, key states, a secrets store with leases and dynamic credentials, policy and audit, caching, and multi-region availability. Key lifecycle and cryptoperiods follow NIST SP 800-57 Part 1 Revision 5, "Recommendation for Key Management: Part 1 - General" (May 2020); Revision 6 was released as an initial public draft on 5 December 2025 and is not final as of October 2026. It follows the structure of the System Design Document template, adds a 45-minute interview path, access-pattern and failure-mode tables and a cited "How real systems do it" box, and is pre-filled with a worked design to adapt. Replace every [bracketed] value and every row marked Example; values marked Example are assumptions, not benchmarks. This is a design format, not security certification advice; confirm cryptographic choices with your security team against the current NIST publications.

Document control

Approvals

Revision history

Contents

1. Overview
2. Requirements
3. Capacity estimation
4. API design
5. Data model
6. High-level design
7. Detailed design
8. Scalability and performance
9. Reliability, security and operations
10. Trade-offs and alternatives
11. Delivery plan and risks
12. Appendices

1. Overview

1.1 Problem statement

[Example: About 3 000 services keep database passwords and API tokens in environment files and deployment variables, rotated by hand once a year at best. Data at rest is encrypted by each storage system with its own static key, so no one can say which keys protect which data, rotate them, or prove who used them. An auditor asked for a key inventory and an access log; we could provide neither.]

1.2 Proposed solution

[Example: A regional KMS holds customer master keys (KEKs) whose key material never leaves HSMs in plaintext. Services encrypt data with data encryption keys (DEKs) generated by the KMS and store each DEK wrapped under a KEK next to the data (envelope encryption). Writers reuse one DEK for at most 5 minutes and readers cache unwrapped DEKs for 5 minutes, so the KMS sees thousands rather than millions of calls per second. KEKs have versions; rotation adds a version for new encryption while old versions still decrypt. A secrets service, encrypted by the same KMS, stores static secrets with versions and issues short-lived dynamic credentials. Workloads authenticate with their platform identity, every operation is authorised by policy and written to an append-only audit log.]

1.3 Decisions requested from reviewers

1.4 45-minute interview path

Use this order when the document serves an interview or a short design review. The written sections below are the long form of each step.

Start the default deep dive with the naive answer: send every object to the KMS to encrypt. Show that payload size limits and call volume make that impossible, then introduce envelope encryption, then show that even one KMS call per object is too many and introduce DEK reuse and caching with their limits.

2. Requirements

2.1 Core functional requirements

2.2 Non-functional requirements

2.3 Below the line (out of scope)

Name these explicitly so reviewers and interviewers do not pull the design toward them.

• [Example: public certificate issuance and TLS certificate lifecycle (see the PKI architecture)]
• [Example: end-user password storage (handled by the identity provider)]
• [Example: payment HSM functions such as PIN translation]
• [Example: migration of every algorithm to post-quantum schemes (tracked in the PQC migration plan)]
• [Item deferred, with the release that will take it]

3. Capacity estimation

3.1 Assumptions

All figures are per region.

3.2 KMS call rate

Naive design, one KMS call per object:

With DEK reuse and reader caching:

Secrets add $60\,000 \times 20 / 3\,600 \approx 333$ reads/s for lease renewals and about 400/s during a deploy that restarts 10% of instances within 5 minutes. The total is about 50 737 operations/s:

Each DEK then protects about $200\,000 / 1\,000 \times 300 = 60\,000$ objects, far below the limit of $2^{32}$ AES-GCM invocations with random IVs per key in NIST SP 800-38D section 8.3.

3.3 Storage and audit

• Secrets: 100 000 secrets x 10 versions x 4 KiB is about 4.1 GB, small.
• Data keys: one new DEK per writer every 5 minutes is 288 000 wrapped DEKs per day and about 105 million per year; they live with the data, not in the KMS.
• Re-wrap after a KEK rotation: 105 million DEKs at 2 000 re-wraps/s take about 14.6 hours, a background job, not an outage.
• Audit: 50 737 events/s x 86 400 s x 500 B is about 2.19 TB per day before compression, about 0.8 PB per year. Cache hits are not KMS events, which is why the audit volume stays manageable; the log goes to append-only storage with compression and tiering.

3.4 Summary

4. API design

4.1 Operations

4.2 Wrapped key format

The wrapped DEK is self-describing so Decrypt needs no key ID from the caller, and the encryption context is bound as additional authenticated data (AAD): unwrapping with a different context fails.

{"format": 1, "keyId": "kek-orders-eu", "keyVersion": 4, "alg": "AES-256-GCM", "iv": "[base64, 12 bytes]", "ciphertext": "[base64 wrapped DEK]", "tag": "[base64, 16 bytes]", "contextHash": "[sha-256 of canonical context]"}

{"context": {"service": "orders", "table": "invoices", "tenant": "t_8812"}}

Errors use RFC 9457 problem details; AccessDenied, KeyDisabled, ContextMismatch and Throttled are distinct types so clients can tell a policy problem from a capacity problem.

5. Data model

5.1 Entities

KEK versions are stored only as blobs wrapped by an HSM-held domain key; DEKs are not stored in the KMS at all.

5.2 Storage choices

5.3 Access patterns

6. High-level design

7. Detailed design

The deep dives in this section, each with options, choice and reason:

7.1 Key hierarchy

7.2 Envelope encryption with reuse and caching

Rules for reuse and caching:

• A writer uses one DEK for at most 5 minutes or a set number of objects, whichever comes first, and for one encryption context only.
• Each object gets a fresh random 96-bit IV; the count per DEK stays far below the SP 800-38D section 8.3 limit (3.2).
• Reader caches are keyed by the exact wrapped DEK bytes and context, hold plaintext DEKs in memory only, and expire after 5 minutes. A disabled KEK therefore stops all new decryption within 5 minutes.
• The trade-off: a compromised workload host exposes the DEKs in its cache, which cover at most 5 minutes of data per context. Highly sensitive data sets a shorter window or turns caching off.

7.3 Rotation and re-wrapping

Rotation creates a new KEK version that becomes the only one used for new wrapping; old versions remain able to unwrap. Data is not re-encrypted: re-wrapping a DEK under the new version (ReEncrypt) changes only the small wrapped blob. A background job re-wraps every stored DEK when policy requires retiring an old version, about 14.6 hours for a year of DEKs (3.3). DEKs themselves are rotated implicitly every writer window.

NIST SP 800-57 Part 1 Rev. 5 section 5.3.6 discusses cryptoperiods by key type. For symmetric data-encryption keys it suggests an originator-usage period on the order of a day or a week when large volumes are encrypted quickly, up to about two years for smaller volumes, and a recipient-usage period of up to three years beyond that; for symmetric key-wrapping keys it links the period to how many keys are wrapped. Set your own periods in the cryptography standard and record them per key in the rotation period field.

7.4 Key states

The KMS tracks each key version through the states described in NIST SP 800-57 Part 1 Rev. 5 section 7. The diagram shows the transitions this service allows.

7.5 Secrets service

• Static secrets: stored encrypted under a KEK, versioned, read by path. Rotation writes a new version; consumers read "current" and the old version stays valid for an overlap window agreed with the target system.
• Dynamic credentials: for supported databases the engine creates a user with the role's grants for each lease (1 hour, renewable) and drops it on expiry or revocation. Leaked credentials expire on their own, and audit shows which workload held which user.
• Delivery: a sidecar or SDK fetches secrets at start and renews leases; secrets never go into images, repositories or environment files.
• Authentication: workloads prove identity with the platform's workload identity (signed service account token or certificate), never with a secret that itself needs storing.

7.6 Availability versus security

7.7 How real systems do it

How real systems do it (paraphrased, not copied). The AWS document "AWS KMS cryptographic details" describes a tiered service: API hosts that terminate TLS and authorise requests, and a fleet of FIPS 140-3 validated HSMs. A KMS key is a logical container for one or more HSM backing keys; each backing key version is exported only in encrypted form, wrapped by HSM-held domain keys, and stored in durable storage. Rotation adds a new backing key for new encryption while older ones still decrypt. The document lists domain keys as rotated daily (relaxed at times to at most weekly), derives a fresh encryption key from the backing key for each encryption, and describes data keys returned in plaintext and encrypted form for envelope encryption, with an optional encryption context bound as additional authenticated data. The document also says direct encryption under a KMS key suits only data under about 4 KB, which is why large data uses data keys. Read the original in 12.3; this box keeps only the general idea.

8. Scalability and performance

Decrypt latency budget (target p99 10 ms):

9. Reliability, security and operations

9.1 Failure modes

9.2 Security

9.3 Observability

9.4 Cost estimate

10. Trade-offs and alternatives

11. Delivery plan and risks

11.1 Milestones

11.2 Rollout and rollback

[Example: Storage systems adopt the SDK one at a time. New data is written with KMS envelopes while old data stays under the old static key; a background job re-encrypts old data. Rollback keeps both readers in place, so data written either way remains readable.]

11.3 Risks

12. Appendices

12.1 Open questions

12.2 Glossary

12.3 References

Standard and licence note

Based on NIST SP 800-57 Part 1 Rev. 5 (May 2020), NIST SP 800-38D, SP 800-38F and FIPS 140-3, which are public US government publications, paraphrased here in original words with section numbers only. Revision 6 of SP 800-57 Part 1 was an initial public draft as of October 2026; check the NIST page for the current status before relying on cryptoperiod or algorithm guidance. The AWS documentation is paraphrased in 7.7 only; no text, figures or tables were copied, and all numbers marked Example are illustrative. This is a design format, not security certification advice. Original text written for this collection. Template licence: MIT.