# Deployment Runbook: [Service name] [Release version]

> Step-by-step runbook for deploying one release to production: release
> metadata, pre-deployment checks, go/no-go decision, deployment steps
> with commands and verification, progressive rollout, monitoring,
> rollback, post-deployment checks and communication. Based on common
> release engineering practice. Replace every `[bracketed]` value and
> every row marked *Example*.

<!--
Guidance: How to use this template
- Copy this runbook for each release, or keep one per service and fill
  section 1 per release. Execute it; do not deploy from memory.
- Every step has a verification. If a verification fails, stop and use
  the rollback section rather than improvising.
- Write the rollback procedure before the deployment procedure, and
  test it in staging with the same artefact.
- Record actual times in the "Done at" columns; they become the audit
  trail and help estimate future windows.
-->

| Field | Value |
|---|---|
| Runbook ID | [DRB-SERVICE-001] |
| Service | [Service name] |
| Release version | [v2.14.0] |
| Release type | [Standard / Hotfix / Emergency] |
| Change request | [CHG-nnnn] |
| Deployment window | YYYY-MM-DD [hh:mm] to [hh:mm] [time zone] |
| Release manager | [@name] |
| Deployer | [@name] |
| Reviewer / second pair of eyes | [@name] |
| Status | [Planned / In progress / Completed / Rolled back] |

## 1. Release metadata

<!-- Guidance: Identify exactly what is being deployed so anyone can reproduce or reverse it. -->

| Item | Value |
|---|---|
| Artefact | *Example:* `registry.example.com/checkout-api:2.14.0` |
| Artefact digest | *Example:* `sha256:[digest]` |
| Source commit / tag | *Example:* `v2.14.0` at `[commit hash]` |
| Previous version (rollback target) | *Example:* `v2.13.2`, digest `sha256:[digest]` |
| Release notes | [Link] |
| Database migrations | *Example:* Yes, `0047_add_order_channel` (backward compatible, additive) |
| Feature flags | *Example:* `new-checkout-flow` ships OFF; enabled separately |
| Configuration changes | *Example:* New env var `PAYMENT_TIMEOUT_MS=4000` |
| Dependencies | *Example:* Requires payment service v5.2 or later (already deployed) |

### 1.1 Scope of change

[Summarise in two or three sentences what changes for users and for
operations. Example: "Adds the order channel field to the orders table
and API responses. No user-visible change until the feature flag is
enabled."]

## 2. Pre-deployment checklist

<!-- Guidance: Complete every item before the go/no-go meeting. Unchecked items are reasons for no-go. -->

| # | Check | Evidence | Done |
|---|---|---|---|
| 1 | Build pipeline green for the release tag | [Pipeline link] | [ ] |
| 2 | All automated tests passed (unit, integration, end-to-end) | [Report link] | [ ] |
| 3 | Security scans show no new high or critical findings | [Report link] | [ ] |
| 4 | Release deployed to staging and smoke tests passed | [Link] | [ ] |
| 5 | Rollback tested in staging with the same artefact | [Link] | [ ] |
| 6 | Database migration tested on a production-sized copy; duration measured | *Example:* 3 min 40 s | [ ] |
| 7 | Database backup or snapshot completed within the last [1 hour] | [Snapshot ID] | [ ] |
| 8 | Change request approved | [CHG-nnnn] | [ ] |
| 9 | No conflicting changes or freezes in the window | [Change calendar link] | [ ] |
| 10 | On-call engineer and dependent teams informed | [Message link] | [ ] |
| 11 | Dashboards and alerts for the release are ready | [Dashboard link] | [ ] |
| 12 | Customer-facing notice sent if downtime or visible change | [Link or "not needed"] | [ ] |

## 3. Go / no-go decision

<!-- Guidance: A short, explicit decision with named people. Record it; it protects the team if the release goes wrong. -->

Hold the go/no-go check [30 minutes] before the window starts.

| Criterion | Status | Comment |
|---|---|---|
| Pre-deployment checklist complete | [Go / No-go] | [Comment] |
| Production currently healthy (no open SEV1 or SEV2, error budget available) | [Go / No-go] | [Comment] |
| Required people present (deployer, reviewer, on-call, DBA if migration) | [Go / No-go] | [Comment] |
| Rollback path confirmed | [Go / No-go] | [Comment] |

| Decision | Decided by | Time |
|---|---|---|
| [Go / No-go / Postpone to YYYY-MM-DD] | [@release-manager] | [hh:mm] |

## 4. Deployment steps

<!-- Guidance: One action per step, with the exact command, the expected result and the check. Replace example commands with your own tooling. -->

| Step | Action | Command or location | Expected result | Done at |
|---|---|---|---|---|
| 4.1 | Announce start in [channel] | Template 9.1 | Message posted | [hh:mm] |
| 4.2 | Pause automated deploys for the service | [Pipeline setting] | Auto-deploy disabled | [hh:mm] |
| 4.3 | Run database migration | `[migration tool] migrate --target 0047` | Exit code 0; migration recorded in schema table | [hh:mm] |
| 4.4 | Verify migration | `[query to check new column exists]` | Column present; no locks held | [hh:mm] |
| 4.5 | Apply configuration change | `[config tool] set PAYMENT_TIMEOUT_MS=4000` | New config version visible | [hh:mm] |
| 4.6 | Deploy to canary (stage 1 in section 5) | `[deploy tool] deploy checkout-api:2.14.0 --canary 5%` | Canary pods ready; health checks pass | [hh:mm] |
| 4.7 | Observe canary for the hold time | Dashboard in section 6 | Metrics within thresholds | [hh:mm] |
| 4.8 | Promote through remaining stages | `[deploy tool] promote --step` | Each stage healthy before next | [hh:mm] |
| 4.9 | Confirm all instances run the new version | `[command listing running versions]` | 100% on `2.14.0` | [hh:mm] |
| 4.10 | Re-enable automated deploys | [Pipeline setting] | Auto-deploy enabled | [hh:mm] |

## 5. Canary and progressive rollout

<!-- Guidance: Define stages, hold times and the thresholds that block promotion. Automate the analysis if your tooling allows. -->

| Stage | Traffic share | Hold time | Promotion criteria |
|---|---|---|---|
| 1 | 5% | 15 min | Error ratio no worse than baseline +0.1 points; p99 latency within +10% |
| 2 | 25% | 15 min | Same as stage 1; no new error types in logs |
| 3 | 50% | 15 min | Same as stage 1; business metric (orders per minute) within normal range |
| 4 | 100% | n/a | Deployment complete |

```mermaid
flowchart LR
  S0["Deploy canary 5%"] --> C1{"Within thresholds?"}
  C1 -- "Yes" --> S1["Promote to 25%"]
  S1 --> C2{"Within thresholds?"}
  C2 -- "Yes" --> S2["Promote to 50% then 100%"]
  C1 -- "No" --> RB["Roll back (section 7)"]
  C2 -- "No" --> RB
```

If the platform does not support traffic splitting, use [alternative,
for example "one instance of N, then one availability zone, then all"].

## 6. Monitoring during rollout

<!-- Guidance: Name the exact dashboards and signals to watch, with the thresholds that trigger rollback. -->

| Signal | Where | Baseline | Rollback threshold |
|---|---|---|---|
| HTTP 5xx ratio | [Dashboard link] | *Example:* 0.08% | > 0.5% for 5 min |
| p99 latency `POST /orders` | [Dashboard link] | *Example:* 420 ms | > 700 ms for 5 min |
| Orders per minute | [Dashboard link] | *Example:* 180 | Drop > 20% versus same time last week |
| Pod restarts | [Dashboard link] | 0 | Any crash loop |
| New error log signatures | [Log query link] | None | Any unexplained new error |
| Database CPU and locks | [Dashboard link] | *Example:* 45% CPU | > 80% CPU or lock waits > 1 s |

The deployer watches these signals for the full rollout and
[30 minutes] after reaching 100%. The on-call engineer is told when the
watch period ends.

## 7. Rollback procedure

<!-- Guidance: The rollback must be faster and simpler than the deployment. Note any step that cannot be reversed, such as destructive migrations. -->

**Decision rule:** roll back immediately if any threshold in section 6
is crossed, or if the deployer or reviewer is unsure the release is
healthy. Do not debug forward in production during the window.

| Step | Action | Command or location | Expected result | Done at |
|---|---|---|---|---|
| 7.1 | Announce rollback in [channel] | Template 9.3 | Message posted | [hh:mm] |
| 7.2 | Route all traffic to previous version | `[deploy tool] rollback checkout-api --to 2.13.2` | 100% on `2.13.2` | [hh:mm] |
| 7.3 | Revert configuration change | `[config tool] rollback PAYMENT_TIMEOUT_MS` | Previous config version active | [hh:mm] |
| 7.4 | Decide on migration | Additive migration `0047` stays in place (old code ignores the column) | Documented in log | [hh:mm] |
| 7.5 | Verify recovery | Signals in section 6 back to baseline | Baseline within 10 min | [hh:mm] |
| 7.6 | Open incident if users were affected | Incident process | Incident ID recorded | [hh:mm] |

Irreversible steps in this release: [list, or "none"]. If a destructive
migration is unavoidable, split it into expand and contract releases so
that each release can be rolled back independently.

## 8. Post-deployment checks

<!-- Guidance: Confirm the release works for users, not only that pods are running. -->

| # | Check | Method | Result |
|---|---|---|---|
| 1 | Smoke tests in production pass | `[smoke test command]` | [Pass / Fail] |
| 2 | Key user journeys work | *Example:* Place an order with the test account | [Pass / Fail] |
| 3 | No increase in support tickets | Support queue | [Result] |
| 4 | Release recorded in deploy log and change request closed | [CHG-nnnn] | [Done] |
| 5 | Release notes published | [Link] | [Done] |
| 6 | Feature flag rollout plan scheduled (if applicable) | [Ticket] | [Done] |

## 9. Communication

<!-- Guidance: Short, predictable messages at start, end and on rollback. Use the same channel each time. -->

| Audience | Channel | When |
|---|---|---|
| Engineering and on-call | [#deployments] | Start, each stage, end |
| Support team | [#support] | Before start if users may notice; at end |
| Customers | [Status page / email] | Only for downtime or visible change |

### 9.1 Start

> Deploying [service] [version] to production now. Change [CHG-nnnn].
> Expected completion [hh:mm timezone]. Deployer @[name].

### 9.2 Completed

> [Service] [version] is fully deployed and healthy. Watch period ends at
> [hh:mm]. Release notes: [link].

### 9.3 Rolled back

> [Service] [version] has been rolled back to [previous version] because
> [short reason]. Service is [healthy / recovering]. Follow-up in [ticket].

## 10. Deployment log and sign-off

<!-- Guidance: Capture what actually happened, including deviations from the plan. -->

| Time | Event | By |
|---|---|---|
| [hh:mm] | *Example:* Go decision recorded | [@name] |
| [hh:mm] | *Example:* Canary at 5%, error ratio 0.07% | [@name] |
| [hh:mm] | [Event] | [@name] |

| Outcome | Signed off by | Date |
|---|---|---|
| [Completed / Rolled back / Partially deployed] | [@release-manager] | YYYY-MM-DD |

Based on common release engineering and progressive delivery practice.
Original text written for this collection; commands are illustrative.
Template licence: MIT.