Markdown Editor

Bộ mẫu SRE và vận hành

SLO và error budget, observability, sẵn sàng production, runbook, FMEA, thử nghiệm chaos, khôi phục thảm họa, ứng phó sự cố và postmortem.

Bước 1

Mục tiêu và quan sát

  1. SLO Document Template

    Đầy đủ

    Service SLO document: user journeys, SLI specs and queries, targets, error budget math, multiwindow burn-rate alerts, review

    Google SRE conventions
  2. Error Budget Policy Template

    Chuẩn

    Error budget policy: SLOs covered, budget and burn-rate maths, thresholds, feature freeze, escalation, exceptions, sign-off

  3. Observability Specification Template (OpenTelemetry)

    Đầy đủ

    Observability spec: logs, metrics and traces standards, semantic conventions, log schema, RED/USE catalogue, sampling, alerts, privacy, cost

    OpenTelemetry concepts
  4. Capacity Plan Template

    Đầy đủ

    Capacity plan: demand forecast with growth model, utilisation, headroom policy, resource models, bottlenecks, scaling, lead times, cost

Bước 2

Sẵn sàng production

  1. Production Readiness Review Template

    Đầy đủ

    Production readiness review: SLOs, load test evidence, observability, on-call, rollback, backups, security, failure modes, sign-off

    Google SRE PRR conventions
  2. Runbook Template

    Chuẩn

    Service runbook: overview, dependencies, dashboards, per-alert diagnosis and mitigation with commands, common tasks, rollback, contacts

  3. Deployment and Release Runbook Template

    Chuẩn

    Deployment runbook: release metadata, pre-deploy checklist, go/no-go, steps with verification, canary rollout, rollback, comms

  4. On-call Handbook Template

    Chuẩn

    On-call handbook: rotation and handover, severity matrix, response targets, escalation, first 15 minutes, comms templates, wellbeing

Bước 3

Khả năng phục hồi

  1. Software FMEA Template (IEC 60812)

    Đầy đủ

    Software failure modes, effects, causes, controls, S/O/D scales, RPN and action priority, actions and re-scoring

    IEC 60812:2018
  2. Chaos Experiment Plan Template

    Chuẩn

    Chaos engineering experiment: steady state, hypothesis, faults, blast radius, abort conditions, runbook, results, actions

    Principles of Chaos Engineering
  3. Game Day Plan Template

    Chuẩn

    Game day exercise: objectives, scenarios, roles, schedule, safety controls, comms, detection and response scoring, debrief

  4. IT Disaster Recovery Plan Template (NIST SP 800-34)

    Đầy đủ

    IT disaster recovery plan: roles, contact tree, BIA with RTO/RPO/MTD, activation flow, recovery procedures, reconstitution, exercises

    NIST SP 800-34 Rev. 1
  5. Backup and Restore Procedure Template

    Chuẩn

    Backup and restore procedure: data inventory, policy table with 3-2-1 check, jobs, monitoring, step-by-step restore, test evidence log

Bước 4

Sự cố

  1. Incident Response Plan Template

    Chuẩn

    IR phases, severity matrix, comms

  2. Incident Communication Template

    Chuẩn

    Incident comms plan: audiences, severity matrix, roles, cadence, status page and customer messages, executive brief

  3. Incident Postmortem Template

    Đầy đủ

    Blameless incident postmortem: impact and error budget consumed, UTC timeline, root cause, 5 Whys, action items, lessons

    Google SRE postmortem practice (concepts)