# Post-Implementation Review: [Project or service name]

> Post-Implementation Review (PIR) of one delivered project or service,
> held once the solution has run in production long enough to show real
> results, typically 6 to 12 months after handover to the operating
> owner. It compares objectives with outcomes, measures benefits against
> the business case, revisits cost and schedule, checks service
> performance, non-functional requirements in production and user
> adoption, examines architecture conformance and drift, security and
> supplier performance, and ends with lessons, recommendations and the
> date of the next review. The timing and its use as input to an external
> benefits review follow the Gateway Gate 5 (Benefits realisation)
> workbook published by Queensland Treasury (2020). Replace every
> `[bracketed]` value and every row marked *Example*.

<!--
Guidance: How to use this template
- A PIR is an evaluation, not a closure checklist. The project closure
  report (`project/project-closure-report`) records what was handed over
  at the end of delivery; the PIR comes months later and asks whether the
  investment is doing what it was meant to do.
- Hold the review when there is enough operating data to measure the
  main benefits: a full business cycle where seasonality matters. A
  review held too early measures hypercare, not steady state.
- Fill in this order: 2 (scope, timing, questions) -> 3 to 8 (evidence
  per topic) -> 9 to 12 (architecture, security, supplier, issues) -> 13
  (lessons) -> 14 (recommendations) -> 15 (conclusion) -> 1 (summary,
  written last).
- Reuse the measures already agreed: KPIs from the architecture vision
  (`architecture/togaf-architecture-vision`), benefits from the benefits
  realisation plan (`project/benefits-realization-plan`) and requirements
  from the contract (`architecture/architecture-contract`). A PIR that
  invents new measures cannot say whether the original promise was kept.
- The worked example follows the fictional insurer's customer data hub
  (WP-02), reviewed nine months after go-live.
-->

## Document control

<!-- Guidance: Identify the review, the service and the decisions it informs. Name the sponsor who commissioned it and the operating owner who now runs the service. -->

| Field | Value |
|---|---|
| Document ID | [PIR-YYYY-NN] |
| Project or service | [Name, project ID] |
| Business case reference | [BC-ID, version approved] |
| Go-live date | YYYY-MM-DD |
| Handover to operating owner | YYYY-MM-DD |
| Review period covered | YYYY-MM-DD to YYYY-MM-DD |
| Version | [0.1] |
| Status | [Draft / In review / Approved] |
| Owner (review lead) | [@review-lead] |
| Commissioned by | [@sponsor] |
| Operating owner | [@service-owner] |
| Approvers / reviewers | [Sponsor, service owner, finance partner, architecture board delegate] |
| Classification | [Internal / Confidential] |
| Last updated | YYYY-MM-DD |

### Approvals

| Role | Name | Decision | Conditions | Date |
|---|---|---|---|---|
| Sponsor | [@sponsor] | [Accepted / Accepted with actions / Not accepted] | [Conditions] | YYYY-MM-DD |
| Operating owner | [@service-owner] | [Accepted] | [Conditions] | YYYY-MM-DD |
| Finance partner | [@finance] | [Figures agreed] | [Conditions] | YYYY-MM-DD |
| Architecture Board delegate | [@architect] | [Accepted] | [Conditions] | YYYY-MM-DD |
| Portfolio or PMO lead | [@pmo] | [Accepted] | [Conditions] | YYYY-MM-DD |

### Revision history

| Version | Date | Author | Change summary |
|---|---|---|---|
| 0.1 | YYYY-MM-DD | [@name] | Terms of reference, review questions and data requests |
| 0.5 | YYYY-MM-DD | [@name] | *Example:* Evidence collected; interviews with 14 stakeholders; draft findings |
| 1.0 | YYYY-MM-DD | [@name] | *Example:* Accepted by sponsor with actions PIR-ACT-01 to PIR-ACT-07 |

## Contents

1. [Review summary](#1-review-summary)
2. [Scope, timing and method](#2-scope-timing-and-method)
3. [Objectives versus outcomes](#3-objectives-versus-outcomes)
4. [Benefits realised](#4-benefits-realised)
5. [Business case, cost and schedule revisited](#5-business-case-cost-and-schedule-revisited)
6. [Service performance and operations](#6-service-performance-and-operations)
7. [Non-functional requirements in production](#7-non-functional-requirements-in-production)
8. [User adoption and satisfaction](#8-user-adoption-and-satisfaction)
9. [Architecture conformance and drift](#9-architecture-conformance-and-drift)
10. [Security, risk and compliance posture](#10-security-risk-and-compliance-posture)
11. [Supplier and contract performance](#11-supplier-and-contract-performance)
12. [Issues, incidents and residual defects](#12-issues-incidents-and-residual-defects)
13. [Lessons learned](#13-lessons-learned)
14. [Recommendations and action plan](#14-recommendations-and-action-plan)
15. [Conclusion and next review](#15-conclusion-and-next-review)
16. [Glossary and references](#16-glossary-and-references)

## 1. Review summary

<!-- Guidance: Give the verdict first: is the investment delivering, partly delivering or not delivering, and is it still justified? Then the three findings that matter most and the decisions requested. Write it last. Pitfall: a summary that celebrates delivery and stays silent on benefits that are behind. -->

| Item | Result |
|---|---|
| Overall verdict | *Example:* Delivering, with two benefits behind target |
| Business case still valid? | *Example:* Yes; payback moves from 3.0 to 3.6 years |
| Benefits on or above target | *Example:* 1 of 4 on target, 1 within 7% of target, 2 behind |
| Capital cost against approved budget | *Example:* USD 1,340,000 against USD 1,200,000 (+11.7%) |
| Run cost against estimate (annualised) | *Example:* USD 214,000 against USD 180,000 (+18.9%) |
| Service availability against SLO | *Example:* 99.86% against 99.9% (one major incident) |
| Architecture conformance | *Example:* Conformant except one dispensation near expiry with no funded exit and one unauthorised direct database read |
| Recommendations and actions | *Example:* 5 recommendations (3 high priority) and 7 actions |
| Decisions requested | [Accept findings; fund recommendation PIR-REC-01; confirm next review date] |

*Example narrative:* The hub achieved its core data goal: duplicate
customer records fell from 15.2% to 1.8%, and handlers spend about half
as long finding customer details. Two benefits are behind: golden ID
coverage is at 96.4% against 98%, because policy system C data needs
more cleansing, and handler lookup time is 3.4 minutes against 3.0.
Only three of five planned systems consume the customer API. Run cost
is higher than estimated, mainly from matching compute. The interim CRM export (DSP-2026-09) expires in two months with
the CRM still running; the sponsor is asked to decide its retirement
funding now.

## 2. Scope, timing and method

<!-- Guidance: Set the terms of reference: what is reviewed, the questions to answer, the evidence and who reviews. Independence matters: the people who delivered the project contribute evidence but should not judge it. Pitfall: a review that only interviews the project team. -->

### 2.1 Scope

| Dimension | In scope | Out of scope |
|---|---|---|
| Solution | *Example:* Customer data hub, matching service, customer API, events, initial load | Claims platform (separate PIR) |
| Organisation | *Example:* Customer operations, claims handlers, data stewards, IT operations | Sales channel partners |
| Time | *Example:* Go-live YYYY-MM-DD to review date (nine months) | Delivery-phase performance beyond schedule and cost totals |
| Topics | Objectives, benefits, cost, service, NFRs, adoption, architecture, security, supplier, issues, lessons | Individual performance |

### 2.2 Timing

Hold the first PIR once the main benefits can be measured, usually 6 to
12 months after handover to the operating owner. Repeat it for
long-lived services where benefits build up over years, and plan a
final review before a major contract ends or the service is replaced.
Where the organisation uses an external benefits realisation review
(Gate 5 Benefits realisation in Queensland; Gate Review 5, Operations
Review and Benefits Realisation, in the UK NISTA process), the
internal PIR is a main input to it, so schedule the PIR first.

```mermaid
flowchart LR
  GL["Go-live"] --> HC["Hypercare<br/>4 to 8 weeks"]
  HC --> HO["Handover to<br/>operating owner"]
  HO --> SS["Steady state<br/>one business cycle"]
  SS --> PIR["Post-implementation<br/>review"]
  PIR --> EXT["External benefits review<br/>(where used)"]
  PIR --> NEXT["Next PIR or<br/>benefits check"]
```

### 2.3 Review questions

| Question ID | Question | Answered in |
|---|---|---|
| RQ-01 | Were the stated objectives met, and if not, why not? | Section 3 |
| RQ-02 | Are the benefits in the business case being realised, on what trajectory, and can they be attributed to this project? | Section 4 |
| RQ-03 | Is the business case still valid given actual cost, schedule and benefits? | Section 5 |
| RQ-04 | Does the service meet its service levels and non-functional requirements in production? | Sections 6 and 7 |
| RQ-05 | Are the intended users using it as intended? | Section 8 |
| RQ-06 | Does the realised architecture match what was approved, and what debt was taken on? | Section 9 |
| RQ-07 | Are security, privacy and compliance obligations being met in operation? | Section 10 |
| RQ-08 | Did suppliers deliver what they contracted for, and is the ongoing contract working? | Section 11 |
| RQ-09 | What should the organisation repeat or avoid next time? | Section 13 |
| [RQ-nn] | [Question] | [Section] |

### 2.4 Method and evidence

| Method | Source | Used for |
|---|---|---|
| Measurement against baseline | *Example:* Data quality dashboard, claims system time stamps, monitoring | RQ-01, RQ-02, RQ-04 |
| Financial reconciliation | *Example:* Ledger extract, cloud billing, supplier invoices | RQ-03, RQ-08 |
| Document review | *Example:* Business case, benefits plan, contract AC-02, compliance assessments, closure report | All |
| Interviews | *Example:* 14 people: sponsor, owners, stewards, 6 handlers, operations, supplier | RQ-05, RQ-09 |
| User survey | *Example:* 48 respondents, 5-point scale | RQ-05 |
| Architecture walk-through | *Example:* As-built models compared with production inventory | RQ-06 |

Attribution: say how you separate this project's effect from other
changes in the same period (a comparison group, a before and after trend
with other causes noted, or the owner's reasoned estimate). State the
confidence of each benefit figure in section 4.

### 2.5 Review team

| Role | Name | Independent of delivery? |
|---|---|---|
| Review lead | [@review-lead] | *Example:* Yes, portfolio office |
| Finance reviewer | [@finance] | Yes |
| Architecture reviewer | [@architect] | *Example:* Yes, enterprise architecture |
| Business reviewer | [@business-reviewer] | *Example:* Yes, from another business unit |
| Project contributors | [@project-manager], [@solution-architect] | No: provide evidence only |

## 3. Objectives versus outcomes

<!-- Guidance: List each objective as approved, the outcome observed, and a plain status. Explain every gap with a cause, not an excuse. Pitfall: re-wording objectives after the fact so that they appear met. -->

| Objective ID | Objective as approved | Outcome observed | Status | Cause of any gap |
|---|---|---|---|---|
| OBJ-01 *(Example)* | One golden customer record per person across all policy lines | 96.4% of customers have a golden ID; target 98% | Partly met | Policy system C has duplicates the rules cannot resolve safely |
| OBJ-02 *(Example)* | Reduce duplicate customer records below 2% | 1.8% | Met | |
| OBJ-03 *(Example)* | Customer data served to consuming systems through published APIs | 3 of 5 planned consumers live | Partly met | Two consumers deprioritised by their own programmes |
| OBJ-04 *(Example)* | Retire the legacy CRM customer store | Still running; read-only for customer fields | Not met | Retirement work package not funded |
| OBJ-05 *(Example)* | Meet data protection retention rules for all customer data | Retention jobs run monthly; zero records past retention in last audit | Met | |
| [OBJ-nn] | [Objective] | [Outcome] | [Met / Partly met / Not met] | [Cause] |

## 4. Benefits realised

<!-- Guidance: Measure each benefit with the method, baseline and owner agreed in the benefits plan. Show target, actual, trajectory and confidence. Include dis-benefits that appeared. Pitfall: claiming benefits that would have happened anyway, or counting the same saving twice. -->

### 4.1 Benefits register status

| Benefit ID | Benefit | Measure | Baseline | Target (by date) | Actual (date) | Trajectory | Confidence | Owner |
|---|---|---|---|---|---|---|---|---|
| BEN-01 *(Example)* | Fewer duplicate customer records | Duplicate rate | 15.2% | Below 2% (month 6) | 1.8% (month 9) | Stable | High: measured | [@data-owner] |
| BEN-02 *(Example)* | Faster customer lookup for handlers | Median minutes to find customer and cover per claim | 6.5 | 3.0 (month 9) | 3.4 (month 9) | Improving | Medium: sampled time stamps | [@head-of-claims] |
| BEN-03 *(Example)* | Golden ID coverage | Share of customers with a golden ID | 0% | 98% (month 6) | 96.4% (month 9) | Flat since month 6 | High: measured | [@data-owner] |
| BEN-04 *(Example)* | Lower data correction effort | Steward hours per month on manual fixes | 420 | 150 (month 9) | 160 (month 9) | Improving | Medium: timesheets | [@customer-ops-lead] |
| [BEN-nn] | [Benefit] | [Measure] | [Baseline] | [Target, date] | [Actual, date] | [Improving / Stable / Declining] | [High / Medium / Low] | [@owner] |

*Example:* BEN-01 is on target. BEN-02, BEN-03 and BEN-04 are behind:
BEN-04 misses its target by 10 hours a month and is treated as near
target; BEN-02 and BEN-03 are the two benefits reported as behind in
section 1.

### 4.2 Dis-benefits and unintended effects

| ID | Effect | Who is affected | Size | Response |
|---|---|---|---|---|
| DIS-01 *(Example)* | Stewards review a queue of uncertain matches daily | Data stewards | About 2 hours per steward per day | Tune thresholds (PIR-REC-03) |
| DIS-02 *(Example)* | Handlers lost free-text notes previously held in the CRM | Claims handlers | Reported by 9 of 48 survey respondents | Migrate notes to the claims platform |
| [DIS-nn] | [Effect] | [Who] | [Size] | [Response] |

### 4.3 Attribution note

*Example:* BEN-02 overlaps with the handler workspace (WP-04), which went
live three months after the hub. Time stamps from the first three months,
before WP-04, show a fall from 6.5 to 4.1 minutes; the review attributes
that part to the hub and the rest to both projects jointly. Report joint
benefits once, at programme level, in the benefits plan.

## 5. Business case, cost and schedule revisited

<!-- Guidance: Compare actual capital and run cost with the approved business case, explain variances, and say whether the case still holds. Finance should agree the figures. Pitfall: reporting capital cost only, when the run cost decides whole-life value. -->

### 5.1 Capital cost

| Cost line | Approved budget (USD) | Actual (USD) | Variance (USD) | Variance | Main cause |
|---|---|---|---|---|---|
| Supplier services *(Example)* | 650,000 | 700,000 | +50,000 | +7.7% | Extra cleansing rules for policy system C |
| Internal staff *(Example)* | 380,000 | 450,000 | +70,000 | +18.4% | Seven-week delay |
| Software licences *(Example)* | 120,000 | 120,000 | 0 | 0.0% | |
| Cloud during build *(Example)* | 50,000 | 70,000 | +20,000 | +40.0% | Repeated full-volume test loads |
| **Total** *(Example)* | **1,200,000** | **1,340,000** | **+140,000** | **+11.7%** | |

### 5.2 Run cost

| Cost line | Estimate per year (USD) | Actual, annualised (USD) | Variance | Main cause |
|---|---|---|---|---|
| Cloud compute and storage *(Example)* | 70,000 | 98,000 | +40.0% | Matching runs on every change, not in batches |
| Licences and support *(Example)* | 60,000 | 60,000 | 0.0% | |
| Operations staff share *(Example)* | 50,000 | 56,000 | +12.0% | Steward review queue |
| **Total** *(Example)* | **180,000** | **214,000** | **+18.9%** | |

Run cost trends and optimisation options belong in the FinOps review
(`ops/finops-cost-review`).

### 5.3 Schedule

| Milestone | Planned | Actual | Variance | Cause |
|---|---|---|---|---|
| Design approved *(Example)* | YYYY-MM-DD | YYYY-MM-DD | +2 weeks | Retention design late (condition C2) |
| Go-live *(Example)* | YYYY-MM-DD | YYYY-MM-DD | +7 weeks | Policy system C data cleansing |
| Hypercare exit *(Example)* | YYYY-MM-DD | YYYY-MM-DD | +7 weeks | Followed go-live |
| [Milestone] | YYYY-MM-DD | YYYY-MM-DD | [Variance] | [Cause] |

### 5.4 Is the business case still valid?

*Example:* Yes. With the actual capital cost, actual run cost and
benefits on their current trajectory, the payback period moves from 3.0
to 3.6 years against a hurdle of 5 years. The case would fail only if
golden ID coverage stayed below 95% and the CRM were never retired,
because the CRM licence saving is a third of the annual benefit. State
the recalculation method and who agreed it: [@finance], YYYY-MM-DD.

## 6. Service performance and operations

<!-- Guidance: Report service levels for the whole period, not the best month, and how the operating team is coping. Include the hypercare exit and whether the operating owner accepted the service with open items. Pitfall: using availability alone; a service can be up and still failing its users. -->

| Measure | Target | Actual (period) | Status |
|---|---|---|---|
| Customer API availability *(Example)* | 99.9% per month | 99.86% over nine months; one month at 99.4% | Below in one month |
| Matching job success rate *(Example)* | 99.5% | 99.8% | Met |
| P1 incidents *(Example)* | No target | 1 | Reviewed in section 12 |
| P2 incidents *(Example)* | No target | 4 | Reviewed in section 12 |
| Mean time to restore, P1 and P2 *(Example)* | Under 4 hours | 2 hours 35 minutes | Met |
| Change failure rate *(Example)* | Under 15% | 9% | Met |
| Support tickets per month *(Example)* | Under 60 after hypercare | 41 average | Met |
| [Measure] | [Target] | [Actual] | [Status] |

| Operational topic | Observation |
|---|---|
| Hypercare exit *(Example)* | Exited after six weeks with two open known errors, accepted by the service owner |
| Runbooks *(Example)* | Complete for matching and API; missing for dead-letter replay, used twice from memory |
| On-call *(Example)* | Shared with the integration team; two engineers trained; single point of knowledge on matching rules |
| Monitoring *(Example)* | Dashboards in place; matching latency now measured (FND-06 from the design assessment closed) |
| [Topic] | [Observation] |

Use the operations handover record (`ops/operations-handover`) and the
hypercare plan (`ops/hypercare-plan`) as the starting evidence.

## 7. Non-functional requirements in production

<!-- Guidance: Compare each contracted non-functional requirement with production evidence, not test results. Note where the load profile differs from the design assumption, because that predicts future trouble. Pitfall: declaring an NFR met because no one complained. -->

| NFR ID | Requirement | Target | Production evidence | Status | Note |
|---|---|---|---|---|---|
| NFR-PERF-01 *(Example)* | Customer API latency, p95 | At most 300 ms | 240 ms (monthly average of p95) | Met | |
| NFR-PERF-02 *(Example)* | Event delivery latency, p95 | At most 5 s | 3.1 s | Met | |
| NFR-CAP-01 *(Example)* | Peak API requests per second | 400 design capacity | Peak observed 310 | Met | 78% of design capacity after nine months; plan capacity review |
| NFR-REL-01 *(Example)* | Recovery time objective | 4 hours | Restore test: 2 hours 50 minutes | Met | |
| NFR-REL-02 *(Example)* | Recovery point objective | 15 minutes | Restore test: 12 minutes of data lost | Met | |
| NFR-SEC-01 *(Example)* | Personal data past retention | 0 records | 0 in last two monthly audits | Met | |
| NFR-OPS-01 *(Example)* | Matching throughput for daily changes | 200,000 per hour | 150,000 per hour at peak | Not met | Causes the run cost increase in 5.2; see PIR-REC-04 |
| [NFR-nn] | [Requirement] | [Target] | [Evidence] | [Met / Not met] | [Note] |

Where NFRs need tuning, update the quality attribute record
(`requirements/quality-attributes`) and the capacity plan
(`ops/capacity-plan`).

## 8. User adoption and satisfaction

<!-- Guidance: Measure whether the intended users and systems actually use the solution, and how they feel about it. Combine usage data with survey and interview evidence. Pitfall: counting log-ins rather than completed tasks. -->

| Measure | Target | Actual | Source |
|---|---|---|---|
| Data stewards actively working the queue *(Example)* | 12 of 12 | 12 of 12 | Stewardship audit log |
| Handlers using hub lookup rather than CRM *(Example)* | 90% of lookups | 83% | API logs versus CRM read logs |
| Consuming systems live on the customer API *(Example)* | 5 | 3 | Gateway inventory |
| User satisfaction, handlers *(Example)* | At least 3.5 of 5 | 3.7 of 5 (48 respondents) | Survey |
| Training completion *(Example)* | 100% of stewards, 90% of handlers | 100%, 94% | Learning system |
| [Measure] | [Target] | [Actual] | [Source] |

Main themes from interviews and free text: *Example:* lookups are faster
and more trusted; uncertain-match queue is tiring; handlers still open
the CRM for old notes (DIS-02).

## 9. Architecture conformance and drift

<!-- Guidance: Compare the realised architecture with what the board approved, close out the Architecture Contract, and record debt taken on. Drift is any difference between the as-built inventory and the approved models, whether or not it was authorised. Pitfall: reviewing documents only; inspect the production inventory, network flows and access lists. -->

### 9.1 Contract and conditions

| Item | Status |
|---|---|
| Architecture Contract | *Example:* AC-02 version 1.1; ready to close except ACC-03 (active dispensation) |
| Design assessment conditions | *Example:* C1 (full-volume matching test) and C2 (retention rules) verified before go-live |
| Pre-production assessment | *Example:* ARB-2026-047 compliant; no open conditions |
| Active dispensations | *Example:* DSP-2026-09 (CRM batch export), renewed once, expires in two months; exit plan unfunded |

### 9.2 Drift found

| Drift ID | Approved design | As built | Authorised? | Risk | Treatment |
|---|---|---|---|---|---|
| DRF-01 *(Example)* | Reporting reads customer data through the API or events | Reporting team reads a database read replica directly | No | Bypasses access control and couples reporting to the schema | Replace with an event feed into the data platform, or seek a dispensation (PIR-REC-02) |
| DRF-02 *(Example)* | Matching runs in nightly batches for bulk changes | Matching runs on every change | Yes, decision record ADR-021 | Higher run cost | Re-evaluate with cost data (PIR-REC-04) |
| [DRF-nn] | [Approved] | [As built] | [Yes / No] | [Risk] | [Treatment] |

### 9.3 Architecture debt

| Debt ID | Description | Interest (ongoing cost) | Register entry |
|---|---|---|---|
| TD-031 *(Example)* | Legacy CRM batch export | Duplicate data path; licence cost | `architecture/tech-debt-register` |
| TD-032 *(Example)* | Direct replica read by reporting | Schema changes break reports | `architecture/tech-debt-register` |
| [TD-nn] | [Description] | [Interest] | [Entry] |

For a full conformance check of the as-built system, use the Architecture
Compliance Assessment (`architecture/architecture-compliance-assessment`)
at the post-implementation review point.

## 10. Security, risk and compliance posture

<!-- Guidance: Check that the controls designed are operating, that risks accepted at go-live are still acceptable, and that regulatory obligations are evidenced. Pitfall: relying on the go-live security sign-off; controls degrade after hypercare. -->

| Area | Expected | Observed | Status |
|---|---|---|---|
| Access reviews *(Example)* | Quarterly review of steward and admin roles | Two reviews done; three leavers removed late (after 9 days) | Partly effective |
| Vulnerability management *(Example)* | Critical findings fixed within 14 days | All met; one high open 30 days with exception | Effective |
| Data protection *(Example)* | DPIA actions closed; retention enforced | DPIA actions closed; NFR-SEC-01 met | Effective |
| Logging *(Example)* | Security events in central monitoring | Confirmed by test alert | Effective |
| Accepted risks *(Example)* | Reviewed at the date set at go-live | One accepted risk past its review date | Action PIR-ACT-05 |
| [Area] | [Expected] | [Observed] | [Status] |

Update the risk register (`compliance/risk-register`) with any change in
residual risk.

## 11. Supplier and contract performance

<!-- Guidance: Assess the supplier against the delivery contract and the ongoing support or licence contract, and check that the client side can still manage the contract. Pitfall: mixing personal opinion of the supplier with contract evidence. -->

| Topic | Contracted | Observed | Assessment |
|---|---|---|---|
| Delivery scope *(Example)* | SC-01 to SC-05 | All delivered | Met |
| Delivery schedule *(Example)* | Go-live YYYY-MM-DD | Seven weeks late; shared cause (data quality) | Partly met |
| Support response *(Example)* | P1 response within 1 hour | Met for the one P1 | Met |
| Knowledge transfer *(Example)* | Internal team able to change matching rules | One engineer able; second in training | Partly met |
| Contract management *(Example)* | Named contract manager on client side | Role vacant for two months after go-live | Gap |
| Value for money *(Example)* | Price review at month 12 | Due; benchmark recommended | Action PIR-ACT-06 |
| [Topic] | [Contracted] | [Observed] | [Assessment] |

## 12. Issues, incidents and residual defects

<!-- Guidance: Summarise significant incidents and open defects, their root causes and whether the fixes held. Link postmortems rather than repeating them. Pitfall: listing incidents without asking which design or process weakness they reveal. -->

| ID | Type | Date | Summary | Root cause | Fix | Fix held? |
|---|---|---|---|---|---|---|
| INC-0412 *(Example)* | P1 incident | YYYY-MM-DD | Customer API unavailable for 2 hours 40 minutes | Certificate expired on the gateway route | Automated renewal and expiry alert | Yes |
| INC-0388 *(Example)* | P2 incident | YYYY-MM-DD | Events delayed 3 hours | Dead-letter queue filled; replay done manually | Replay runbook (PIR-ACT-02) | Pending |
| KE-07 *(Example)* | Known error | Since go-live | Merge of two households loses one contact preference | Survivorship rule order | Fix in release 1.6 | Not yet |
| [ID] | [Incident / Problem / Known error / Defect] | YYYY-MM-DD | [Summary] | [Cause] | [Fix] | [Yes / No / Pending] |

Postmortems are in [location] (`dev/postmortem`).

## 13. Lessons learned

<!-- Guidance: Draw lessons from the whole life cycle, including what went well. A lesson states what happened, why, and what the organisation will do differently, with an owner. Record them in the lessons register so other projects can find them. Pitfall: lessons that are complaints with no change attached. -->

| Lesson ID | What happened | Why | Do differently next time | Applies to | Owner |
|---|---|---|---|---|---|
| LL-01 *(Example)* | Data cleansing for policy system C delayed go-live by seven weeks | Data profiling was done on a sample only | Profile full source data before committing a go-live date | All data migration projects | [@pmo] |
| LL-02 *(Example)* | Run cost 19% above estimate | Cost model assumed batch matching; design changed without re-costing | Re-cost when a decision record changes a cost driver | Architecture review process | [@architect] |
| LL-03 *(Example)* | Retention design was late and became a board condition | Privacy requirements were added after design started | Include retention design in the first design deliverable | Contract template | [@contract-architect] |
| LL-04 *(Example, positive)* | Stewards were productive from day one | Stewards co-designed the queue and trained early | Repeat early co-design with operational users | All projects with new operational roles | [@change-lead] |
| [LL-nn] | [What happened] | [Why] | [Change] | [Scope] | [@owner] |

Record and share them in the lessons learned register
(`project/lessons-learned`).

## 14. Recommendations and action plan

<!-- Guidance: Turn findings into a short list of recommendations with a priority, owner, cost and date, and say which need a funding decision. Separate recommendations (need a decision) from actions (owners can just do them). Pitfall: twenty recommendations with no priority, which guarantees none are funded. -->

### 14.1 Recommendations

| Rec ID | Recommendation | Finding | Priority | Indicative cost | Decision needed from | Due |
|---|---|---|---|---|---|---|
| PIR-REC-01 *(Example)* | Fund the legacy CRM retirement work package so DSP-2026-09 can close | OBJ-04, 9.1 | High | USD [amount] | Sponsor | YYYY-MM-DD |
| PIR-REC-02 *(Example)* | Replace the direct replica read with an event feed to the data platform | DRF-01 | High | USD [amount] | Architecture Board | YYYY-MM-DD |
| PIR-REC-03 *(Example)* | Tune match thresholds and add a second steward review shift at month-end | DIS-01, BEN-03 | High | Internal effort | Service owner | YYYY-MM-DD |
| PIR-REC-04 *(Example)* | Re-evaluate on-change matching against micro-batch with measured costs | DRF-02, NFR-OPS-01 | Medium | Internal effort | Architecture Board | YYYY-MM-DD |
| PIR-REC-05 *(Example)* | Re-engage the two deferred API consumers through the portfolio board | OBJ-03 | Medium | None | Portfolio board | YYYY-MM-DD |
| [PIR-REC-nn] | [Recommendation] | [Ref] | [High / Medium / Low] | [Cost] | [Who] | YYYY-MM-DD |

### 14.2 Actions

| Action ID | Action | Owner | Due | Status |
|---|---|---|---|---|
| PIR-ACT-01 *(Example)* | Close AC-02 once DSP-2026-09 is resolved | [@contract-architect] | YYYY-MM-DD | Open |
| PIR-ACT-02 *(Example)* | Write and test the dead-letter replay runbook | [@sre-lead] | YYYY-MM-DD | Open |
| PIR-ACT-03 *(Example)* | Train a second engineer on matching rules | [@service-owner] | YYYY-MM-DD | Open |
| PIR-ACT-04 *(Example)* | Fill the client-side contract manager role | [@procurement] | YYYY-MM-DD | Open |
| PIR-ACT-05 *(Example)* | Review the accepted risk past its review date | [@risk-owner] | YYYY-MM-DD | Open |
| PIR-ACT-06 *(Example)* | Benchmark the supplier price before the month 12 review | [@procurement] | YYYY-MM-DD | Open |
| PIR-ACT-07 *(Example)* | Speed up leaver removal from steward roles | [@iam-lead] | YYYY-MM-DD | Open |
| [PIR-ACT-nn] | [Action] | [@owner] | YYYY-MM-DD | [Open / Done] |

## 15. Conclusion and next review

<!-- Guidance: Answer each review question in one line, state the overall verdict, and fix who owns benefits from now on and when the next check happens. Pitfall: a PIR that ends without a next date, so benefits stop being tracked. -->

| Question | Answer |
|---|---|
| RQ-01 Objectives | *Example:* Two of five met, two partly met, one not met |
| RQ-02 Benefits | *Example:* One on target, one near target, two behind; attribution shared with WP-04 for BEN-02 |
| RQ-03 Business case | *Example:* Still valid; payback 3.6 years |
| RQ-04 Service and NFRs | *Example:* Meets service levels except one month; matching throughput below target |
| RQ-05 Adoption | *Example:* Good with stewards and handlers; two system consumers missing |
| RQ-06 Architecture | *Example:* One unauthorised drift, one overdue dispensation exit |
| RQ-07 Security and compliance | *Example:* Controls operating; access removal and one risk review late |
| RQ-08 Supplier | *Example:* Delivered scope; knowledge transfer and contract management gaps |
| RQ-09 Lessons | *Example:* Four lessons recorded, one positive |

| Item | Decision |
|---|---|
| Benefits owner from now on | [@service-owner] |
| Next benefits check | *Example:* Month 15, focused on BEN-02, BEN-03 and the CRM retirement |
| External review | [Not required / Gate 5 review planned YYYY-MM-DD] |
| Periodic architecture health check | *Example:* Annually from month 12 (`architecture/architecture-health-check`) |

## 16. Glossary and references

<!-- Guidance: Define evaluation terms for business readers and list every evidence source with its version. -->

### 16.1 Glossary

| Term | Meaning in this document |
|---|---|
| Post-implementation review | An evaluation held after a solution has run in production, comparing its results with the original objectives and business case |
| Benefit | A measurable improvement that a stakeholder values and that the investment is expected to cause |
| Dis-benefit | A measurable negative effect caused by the change |
| Attribution | The share of an observed change that can reasonably be credited to this project |
| Drift | A difference between the approved architecture and what is running in production |
| Hypercare | The period of heightened support right after go-live |
| Business case | The approved justification for the investment, with costs, benefits and risks |
| Gateway review | An independent assurance review at a fixed point in a project's life; Gate 5 focuses on benefits realisation |
| [Term] | [Meaning] |

### 16.2 References

| ID | Reference | Version / date |
|---|---|---|
| REF-01 | Queensland Treasury, Gateway review process, Gate 5: Benefits realisation (workbook) | 2020 |
| REF-02 | HM Treasury and Evaluation Task Force, The Magenta Book: central government guidance on evaluation | 2020 edition; HTML version added 2026-05-15 |
| REF-03 | Business case (`business/business-case`) | [x.y] |
| REF-04 | Benefits realisation plan (`project/benefits-realization-plan`) | [x.y] |
| REF-05 | Project closure report (`project/project-closure-report`) | [x.y] |
| REF-06 | Architecture Contract and compliance assessments (`architecture/architecture-contract`, `architecture/architecture-compliance-assessment`) | [x.y] |
| REF-07 | Go-live readiness review (`project/go-live-readiness-review`) | [x.y] |
| REF-08 | Lessons learned register (`project/lessons-learned`) | [x.y] |
| REF-09 | National Infrastructure and Service Transformation Authority (NISTA), Gate Review 5: Operations Review and Benefits Realisation, gov.uk | Updated 2026-08-20 |

## Standard and licence note

Based on public-sector assurance and evaluation guidance: the Gateway
review process Gate 5 (Benefits realisation) workbook published by
Queensland Treasury (2020, licensed CC BY-NC-ND 4.0), which places the
first benefits review typically 6 to 12 months after handover and treats
the internal post-implementation review as a main input to the external
review, and HM Treasury's Magenta Book on evaluation design. Only that
timing and the general concepts are used; the structure, review
questions, tables, examples and all text of this template are original,
and no text from either source is reproduced or adapted. The UK NISTA
Gate Review 5 guidance is cited by title and date only. Gateway and OGC
Gateway are trade marks of their owner; in the UK the Gate Review
process is now run by the National Infrastructure and Service
Transformation Authority (NISTA). Check the
current versions at treasury.qld.gov.au and gov.uk. Template licence: MIT.