Markdown Editor

Post-Implementation Review Template

Review questions, objectives versus outcomes, benefits and attribution, cost and schedule, service and NFRs in production, adoption, architecture drift, lessons

Управління проєктамиПовнийQueensland Treasury Gateway Gate 5 workbook (2020); HM Treasury Magenta Bookpostimplementationreview

Як використовувати: Use it 6 to 12 months after handover, once the main benefits can be measured in steady state. Fill the review questions and the original measures from the business case and benefits plan first, then gather production evidence per topic. The main pitfall is inventing new measures after the fact instead of testing the original promise.

Перегляд

Post-Implementation Review: [Project or service name]

Post-Implementation Review (PIR) of one delivered project or service, held once the solution has run in production long enough to show real results, typically 6 to 12 months after handover to the operating owner. It compares objectives with outcomes, measures benefits against the business case, revisits cost and schedule, checks service performance, non-functional requirements in production and user adoption, examines architecture conformance and drift, security and supplier performance, and ends with lessons, recommendations and the date of the next review. The timing and its use as input to an external benefits review follow the Gateway Gate 5 (Benefits realisation) workbook published by Queensland Treasury (2020). Replace every [bracketed] value and every row marked Example.

Document control

FieldValue
Document ID[PIR-YYYY-NN]
Project or service[Name, project ID]
Business case reference[BC-ID, version approved]
Go-live dateYYYY-MM-DD
Handover to operating ownerYYYY-MM-DD
Review period coveredYYYY-MM-DD to YYYY-MM-DD
Version[0.1]
Status[Draft / In review / Approved]
Owner (review lead)[@review-lead]
Commissioned by[@sponsor]
Operating owner[@service-owner]
Approvers / reviewers[Sponsor, service owner, finance partner, architecture board delegate]
Classification[Internal / Confidential]
Last updatedYYYY-MM-DD

Approvals

RoleNameDecisionConditionsDate
Sponsor[@sponsor][Accepted / Accepted with actions / Not accepted][Conditions]YYYY-MM-DD
Operating owner[@service-owner][Accepted][Conditions]YYYY-MM-DD
Finance partner[@finance][Figures agreed][Conditions]YYYY-MM-DD
Architecture Board delegate[@architect][Accepted][Conditions]YYYY-MM-DD
Portfolio or PMO lead[@pmo][Accepted][Conditions]YYYY-MM-DD

Revision history

VersionDateAuthorChange summary
0.1YYYY-MM-DD[@name]Terms of reference, review questions and data requests
0.5YYYY-MM-DD[@name]Example: Evidence collected; interviews with 14 stakeholders; draft findings
1.0YYYY-MM-DD[@name]Example: Accepted by sponsor with actions PIR-ACT-01 to PIR-ACT-07

Contents

1. Review summary

ItemResult
Overall verdictExample: Delivering, with two benefits behind target
Business case still valid?Example: Yes; payback moves from 3.0 to 3.6 years
Benefits on or above targetExample: 1 of 4 on target, 1 within 7% of target, 2 behind
Capital cost against approved budgetExample: USD 1,340,000 against USD 1,200,000 (+11.7%)
Run cost against estimate (annualised)Example: USD 214,000 against USD 180,000 (+18.9%)
Service availability against SLOExample: 99.86% against 99.9% (one major incident)
Architecture conformanceExample: Conformant except one dispensation near expiry with no funded exit and one unauthorised direct database read
Recommendations and actionsExample: 5 recommendations (3 high priority) and 7 actions
Decisions requested[Accept findings; fund recommendation PIR-REC-01; confirm next review date]

Example narrative: The hub achieved its core data goal: duplicate customer records fell from 15.2% to 1.8%, and handlers spend about half as long finding customer details. Two benefits are behind: golden ID coverage is at 96.4% against 98%, because policy system C data needs more cleansing, and handler lookup time is 3.4 minutes against 3.0. Only three of five planned systems consume the customer API. Run cost is higher than estimated, mainly from matching compute. The interim CRM export (DSP-2026-09) expires in two months with the CRM still running; the sponsor is asked to decide its retirement funding now.

2. Scope, timing and method

2.1 Scope

DimensionIn scopeOut of scope
SolutionExample: Customer data hub, matching service, customer API, events, initial loadClaims platform (separate PIR)
OrganisationExample: Customer operations, claims handlers, data stewards, IT operationsSales channel partners
TimeExample: Go-live YYYY-MM-DD to review date (nine months)Delivery-phase performance beyond schedule and cost totals
TopicsObjectives, benefits, cost, service, NFRs, adoption, architecture, security, supplier, issues, lessonsIndividual performance

2.2 Timing

Hold the first PIR once the main benefits can be measured, usually 6 to 12 months after handover to the operating owner. Repeat it for long-lived services where benefits build up over years, and plan a final review before a major contract ends or the service is replaced. Where the organisation uses an external benefits realisation review (Gate 5 Benefits realisation in Queensland; Gate Review 5, Operations Review and Benefits Realisation, in the UK NISTA process), the internal PIR is a main input to it, so schedule the PIR first.

2.3 Review questions

Question IDQuestionAnswered in
RQ-01Were the stated objectives met, and if not, why not?Section 3
RQ-02Are the benefits in the business case being realised, on what trajectory, and can they be attributed to this project?Section 4
RQ-03Is the business case still valid given actual cost, schedule and benefits?Section 5
RQ-04Does the service meet its service levels and non-functional requirements in production?Sections 6 and 7
RQ-05Are the intended users using it as intended?Section 8
RQ-06Does the realised architecture match what was approved, and what debt was taken on?Section 9
RQ-07Are security, privacy and compliance obligations being met in operation?Section 10
RQ-08Did suppliers deliver what they contracted for, and is the ongoing contract working?Section 11
RQ-09What should the organisation repeat or avoid next time?Section 13
[RQ-nn][Question][Section]

2.4 Method and evidence

MethodSourceUsed for
Measurement against baselineExample: Data quality dashboard, claims system time stamps, monitoringRQ-01, RQ-02, RQ-04
Financial reconciliationExample: Ledger extract, cloud billing, supplier invoicesRQ-03, RQ-08
Document reviewExample: Business case, benefits plan, contract AC-02, compliance assessments, closure reportAll
InterviewsExample: 14 people: sponsor, owners, stewards, 6 handlers, operations, supplierRQ-05, RQ-09
User surveyExample: 48 respondents, 5-point scaleRQ-05
Architecture walk-throughExample: As-built models compared with production inventoryRQ-06

Attribution: say how you separate this project's effect from other changes in the same period (a comparison group, a before and after trend with other causes noted, or the owner's reasoned estimate). State the confidence of each benefit figure in section 4.

2.5 Review team

RoleNameIndependent of delivery?
Review lead[@review-lead]Example: Yes, portfolio office
Finance reviewer[@finance]Yes
Architecture reviewer[@architect]Example: Yes, enterprise architecture
Business reviewer[@business-reviewer]Example: Yes, from another business unit
Project contributors[@project-manager], [@solution-architect]No: provide evidence only

3. Objectives versus outcomes

Objective IDObjective as approvedOutcome observedStatusCause of any gap
OBJ-01 (Example)One golden customer record per person across all policy lines96.4% of customers have a golden ID; target 98%Partly metPolicy system C has duplicates the rules cannot resolve safely
OBJ-02 (Example)Reduce duplicate customer records below 2%1.8%Met
OBJ-03 (Example)Customer data served to consuming systems through published APIs3 of 5 planned consumers livePartly metTwo consumers deprioritised by their own programmes
OBJ-04 (Example)Retire the legacy CRM customer storeStill running; read-only for customer fieldsNot metRetirement work package not funded
OBJ-05 (Example)Meet data protection retention rules for all customer dataRetention jobs run monthly; zero records past retention in last auditMet
[OBJ-nn][Objective][Outcome][Met / Partly met / Not met][Cause]

4. Benefits realised

4.1 Benefits register status

Benefit IDBenefitMeasureBaselineTarget (by date)Actual (date)TrajectoryConfidenceOwner
BEN-01 (Example)Fewer duplicate customer recordsDuplicate rate15.2%Below 2% (month 6)1.8% (month 9)StableHigh: measured[@data-owner]
BEN-02 (Example)Faster customer lookup for handlersMedian minutes to find customer and cover per claim6.53.0 (month 9)3.4 (month 9)ImprovingMedium: sampled time stamps[@head-of-claims]
BEN-03 (Example)Golden ID coverageShare of customers with a golden ID0%98% (month 6)96.4% (month 9)Flat since month 6High: measured[@data-owner]
BEN-04 (Example)Lower data correction effortSteward hours per month on manual fixes420150 (month 9)160 (month 9)ImprovingMedium: timesheets[@customer-ops-lead]
[BEN-nn][Benefit][Measure][Baseline][Target, date][Actual, date][Improving / Stable / Declining][High / Medium / Low][@owner]

Example: BEN-01 is on target. BEN-02, BEN-03 and BEN-04 are behind: BEN-04 misses its target by 10 hours a month and is treated as near target; BEN-02 and BEN-03 are the two benefits reported as behind in section 1.

4.2 Dis-benefits and unintended effects

IDEffectWho is affectedSizeResponse
DIS-01 (Example)Stewards review a queue of uncertain matches dailyData stewardsAbout 2 hours per steward per dayTune thresholds (PIR-REC-03)
DIS-02 (Example)Handlers lost free-text notes previously held in the CRMClaims handlersReported by 9 of 48 survey respondentsMigrate notes to the claims platform
[DIS-nn][Effect][Who][Size][Response]

4.3 Attribution note

Example: BEN-02 overlaps with the handler workspace (WP-04), which went live three months after the hub. Time stamps from the first three months, before WP-04, show a fall from 6.5 to 4.1 minutes; the review attributes that part to the hub and the rest to both projects jointly. Report joint benefits once, at programme level, in the benefits plan.

5. Business case, cost and schedule revisited

5.1 Capital cost

Cost lineApproved budget (USD)Actual (USD)Variance (USD)VarianceMain cause
Supplier services (Example)650,000700,000+50,000+7.7%Extra cleansing rules for policy system C
Internal staff (Example)380,000450,000+70,000+18.4%Seven-week delay
Software licences (Example)120,000120,00000.0%
Cloud during build (Example)50,00070,000+20,000+40.0%Repeated full-volume test loads
Total (Example)1,200,0001,340,000+140,000+11.7%

5.2 Run cost

Cost lineEstimate per year (USD)Actual, annualised (USD)VarianceMain cause
Cloud compute and storage (Example)70,00098,000+40.0%Matching runs on every change, not in batches
Licences and support (Example)60,00060,0000.0%
Operations staff share (Example)50,00056,000+12.0%Steward review queue
Total (Example)180,000214,000+18.9%

Run cost trends and optimisation options belong in the FinOps review (ops/finops-cost-review).

5.3 Schedule

MilestonePlannedActualVarianceCause
Design approved (Example)YYYY-MM-DDYYYY-MM-DD+2 weeksRetention design late (condition C2)
Go-live (Example)YYYY-MM-DDYYYY-MM-DD+7 weeksPolicy system C data cleansing
Hypercare exit (Example)YYYY-MM-DDYYYY-MM-DD+7 weeksFollowed go-live
[Milestone]YYYY-MM-DDYYYY-MM-DD[Variance][Cause]

5.4 Is the business case still valid?

Example: Yes. With the actual capital cost, actual run cost and benefits on their current trajectory, the payback period moves from 3.0 to 3.6 years against a hurdle of 5 years. The case would fail only if golden ID coverage stayed below 95% and the CRM were never retired, because the CRM licence saving is a third of the annual benefit. State the recalculation method and who agreed it: [@finance], YYYY-MM-DD.

6. Service performance and operations

MeasureTargetActual (period)Status
Customer API availability (Example)99.9% per month99.86% over nine months; one month at 99.4%Below in one month
Matching job success rate (Example)99.5%99.8%Met
P1 incidents (Example)No target1Reviewed in section 12
P2 incidents (Example)No target4Reviewed in section 12
Mean time to restore, P1 and P2 (Example)Under 4 hours2 hours 35 minutesMet
Change failure rate (Example)Under 15%9%Met
Support tickets per month (Example)Under 60 after hypercare41 averageMet
[Measure][Target][Actual][Status]
Operational topicObservation
Hypercare exit (Example)Exited after six weeks with two open known errors, accepted by the service owner
Runbooks (Example)Complete for matching and API; missing for dead-letter replay, used twice from memory
On-call (Example)Shared with the integration team; two engineers trained; single point of knowledge on matching rules
Monitoring (Example)Dashboards in place; matching latency now measured (FND-06 from the design assessment closed)
[Topic][Observation]

Use the operations handover record (ops/operations-handover) and the hypercare plan (ops/hypercare-plan) as the starting evidence.

7. Non-functional requirements in production

NFR IDRequirementTargetProduction evidenceStatusNote
NFR-PERF-01 (Example)Customer API latency, p95At most 300 ms240 ms (monthly average of p95)Met
NFR-PERF-02 (Example)Event delivery latency, p95At most 5 s3.1 sMet
NFR-CAP-01 (Example)Peak API requests per second400 design capacityPeak observed 310Met78% of design capacity after nine months; plan capacity review
NFR-REL-01 (Example)Recovery time objective4 hoursRestore test: 2 hours 50 minutesMet
NFR-REL-02 (Example)Recovery point objective15 minutesRestore test: 12 minutes of data lostMet
NFR-SEC-01 (Example)Personal data past retention0 records0 in last two monthly auditsMet
NFR-OPS-01 (Example)Matching throughput for daily changes200,000 per hour150,000 per hour at peakNot metCauses the run cost increase in 5.2; see PIR-REC-04
[NFR-nn][Requirement][Target][Evidence][Met / Not met][Note]

Where NFRs need tuning, update the quality attribute record (requirements/quality-attributes) and the capacity plan (ops/capacity-plan).

8. User adoption and satisfaction

MeasureTargetActualSource
Data stewards actively working the queue (Example)12 of 1212 of 12Stewardship audit log
Handlers using hub lookup rather than CRM (Example)90% of lookups83%API logs versus CRM read logs
Consuming systems live on the customer API (Example)53Gateway inventory
User satisfaction, handlers (Example)At least 3.5 of 53.7 of 5 (48 respondents)Survey
Training completion (Example)100% of stewards, 90% of handlers100%, 94%Learning system
[Measure][Target][Actual][Source]

Main themes from interviews and free text: Example: lookups are faster and more trusted; uncertain-match queue is tiring; handlers still open the CRM for old notes (DIS-02).

9. Architecture conformance and drift

9.1 Contract and conditions

ItemStatus
Architecture ContractExample: AC-02 version 1.1; ready to close except ACC-03 (active dispensation)
Design assessment conditionsExample: C1 (full-volume matching test) and C2 (retention rules) verified before go-live
Pre-production assessmentExample: ARB-2026-047 compliant; no open conditions
Active dispensationsExample: DSP-2026-09 (CRM batch export), renewed once, expires in two months; exit plan unfunded

9.2 Drift found

Drift IDApproved designAs builtAuthorised?RiskTreatment
DRF-01 (Example)Reporting reads customer data through the API or eventsReporting team reads a database read replica directlyNoBypasses access control and couples reporting to the schemaReplace with an event feed into the data platform, or seek a dispensation (PIR-REC-02)
DRF-02 (Example)Matching runs in nightly batches for bulk changesMatching runs on every changeYes, decision record ADR-021Higher run costRe-evaluate with cost data (PIR-REC-04)
[DRF-nn][Approved][As built][Yes / No][Risk][Treatment]

9.3 Architecture debt

Debt IDDescriptionInterest (ongoing cost)Register entry
TD-031 (Example)Legacy CRM batch exportDuplicate data path; licence costarchitecture/tech-debt-register
TD-032 (Example)Direct replica read by reportingSchema changes break reportsarchitecture/tech-debt-register
[TD-nn][Description][Interest][Entry]

For a full conformance check of the as-built system, use the Architecture Compliance Assessment (architecture/architecture-compliance-assessment) at the post-implementation review point.

10. Security, risk and compliance posture

AreaExpectedObservedStatus
Access reviews (Example)Quarterly review of steward and admin rolesTwo reviews done; three leavers removed late (after 9 days)Partly effective
Vulnerability management (Example)Critical findings fixed within 14 daysAll met; one high open 30 days with exceptionEffective
Data protection (Example)DPIA actions closed; retention enforcedDPIA actions closed; NFR-SEC-01 metEffective
Logging (Example)Security events in central monitoringConfirmed by test alertEffective
Accepted risks (Example)Reviewed at the date set at go-liveOne accepted risk past its review dateAction PIR-ACT-05
[Area][Expected][Observed][Status]

Update the risk register (compliance/risk-register) with any change in residual risk.

11. Supplier and contract performance

TopicContractedObservedAssessment
Delivery scope (Example)SC-01 to SC-05All deliveredMet
Delivery schedule (Example)Go-live YYYY-MM-DDSeven weeks late; shared cause (data quality)Partly met
Support response (Example)P1 response within 1 hourMet for the one P1Met
Knowledge transfer (Example)Internal team able to change matching rulesOne engineer able; second in trainingPartly met
Contract management (Example)Named contract manager on client sideRole vacant for two months after go-liveGap
Value for money (Example)Price review at month 12Due; benchmark recommendedAction PIR-ACT-06
[Topic][Contracted][Observed][Assessment]

12. Issues, incidents and residual defects

IDTypeDateSummaryRoot causeFixFix held?
INC-0412 (Example)P1 incidentYYYY-MM-DDCustomer API unavailable for 2 hours 40 minutesCertificate expired on the gateway routeAutomated renewal and expiry alertYes
INC-0388 (Example)P2 incidentYYYY-MM-DDEvents delayed 3 hoursDead-letter queue filled; replay done manuallyReplay runbook (PIR-ACT-02)Pending
KE-07 (Example)Known errorSince go-liveMerge of two households loses one contact preferenceSurvivorship rule orderFix in release 1.6Not yet
[ID][Incident / Problem / Known error / Defect]YYYY-MM-DD[Summary][Cause][Fix][Yes / No / Pending]

Postmortems are in [location] (dev/postmortem).

13. Lessons learned

Lesson IDWhat happenedWhyDo differently next timeApplies toOwner
LL-01 (Example)Data cleansing for policy system C delayed go-live by seven weeksData profiling was done on a sample onlyProfile full source data before committing a go-live dateAll data migration projects[@pmo]
LL-02 (Example)Run cost 19% above estimateCost model assumed batch matching; design changed without re-costingRe-cost when a decision record changes a cost driverArchitecture review process[@architect]
LL-03 (Example)Retention design was late and became a board conditionPrivacy requirements were added after design startedInclude retention design in the first design deliverableContract template[@contract-architect]
LL-04 (Example, positive)Stewards were productive from day oneStewards co-designed the queue and trained earlyRepeat early co-design with operational usersAll projects with new operational roles[@change-lead]
[LL-nn][What happened][Why][Change][Scope][@owner]

Record and share them in the lessons learned register (project/lessons-learned).

14. Recommendations and action plan

14.1 Recommendations

Rec IDRecommendationFindingPriorityIndicative costDecision needed fromDue
PIR-REC-01 (Example)Fund the legacy CRM retirement work package so DSP-2026-09 can closeOBJ-04, 9.1HighUSD [amount]SponsorYYYY-MM-DD
PIR-REC-02 (Example)Replace the direct replica read with an event feed to the data platformDRF-01HighUSD [amount]Architecture BoardYYYY-MM-DD
PIR-REC-03 (Example)Tune match thresholds and add a second steward review shift at month-endDIS-01, BEN-03HighInternal effortService ownerYYYY-MM-DD
PIR-REC-04 (Example)Re-evaluate on-change matching against micro-batch with measured costsDRF-02, NFR-OPS-01MediumInternal effortArchitecture BoardYYYY-MM-DD
PIR-REC-05 (Example)Re-engage the two deferred API consumers through the portfolio boardOBJ-03MediumNonePortfolio boardYYYY-MM-DD
[PIR-REC-nn][Recommendation][Ref][High / Medium / Low][Cost][Who]YYYY-MM-DD

14.2 Actions

Action IDActionOwnerDueStatus
PIR-ACT-01 (Example)Close AC-02 once DSP-2026-09 is resolved[@contract-architect]YYYY-MM-DDOpen
PIR-ACT-02 (Example)Write and test the dead-letter replay runbook[@sre-lead]YYYY-MM-DDOpen
PIR-ACT-03 (Example)Train a second engineer on matching rules[@service-owner]YYYY-MM-DDOpen
PIR-ACT-04 (Example)Fill the client-side contract manager role[@procurement]YYYY-MM-DDOpen
PIR-ACT-05 (Example)Review the accepted risk past its review date[@risk-owner]YYYY-MM-DDOpen
PIR-ACT-06 (Example)Benchmark the supplier price before the month 12 review[@procurement]YYYY-MM-DDOpen
PIR-ACT-07 (Example)Speed up leaver removal from steward roles[@iam-lead]YYYY-MM-DDOpen
[PIR-ACT-nn][Action][@owner]YYYY-MM-DD[Open / Done]

15. Conclusion and next review

QuestionAnswer
RQ-01 ObjectivesExample: Two of five met, two partly met, one not met
RQ-02 BenefitsExample: One on target, one near target, two behind; attribution shared with WP-04 for BEN-02
RQ-03 Business caseExample: Still valid; payback 3.6 years
RQ-04 Service and NFRsExample: Meets service levels except one month; matching throughput below target
RQ-05 AdoptionExample: Good with stewards and handlers; two system consumers missing
RQ-06 ArchitectureExample: One unauthorised drift, one overdue dispensation exit
RQ-07 Security and complianceExample: Controls operating; access removal and one risk review late
RQ-08 SupplierExample: Delivered scope; knowledge transfer and contract management gaps
RQ-09 LessonsExample: Four lessons recorded, one positive
ItemDecision
Benefits owner from now on[@service-owner]
Next benefits checkExample: Month 15, focused on BEN-02, BEN-03 and the CRM retirement
External review[Not required / Gate 5 review planned YYYY-MM-DD]
Periodic architecture health checkExample: Annually from month 12 (architecture/architecture-health-check)

16. Glossary and references

16.1 Glossary

TermMeaning in this document
Post-implementation reviewAn evaluation held after a solution has run in production, comparing its results with the original objectives and business case
BenefitA measurable improvement that a stakeholder values and that the investment is expected to cause
Dis-benefitA measurable negative effect caused by the change
AttributionThe share of an observed change that can reasonably be credited to this project
DriftA difference between the approved architecture and what is running in production
HypercareThe period of heightened support right after go-live
Business caseThe approved justification for the investment, with costs, benefits and risks
Gateway reviewAn independent assurance review at a fixed point in a project's life; Gate 5 focuses on benefits realisation
[Term][Meaning]

16.2 References

IDReferenceVersion / date
REF-01Queensland Treasury, Gateway review process, Gate 5: Benefits realisation (workbook)2020
REF-02HM Treasury and Evaluation Task Force, The Magenta Book: central government guidance on evaluation2020 edition; HTML version added 2026-05-15
REF-03Business case (business/business-case)[x.y]
REF-04Benefits realisation plan (project/benefits-realization-plan)[x.y]
REF-05Project closure report (project/project-closure-report)[x.y]
REF-06Architecture Contract and compliance assessments (architecture/architecture-contract, architecture/architecture-compliance-assessment)[x.y]
REF-07Go-live readiness review (project/go-live-readiness-review)[x.y]
REF-08Lessons learned register (project/lessons-learned)[x.y]
REF-09National Infrastructure and Service Transformation Authority (NISTA), Gate Review 5: Operations Review and Benefits Realisation, gov.ukUpdated 2026-08-20

Standard and licence note

Based on public-sector assurance and evaluation guidance: the Gateway review process Gate 5 (Benefits realisation) workbook published by Queensland Treasury (2020, licensed CC BY-NC-ND 4.0), which places the first benefits review typically 6 to 12 months after handover and treats the internal post-implementation review as a main input to the external review, and HM Treasury's Magenta Book on evaluation design. Only that timing and the general concepts are used; the structure, review questions, tables, examples and all text of this template are original, and no text from either source is reproduced or adapted. The UK NISTA Gate Review 5 guidance is cited by title and date only. Gateway and OGC Gateway are trade marks of their owner; in the UK the Gate Review process is now run by the National Infrastructure and Service Transformation Authority (NISTA). Check the current versions at treasury.qld.gov.au and gov.uk. Template licence: MIT.

Схожі шаблони