Proof of Concept Plan and Report: [Subject of the proof of concept]
One document for a time-boxed proof of concept (PoC) or proof of value (PoV): Part A is the plan
(decision supported, hypotheses, success criteria and measures, environment and data, scenarios,
roles, timeline, risks and stop rules), agreed before any test runs; Part B is the report
(execution log, deviations, results per criterion, evidence, findings and limits,
recommendation, sign-off and disposal). The evaluation steps follow the quality evaluation
process reference model of ISO/IEC 25040:2024 (define, design, plan, execute, conclude) and the
measures follow the product quality characteristics of ISO/IEC 25010:2023. Replace every
[bracketed] value and every row marked Example; the worked example tests an order event stream
with change data capture, a condition of the option chosen in Solution Options Analysis
(architecture/solution-options-analysis).
Document control
| Field | Value |
|---|
| Document ID | [POC-001] |
| Type | [Proof of concept / Proof of value / Pilot / Technical spike] |
| Decision supported | [Decision and document ID, for example SOA-001 condition CND-002] |
| Time box | YYYY-MM-DD to YYYY-MM-DD ([n] weeks) |
| Budget | [Amount and what it covers] |
| Vendor or partner involved | [Name and role, or none] |
| Version | [0.1] |
| Status | [Plan draft / Plan approved / Executing / Report draft / Closed] |
| Owner | [@solution-architect] |
| Approvers/reviewers | [@decision-owner], [@engineering-lead], [@security-architect], [@product-owner] |
| Last updated | YYYY-MM-DD |
| Classification | [Internal / Confidential] |
Approvals
| Stage | Role | Name | Decision | Date |
|---|
| Plan (Part A) | Decision owner | [@name] | [Approve / Approve with changes / Reject] | YYYY-MM-DD |
| Plan (Part A) | Security | [@name] | [Environment and data approved] | YYYY-MM-DD |
| Report (Part B) | Decision owner | [@name] | [Accept findings / Request more evidence] | YYYY-MM-DD |
| Report (Part B) | Architecture board | [@name] | [Noted / Decision taken] | YYYY-MM-DD |
Revision history
| Version | Date | Author | Change summary |
|---|
| 0.1 | YYYY-MM-DD | [@name] | Plan drafted |
| 1.0 | YYYY-MM-DD | [@name] | Example: Plan approved; criteria frozen |
| 1.1 | YYYY-MM-DD | [@name] | Example: Deviation DV-001 recorded (load generator replaced) |
| 2.0 | YYYY-MM-DD | [@name] | Example: Report completed and signed off |
Contents
1. Purpose, type and decision supported
1.1 Decision and possible outcomes
[State the decision, who takes it, by when, and the action for each outcome.] Example: The
architecture board will confirm or reject the incremental replacement option for order management.
If the event stream meets the criteria, the board confirms the option; if it fails, the board
re-scores the options with the PoC evidence; if it is inconclusive, the PoC is extended once by at
most one week.
| Outcome | Action |
|---|
| Pass: every must criterion met | Example: Confirm OPT-2; move to design of the order capture service |
| Conditional pass: a must criterion partly met with a credible fix | Example: Confirm OPT-2 with a condition and an owner for the fix |
| Fail | Example: Re-score C2 in the options analysis; consider OPT-3 for orchestration |
| Inconclusive | Example: One extension of at most one week, then decide on the evidence available |
1.2 Type of exercise
| Type | Answers the question | Typical length | Who runs it |
|---|
| Technical spike | Can we do this at all, and how? | Days | One or two engineers |
| Proof of concept | Does this approach meet our technical criteria in our context? | 2 to 6 weeks | Architecture and engineering, with vendor support if needed |
| Proof of value | Does it deliver measurable business value for our users and data? | 4 to 12 weeks | Business and IT together, often with the vendor |
| Pilot | Does it work in production for a limited group? | Months | Delivery team under production controls |
This document is a [type]. Example: Proof of concept.
1.3 Evaluation purpose
ISO/IEC 25040:2024 describes evaluations carried out for different purposes, for example judging
suitability for a specific use, checking conformity to stated requirements, or comparing candidate
products; check the standard for its own list. State which purpose applies, because it decides how
strict the criteria and the evidence must be. Example: Suitability for a specific use (our order
volumes, our database, our cloud account).
2. Objectives, hypotheses and scope
2.1 Hypotheses
| ID | Hypothesis | Assumption it tests | Criteria |
|---|
| HYP-001 (Example) | A managed event stream with change data capture carries order events from the monolith to new services within 2 seconds at ten times the production target rate | Event latency is acceptable for order capture (SOA-001 criterion C2) | SC-001, SC-002 |
| HYP-002 (Example) | No order event is lost or applied twice through restarts and failover | Consistency across monolith and new services | SC-003, SC-004 |
| HYP-003 (Example) | Change data capture works without changing monolith code | Low risk to the legacy system | SC-005 |
| HYP-004 (Example) | The running cost at production volume fits the TCO | SOA-001 OPT-2 cost line | SC-008 |
| [HYP-NNN] | [Hypothesis] | [Assumption] | [SC IDs] |
2.2 Scope
| In scope | Out of scope |
|---|
| Example: Change data capture from the order database, managed event stream, one consumer service | Order capture user interface and business logic |
| Example: Load, failover and schema change tests | Production data and production network paths |
| Example: Cost projection from measured usage | Commercial negotiation with the provider |
| [Item] | [Item] |
2.3 Stakeholders and their concerns
| Stakeholder | Concern the PoC must answer |
|---|
| Example: Architecture board | Is the chosen option technically sound? |
| Example: Order operations | Will any order be lost or duplicated? |
| Example: Database administrators | Does change data capture load the production database? |
| [Stakeholder] | [Concern] |
3. Success criteria and measures
| ID | Characteristic | Measure | Threshold (Example) | Method | Priority |
|---|
| SC-001 | Performance efficiency | End-to-end latency, database commit to consumer acknowledgement, 95th percentile | At most 2 s at 300 events/s sustained for 60 min | Load generator plus timestamps in each event | Must |
| SC-002 | Performance efficiency | Consumer lag recovery after a burst | Lag under 5 s within 2 min after 600 events/s for 5 min | Lag metric from the stream service | Must |
| SC-003 | Reliability | Lost or duplicated orders after consumer restarts and broker failover | Zero lost, zero applied twice, over 1,080,000 events | Reconciliation by order ID and event sequence | Must |
| SC-004 | Reliability | Recovery time after loss of one availability zone | Consumer processing again within 5 min | Fault injection | Should |
| SC-005 | Compatibility | Change types captured without monolith code change | All 14 change types on the order tables | Change catalogue replay | Must |
| SC-006 | Maintainability | Adding an optional field to the event schema | No consumer downtime; old consumers keep working | Schema registry compatibility test | Should |
| SC-007 | Security | Encryption, access control and audit logging | Every item of the security checklist met | Configuration review by security | Must |
| SC-008 | Cost (TCO input) | Projected monthly run cost at production volume | At most USD 9,000 per month | Provider bill for the test, scaled to forecast volume | Should |
| [SC-NNN] | [Characteristic] | [Measure] | [Threshold] | [Method] | [Must / Should] |
Derived values: today's peak is 120 orders per minute; the target is three times that, 360 orders
per minute; each order creates about 5 events, so the production target is 1,800 events per minute
or 30 events/s. The PoC tests at ten times the production target (300 events/s) to leave headroom
for growth and replays. SC-003 covers 300 events/s for 3,600 s, which is 1,080,000 events.
Decision rule: pass when every must criterion is met and at least two of the three should
criteria are met. A must criterion that is partly met, with a fix that is understood, costed and
owned, gives a conditional pass. Anything else is a fail.
4. Environment, data and tooling
4.1 Representativeness
| Element | PoC environment | Production | Gap and its effect on validity |
|---|
| Example: Database | Copy of the order database schema, same engine and version, 1 TB synthetic data | Same engine and version, 3 TB | Smaller data; CDC log volume scaled by the event rate, not by size |
| Example: Event stream | Managed service, 3 brokers across 3 zones | Same service, sized by the PoC result | None for latency; cost scaled |
| Example: Network | Same cloud region, private endpoints | Same | None |
| Example: Consumer | One reference consumer with idempotent writes | Several domain services | Results apply per consumer |
| [Element] | [PoC] | [Production] | [Gap] |
4.2 Data
| Item | Decision |
|---|
| Data source | Example: Synthetic orders generated from production statistics; no production records |
| Personal data | Example: None; names and addresses generated |
| Approval | Example: Security approved the data plan on YYYY-MM-DD |
| Retention and deletion | Example: All data deleted at closure (section 13) |
4.3 Tools and instrumentation
| Tool | Purpose | Owner |
|---|
| Example: Load generator | Inserts and updates orders at a set rate | [@engineer] |
| Example: Tracing timestamps in event headers | Latency per event from commit to acknowledgement | [@engineer] |
| Example: Reconciliation script | Compares source changes with consumer writes by order ID and sequence | [@engineer] |
| Example: Cost explorer tags | Isolates the PoC spend | [@finops] |
| [Tool] | [Purpose] | [Owner] |
5. Test scenarios
| ID | Scenario | Steps | Expected result | Criteria |
|---|
| SCN-001 (Example) | Steady peak | Run 300 events/s for 60 min | p95 latency at most 2 s; no lag growth | SC-001, SC-003 |
| SCN-002 (Example) | Burst | Raise to 600 events/s for 5 min, then back to 300 | Lag under 5 s within 2 min after the burst | SC-002 |
| SCN-003 (Example) | Consumer restart | Kill the consumer three times during SCN-001 | Reconciliation shows no loss or duplicate | SC-003 |
| SCN-004 (Example) | Zone failure | Remove one zone's broker and consumer | Processing resumes within 5 min; no loss | SC-003, SC-004 |
| SCN-005 (Example) | Change catalogue | Replay each of the 14 change types, including bulk updates by stored procedure | Every change appears as an event | SC-005 |
| SCN-006 (Example) | Schema evolution | Add an optional field; deploy the new producer first | Old consumer keeps processing | SC-006 |
| SCN-007 (Example) | Security review | Review encryption, roles, network paths and audit logs | Checklist complete | SC-007 |
| [SCN-NNN] | [Scenario] | [Steps] | [Expected] | [SC IDs] |
Traceability check: every SC from SC-001 to SC-007 has at least one scenario; SC-008 is computed
from the bill for the whole PoC, so it needs no scenario of its own.
6. Roles, timeline and budget
6.1 Roles (RACI)
R = responsible, A = accountable, C = consulted, I = informed.
| Activity | Solution architect | Engineers (2) | Security | Database administrator | Vendor support | Decision owner |
|---|
| Example: Approve plan and criteria | R | C | C | C | I | A |
| Example: Build environment and tooling | A | R | C | C | C | I |
| Example: Execute scenarios | A | R | I | C | C | I |
| Example: Independent observation of SCN-003 and SCN-004 | R | C | I | C | I | A |
| Example: Write the report | R | C | C | C | I | A |
| [Activity] | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] |
6.2 Timeline
The four blocks of five working days fill the four-week time box exactly; there is no slack, so a
lost day is recovered by the stop rules in section 7, not by extending the box.
6.3 Budget
| Item | Estimate (Example) | Basis |
|---|
| Cloud services | USD 6,000 | Brokers, database copy and load generators for 4 weeks |
| Engineering time | 2 engineers x 4 weeks | Internal cost centre [code] |
| Architect and reviewers | 0.5 FTE x 4 weeks | Internal |
| Vendor support | Included in the existing support plan | No extra charge agreed in writing |
| Total external spend | USD 6,000 | |
| [Item] | [Estimate] | [Basis] |
7. Risks, constraints and stop rules
| ID | Risk or constraint | Effect | Mitigation |
|---|
| PR-001 (Example) | Synthetic data does not reflect real change patterns | Latency or CDC results too optimistic | Generate from production statistics; include the bulk update procedure |
| PR-002 (Example) | Vendor tunes the service beyond what we would run | Results not reproducible | Use only settings documented for general use; record every setting |
| PR-003 (Example) | Load generator becomes the bottleneck | False latency readings | Monitor generator CPU; scale out before SCN-001 |
| PR-004 (Example) | Confirmation bias in the team that proposed the option | Results read generously | Criteria frozen in Part A; independent observer for failure tests |
| [PR-NNN] | [Risk] | [Effect] | [Mitigation] |
- Stop and report fail if a must criterion is missed by more than 50% after two tuning attempts.
- Stop and report inconclusive if the environment is not ready by day 7.
- Never extend the time box more than once, and never by more than one week.
- Stop at once if any production system or real personal data is touched by mistake.
8. Execution log and deviations
8.1 Execution log
| Run | Date | Scenario | Configuration | Outcome | Evidence |
|---|
| R-01 (Example) | YYYY-MM-DD | SCN-001 | 3 brokers, 12 partitions | Aborted after 10 min: generator saturated | EV-001 |
| R-02 (Example) | YYYY-MM-DD | SCN-001 | Same, 3 generators | Completed | EV-002 |
| R-03 (Example) | YYYY-MM-DD | SCN-003 | Same | 2 duplicates found | EV-004 |
| R-04 (Example) | YYYY-MM-DD | SCN-003 | Idempotency key on order ID and sequence | Completed, no duplicates | EV-005 |
| [R-NN] | [Date] | [SCN ID] | [Configuration] | [Outcome] | [EV ID] |
8.2 Deviations from the plan
| ID | Deviation | Reason | Effect on validity | Approved by |
|---|
| DV-001 (Example) | Load generated by three instances instead of one | Single generator saturated (R-01) | None; latency measured inside events | [@decision-owner] |
| DV-002 (Example) | Consumer changed to idempotent writes after R-03 | Duplicates on restart | Design change recorded as finding F-002 | [@decision-owner] |
| [DV-NNN] | [Deviation] | [Reason] | [Effect] | [Approver] |
9. Results per criterion
| ID | Threshold | Measured (Example) | Verdict | Evidence |
|---|
| SC-001 | p95 at most 2 s at 300 events/s | p95 1.4 s, p99 2.3 s (R-02) | Pass | EV-002 |
| SC-002 | Lag under 5 s within 2 min after burst | Under 5 s after 95 s | Pass | EV-003 |
| SC-003 | Zero lost, zero duplicates | Zero lost; 2 duplicates in R-03 (planned run); zero in R-04 after fix | Pass with design change (DV-002, F-002); carried as a design rule (PG-005) | EV-004, EV-005 |
| SC-004 | Processing again within 5 min | 3 min 40 s | Pass | EV-006 |
| SC-005 | All 14 change types captured | 13 of 14: bulk update by stored procedure bypasses the captured log path | Partly met | EV-007 |
| SC-006 | No consumer downtime on optional field | No downtime | Pass | EV-008 |
| SC-007 | Security checklist complete | Complete | Pass | EV-009 |
| SC-008 | At most USD 9,000 per month | USD 7,800 per month projected | Pass | EV-010 |
Decision rule applied: must criteria SC-001, SC-002 and SC-007 met; SC-003 met only after design
change DV-002, so the planned run R-03 is reported and the fix is carried as PG-005; SC-005 partly
met; should criteria SC-004, SC-006 and SC-008 all met (three of three). Verdict: conditional
pass, provided the SC-005 gap and the SC-003 design rule have owned, costed actions (section 12).
Cost projection check (Example): the four-week bill for the stream service was USD 2,600 at an
average of 300 events/s, ten times the production rate. Brokers are billed mostly by capacity, not
by event, so the projection uses the production broker size quoted by the provider, USD 7,200 per
month. That quote is higher than the PoC bill (about USD 2,800 per month) because it covers
production plus a staging cluster, 30-day retention and the provider's multi-zone production tier,
none of which the PoC used. Add storage and transfer measured in the PoC, USD 600 per month, to give
USD 7,800 per month.
10. Evidence register
| ID | Evidence | Run | Location |
|---|
| EV-001 (Example) | Generator CPU graph showing saturation | R-01 | [Repository path or document store link] |
| EV-002 (Example) | Latency histogram and raw percentiles export | R-02 | [Location] |
| EV-003 (Example) | Consumer lag graph during burst | SCN-002 run | [Location] |
| EV-004 (Example) | Reconciliation output listing two duplicate order IDs | R-03 | [Location] |
| EV-005 (Example) | Reconciliation output with zero differences | R-04 | [Location] |
| EV-006 (Example) | Fault injection log and recovery timeline | SCN-004 run | [Location] |
| EV-007 (Example) | Change catalogue results per change type | SCN-005 run | [Location] |
| EV-008 (Example) | Schema compatibility test output | SCN-006 run | [Location] |
| EV-009 (Example) | Signed security checklist | SCN-007 | [Location] |
| EV-010 (Example) | Tagged cost report and projection sheet | Whole PoC | [Location] |
| [EV-NNN] | [Evidence] | [Run] | [Location] |
11. Findings, limitations and validity
11.1 Findings
| ID | Finding | Consequence for the design |
|---|
| F-001 (Example) | p99 latency (2.3 s) is above the p95 target, driven by broker leader elections | Order capture must not wait synchronously on the stream |
| F-002 (Example) | Consumers must be idempotent on order ID and sequence | Make idempotency a design rule for every consumer |
| F-003 (Example) | Bulk updates by one stored procedure are not captured | Rewrite the procedure to row-level updates, or publish those changes through an outbox table |
| [F-NNN] | [Finding] | [Consequence] |
11.2 Limitations and validity
| Limitation | Effect on how far the results apply |
|---|
| Example: Synthetic data, one third of production size | Latency and CDC behaviour valid; storage cost scaled, not measured |
| Example: One reference consumer | Each new consumer needs its own idempotency test |
| Example: Four-week run | Long-term effects such as log retention growth not observed |
| [Limitation] | [Effect] |
12. Recommendation and conclusion
Verdict: [Pass / Conditional pass / Fail / Inconclusive]. Example: Conditional pass.
Recommendation: [What the decision owner should decide.] Example: Confirm OPT-2 in SOA-001,
with condition CND-002 recorded as met subject to the fix for F-003 and the idempotency rule
(PG-005) below.
| ID | Condition or production gap | Owner | Due | Estimate (Example) |
|---|
| PG-001 (Example) | Fix F-003: outbox table for the bulk update procedure | [@engineering-lead] | Before order capture go-live | 8 person-days |
| PG-002 (Example) | Replace PoC credentials and broad roles with least-privilege roles from the security design | [@security-architect] | Design phase | 3 person-days |
| PG-003 (Example) | Add monitoring and alerts for consumer lag and end-to-end latency | [@sre-lead] | Before go-live | 5 person-days |
| PG-004 (Example) | Size brokers for production from the PoC result and the forecast | [@solution-architect] | Design phase | 1 person-day |
| PG-005 (Example) | Idempotency test per consumer (F-002, SC-003 design rule) in each service's test suite | [@engineering-lead] | Before each consumer goes live | 2 person-days per consumer |
| [PG-NNN] | [Gap] | [Owner] | [Date] | [Estimate] |
Update the decision record (ADR, architecture/adr-full) and the options analysis scores with the
evidence, and carry the production gaps into the delivery backlog.
13. Sign-off, handover and disposal
| Item | Decision (Example) | Done |
|---|
| Report accepted | Decision owner and architecture board | [ ] |
| Code kept | Reconciliation script and load generator kept in the platform repository; consumer code discarded | [ ] |
| Knowledge transfer | One-hour walkthrough for the order capture team | [ ] |
| Environment deleted | Within five working days of sign-off | [ ] |
| Data deleted | Synthetic data deleted with the environment; deletion confirmed by the cost report | [ ] |
| Vendor informed | Outcome and next steps shared under the existing agreement | [ ] |
14. Glossary and references
14.1 Glossary
| Term | Meaning |
|---|
| Proof of concept | Time-boxed test of whether an approach meets stated technical criteria in the organisation's context |
| Proof of value | Time-boxed test of whether a solution delivers measurable business value with real users or data |
| Hypothesis | A statement the PoC can prove false |
| Success criterion | A measure, threshold and method agreed before testing |
| Must and should criteria | Criteria that decide the verdict, and criteria that inform it |
| Stop rule | A condition, agreed in advance, that ends the PoC |
| Change data capture | Reading changes from a database's log and publishing them as events |
| Production gap | Something the PoC skipped or simplified that production must have |
| [Term] | [Meaning] |
14.2 References
| ID | Reference | Version / date |
|---|
| REF-01 | ISO/IEC 25040:2024, SQuaRE: Quality evaluation framework (clause 5, quality evaluation process reference model) | Edition 2, 2024-09 |
| REF-02 | ISO/IEC 25010:2023, SQuaRE: Product quality model | Edition 2, 2023-11 |
| REF-03 | Solution Options Analysis (architecture/solution-options-analysis) | [x.y] |
| REF-04 | Performance Test Plan (quality/performance-test-plan) | [x.y] |
| REF-05 | [Vendor documentation and service limits used for the configuration] | [Date] |
Standard and licence note
Structure aligned with ISO/IEC 25040:2024 (Systems and software engineering, SQuaRE, Quality
evaluation framework; second edition, September 2024), whose quality evaluation process reference
model runs through defining, designing, planning, executing and concluding an evaluation, and with
the product quality characteristics of ISO/IEC 25010:2023. These standards are copyrighted: only
the clause structure, activity names and characteristic names are used; all text, tables and
examples are original, and no definitions, tables or requirements are reproduced. Obtain the
standards for their normative content and check the current editions at iso.org. Template licence:
MIT.