Architecture Cost-Benefit Analysis (CBAM): [System name]

Report of an economic analysis of architectural strategies following the Cost Benefit Analysis Method (CBAM) developed at the Software Engineering Institute (SEI), Carnegie Mellon University, by Rick Kazman, Jai Asundi and Mark Klein (SEI technical reports CMU/SEI-2001-TR-035, December 2001, and CMU/SEI-2002-TR-035, September 2002). It records the scenarios carried over from an architecture evaluation, their response levels and utility curves, stakeholder votes and weights, the candidate architectural strategies with their expected responses, benefit, cost, return on investment (ROI), a sensitivity analysis and the investment decision. Replace every [bracketed] value and every row marked Example.

Document control

Approvals

Revision history

Contents

1. Executive summary
2. Context, scope and participants
3. Step 1: Collate scenarios
4. Step 2: Refine scenarios
5. Step 3: Prioritise scenarios
6. Step 4: Assign utility
7. Step 5: Develop architectural strategies
8. Step 6: Determine expected utility
9. Step 7: Calculate total benefit
10. Step 8: Choose strategies by ROI
11. Step 9: Confirm results with intuition
12. Uncertainty and sensitivity analysis
13. Decision, follow-up and traceability
14. Glossary and references

1. Executive summary

[Two or three paragraphs: the decision being supported, the budget, the recommended strategies in ROI order, what they buy in quality terms and the main caveats.]

Example: The payments platform team had USD 500,000 for architecture work in the next two quarters and five candidate strategies addressing the four highest-priority scenarios from the ATAM held on YYYY-MM-DD. The CBAM recommends funding AS-01 (read cache and asynchronous confirmation), AS-04 (payment provider adapter layer) and AS-02 (warm standby region), in that ROI order. Together they cost exactly the budget and deliver 101.3 weighted utility points. AS-03 (active-active multi-region) delivers the largest single benefit (54.2 points) but costs USD 900,000 and has the lowest ROI; it is rejected for this horizon and recorded as a future option. The ranking is stable under a plus or minus 30 percent cost range for AS-02; it is most sensitive to the latency AS-01 actually achieves, which a two-week spike will test before the full build, and to the weight of the fraud scenario, which will be re-voted at the next quarterly review.

2. Context, scope and participants

2.1 Decision and scope

The CBAM can run in more than one iteration. A first iteration narrows the field using rough estimates; a second iteration refines costs, response estimates and utility curves for the remaining strategies. Record which iteration this report covers and what changed since the previous one.

2.2 Participants

2.3 Inputs

3. Step 1: Collate scenarios

Collect the scenarios elicited in the preceding evaluation, invite stakeholders to add scenarios that reflect business goals not yet covered, and shortlist roughly the top third by contribution to those goals. Each scenario keeps its original ID where one exists so it can be traced back. Example: 24 scenarios (19 from the ATAM, 5 new) were reviewed; the eight below form the shortlisted top third.

4. Step 2: Refine scenarios

Each scenario is refined to a single response measure with four response levels. The worst case is the level at which the scenario becomes worthless to the business; the best case is the level beyond which further improvement adds no value.

Check every row: the four levels must be ordered (worst, current, desired, best) in the direction that the business prefers, and the current level must come from measurement or monitoring, not memory. Record the source of each current level in the evidence column of the appendix table in section 13.3.

5. Step 3: Prioritise scenarios

Each of the [n] voting stakeholders received 100 votes. Scenarios were ranked by total votes and the top half retained. Weights are normalised so that the highest-ranked scenario has weight 1.00.

Scenarios not retained are not discarded: list them in section 13.2 as candidates for a later iteration or for the technical debt register. Example: SCN-05 (audit trail) received few votes because a manual workaround exists; the compliance lead asked for it to be re-examined if the audit frequency rises.

6. Step 4: Assign utility

Utility expresses how much a response level is worth to the stakeholders on a scale from 0 (worthless) to 100 (no further value). The four points per scenario define a utility-response curve; values between points are obtained by linear interpolation in step 6.

Record, for each curve, who proposed the values and whether the group reached consensus or the product owner decided. Utility values carry more weight than any other input, so they must be defensible.

7. Step 5: Develop architectural strategies

7.1 Strategy catalogue

7.2 Cost basis

Use one cost basis for every strategy. If one strategy includes twelve months of run cost and another does not, the ROI ranking is meaningless. For lifetime cost comparisons, reference the TCO analysis (architecture/tco-analysis) and bring a single figure into this table.

7.3 Expected response levels

8. Step 6: Determine expected utility

For an expected response x that lies between two known points (x0, u0) and (x1, u1) on the utility curve, the expected utility is obtained by linear interpolation.

U(x) = u0 + (x0 - x) / (x0 - x1) * (u1 - u0)

Keep unrounded values in the working spreadsheet and round only when reporting. The worked example rounds to one decimal place in this table but uses the unrounded values in section 9.

9. Step 7: Calculate total benefit

b(i, j) = U_expected(i, j) - U_current(j)
B(i)    = sum over j of b(i, j) x W(j)

Here i is the strategy, j is the scenario and W(j) is the weight from step 3.

A strategy with a negative total benefit is never funded on CBAM grounds, whatever its cost. If such a strategy is mandatory (for example to meet a regulation), record it as a constraint in section 10.2 and deduct its cost from the budget before ranking the rest.

10. Step 8: Choose strategies by ROI

10.1 ROI ranking

ROI(i) = B(i) / C(i)

ROI in this report is expressed in weighted utility points per USD 1,000. Some presentations multiply the ratio by 100; the ranking is the same either way.

Example: The funded set costs USD 500,000, equal to the budget, and delivers 40.5 + 19.7 + 41.1 = 101.3 weighted utility points (101.33 unrounded). Replacing AS-02 with AS-03 would raise the availability benefit but the set would cost USD 1,100,000, more than twice the budget.

10.2 Constraints applied

10.3 Selection procedure

1. Remove strategies with negative total benefit.
2. Fund mandatory strategies first and deduct their cost.
3. Rank the remaining strategies by ROI, highest first.
4. Walk down the list. Fund a strategy if it fits the remaining budget and schedule and does not conflict with one already funded.
5. When a strategy does not fit, check whether a cheaper one further down fits the remainder before stopping.
6. Record every skipped strategy with its reason.

11. Step 9: Confirm results with intuition

Example: Stakeholders questioned why AS-05 (fraud rules) ranked below AS-02. Tracing the result showed that the fraud scenario's low weight (0.30) came from a vote held before the most recent fraud incident. The product owner chose to keep the weight for this iteration and to rerun the vote at the next quarterly review; the decision is recorded in section 13.

12. Uncertainty and sensitivity analysis

12.1 Input confidence

12.2 Sensitivity results

The last row shows why the weights matter, and why a pure ROI walk is not the whole answer. With the fraud weight at 0.60, ranking by ROI funds AS-01, AS-04 and AS-05 (USD 350,000, benefit 40.5 + 19.7 + 24.0 = 84.2 points) and then stops, because AS-02 (USD 300,000) no longer fits the remaining USD 150,000. The original set AS-01, AS-04 and AS-02 still fits the budget exactly and delivers 101.3 points. When the budget is tight, compare the total benefit of every feasible set of strategies, not only the order of the ratios, and present such breakpoints to decision makers explicitly.

12.3 Risk to the decision

13. Decision, follow-up and traceability

13.1 Decision record

13.2 Deferred items and residual risk

13.3 Traceability matrix

13.4 Benefit realisation

After delivery, measure the actual response level for each funded strategy and compare it with the expected level. Feed the difference back into future estimates: a team that overestimates gains by 30 percent should know it before the next CBAM.

14. Glossary and references

14.1 Glossary

14.2 References

Standard and licence note

Based on the Cost Benefit Analysis Method (CBAM) published by the Software Engineering Institute, Carnegie Mellon University, in technical reports CMU/SEI-2001-TR-035 and CMU/SEI-2002-TR-035: the nine steps (collate, refine and prioritise scenarios, assign utility, develop strategies, determine expected utility, calculate benefit, choose by ROI, confirm with intuition), response levels, utility curves, weights and the benefit-to-cost ratio. This template is original text written for this collection and reproduces no text, tables or figures from SEI publications or from books on the method; the worked example is invented and marked Example. Check the SEI digital library for the current versions of the reports before use. Carnegie Mellon, SEI and ATAM may be trademarks or service marks of Carnegie Mellon University; this template is not affiliated with or endorsed by the SEI. Template licence: MIT.