Markdown Editor

Technology Selection and Build vs Buy Template

Build, buy, open source or managed service evaluation with weighted decision matrix, PoC results, risks, cost and decision

How to use: Use before committing to a product or platform for a capability. Fill the problem, must-haves and criteria weights first and agree them before scoring. The main pitfall is adjusting weights after seeing scores; run the sensitivity check and record the final decision in an ADR.

Preview

Technology Selection: [Capability or component name]

Technology selection and build-versus-buy evaluation for one capability: problem and requirements, candidate options (build, buy, open source, managed service), a weighted decision matrix, proof-of-concept results, risks, a cost summary and the decision with its ADR link. Based on the common weighted decision matrix practice. Replace every [bracketed] value and every row marked Example; the worked example evaluates a product search capability.

FieldValue
Document ID[TS-001]
Capability[e.g. Product search for the web shop]
Version[0.1]
Status[Draft / In review / Decided / Superseded]
Owner[@architect]
Decision makers[@cto], [@product-lead], [@platform-lead]
Evaluation periodYYYY-MM-DD to YYYY-MM-DD
Related[ADR-NNN], [TCO analysis], [Architecture review request]

1. Problem and context

[Describe the problem in two or three paragraphs. Include the current state, the pain it causes and the trigger for this evaluation.]

Example: The web shop uses SQL LIKE queries for product search. Median search latency is 1.8 s at peak, typo tolerance is absent, and analytics show that 31% of searches return zero results. Search-led conversion is 40% below the industry benchmark our product team tracks. The catalogue will grow from 120,000 to 600,000 SKUs after the marketplace launch in Q3, which the current approach cannot support.

ConstraintValue
Decision needed byYYYY-MM-DD
Budget envelope (3 years)[e.g. USD 500,000]
Mandatory compliance[e.g. GDPR, data stored in the EU]
Team capacity[e.g. 2 engineers for 3 months, 0.5 FTE run]
Strategic principles[e.g. "Buy before build for non-differentiating capabilities"]

2. Requirements and evaluation criteria

2.1 Must-have requirements (knock-out)

Any option that fails one of these is excluded and not scored.

IDRequirementVerification
TS-MH-01Example: p95 search latency <= 200 ms at 300 queries per secondPoC load test
TS-MH-02Example: Data processed and stored in the EU onlyContract and DPA review
TS-MH-03Example: Typo tolerance and synonyms configurable by merchandisersPoC demo
TS-MH-04Example: Index update visible within 60 s of a price changePoC test
[TS-MH-nn][Requirement][Method]

2.2 Scored criteria and weights

Weights are agreed before scoring and sum to 1.00. Scores use a 1 to 5 scale defined in the scale table.

IDCriterionWhat it measuresWeight wiw_i
C1Functional fitRelevance quality, facets, merchandising tools0.30
C2Cost (3-year TCO)Lower cost scores higher0.20
C3Security and complianceCertifications, data residency, access control0.15
C4OperabilityEffort to run, upgrade, monitor and scale0.15
C5Team skills fitExisting skills, learning curve, hiring market0.10
C6Exit and vendor riskLock-in, data portability, vendor viability0.10
Total1.00
ScoreMeaning
5Exceeds the need with evidence
4Fully meets the need
3Meets the need with workarounds
2Partly meets the need; significant gaps
1Does not meet the need

3. Options considered

IDOptionArchetypeDescriptionMust-haves met?
O0Do nothingBaselineExample: Keep SQL LIKE search; add cachingNo (TS-MH-01, TS-MH-03)
OABuild in-houseBuildExample: Search service on PostgreSQL full-text search with custom rankingYes
OB[SaaS search vendor]BuyExample: Hosted search API with merchandising console, EU regionYes
OC[Open-source engine], self-hostedOpen sourceExample: Open-source search engine on our Kubernetes clusterYes
OD[Managed engine service]Managed serviceExample: Cloud provider's managed open-source search engineYes

3.1 Option notes

For each option, record what was learned beyond the matrix.

  • OA Build: [Strengths, gaps, effort estimate.] Example: Full control and no licence fee; ranking and typo tolerance need about 14 engineer-weeks; merchandiser tooling would be built from scratch.
  • OB Buy: [Notes.] Example: Best relevance out of the box; pricing is per 1,000 searches, so cost rises with traffic; export of synonyms and rules is possible through the API.
  • OC Open source: [Notes.] Example: Mature engine and large community; we would own cluster upgrades and on-call; no merchandiser UI.
  • OD Managed service: [Notes.] Example: Same engine as OC with upgrades and snapshots handled by the provider; still no merchandiser UI.

4. Weighted decision matrix

The weighted score of option jj is the sum over criteria of weight times score:

Sj=∑i=1nwi sij,∑i=1nwi=1S_j = \sum_{i=1}^{n} w_i \, s_{ij}, \qquad \sum_{i=1}^{n} w_i = 1

where wiw_i is the weight of criterion ii and sijs_{ij} is the score of option jj on criterion ii (1 to 5). The maximum possible score is 5.00.

Criterionwiw_iOA BuildOB BuyOC Open sourceOD Managed
C1 Functional fit0.305434
C2 Cost (3-year TCO)0.202343
C3 Security and compliance0.153434
C4 Operability0.152524
C5 Team skills fit0.104423
C6 Exit and vendor risk0.105243
Weighted score SjS_j1.003.553.753.053.60
Rank3142

Example arithmetic for OB Buy: SB=0.30(4)+0.20(3)+0.15(4)+0.15(5)+0.10(4)+0.10(2)=1.20+0.60+0.60+0.75+0.40+0.20=3.75S_B = 0.30(4) + 0.20(3) + 0.15(4) + 0.15(5) + 0.10(4) + 0.10(2) = 1.20 + 0.60 + 0.60 + 0.75 + 0.40 + 0.20 = 3.75.

Example arithmetic for OD Managed: SD=1.20+0.60+0.60+0.60+0.30+0.30=3.60S_D = 1.20 + 0.60 + 0.60 + 0.60 + 0.30 + 0.30 = 3.60.

4.1 Sensitivity check

A difference of less than about 0.25 between the top two options is within scoring noise. Re-run the matrix with plausible alternative weights and record whether the ranking changes.

ScenarioWeight changeOAOBOCODLeader
BaselineAs in 2.23.553.753.053.60OB
Example: Exit risk matters moreC6 0.20, C1 0.203.553.553.153.50OA and OB tie
[Scenario][Change][ ][ ][ ][ ][ ]

The second row is computed by moving 0.10 of weight from C1 to C6: for OB, 3.75−0.10(4)+0.10(2)=3.553.75 - 0.10(4) + 0.10(2) = 3.55; for OA, 3.55−0.10(5)+0.10(5)=3.553.55 - 0.10(5) + 0.10(5) = 3.55.

5. Proof-of-concept results

PoC scope: [dataset, query set, load profile, duration, team]. Example: 120,000 real SKUs, 5,000 logged queries replayed, 300 QPS for 30 minutes, two engineers for two weeks per option.

TestTargetOA BuildOB BuyOC Open sourceOD Managed
p95 latency at 300 QPS<= 200 msExample: 240 ms48 ms95 ms110 ms
Zero-result rate on replay<= 10%14%6%9%9%
Index freshness after update<= 60 s5 s20 s2 s2 s
Merchandiser rule set-up< 1 dayNot built2 hoursNot availableNot available
[Test][Target][Result][Result][Result][Result]

Findings: [Summarise what surprised the team and which scores changed after the PoC.] Example: OA missed the latency target until a dedicated read replica was added; this cost is reflected in C2.

6. Risks

IDOptionRiskLikelihoodImpactMitigationOwner
RSK-01OBExample: Per-search pricing grows faster than revenue during sales peaksMediumHighNegotiate a committed-volume tier; cache popular queries[@name]
RSK-02OBExample: Vendor acquisition changes the roadmapLowMediumKeep rules and synonyms exportable; annual exit drill[@name]
RSK-03OAExample: Relevance tuning depends on one engineerHighHighPair work; document ranking model[@name]
[RSK-nn][Option][Risk][L/M/H][L/M/H][Mitigation][@name]

7. Cost summary

Detailed figures are in [TCO analysis link]. Currency: [USD], horizon [3] years, nominal values.

Cost itemOA BuildOB BuyOC Open sourceOD Managed
Implementation (one-off)Example: 180,00060,00090,00075,000
Licences or subscription (3 years)0270,00000
Infrastructure (3 years)108,0000126,000198,000
People to run (3 years)270,00090,000180,000135,000
Exit allowance030,00010,00020,000
Total (3 years)558,000450,000406,000428,000

8. Decision

FieldValue
DecisionExample: Adopt OB [SaaS search vendor] for product search
Decided by[@cto] with [@product-lead], YYYY-MM-DD
ADR[ADR-NNN: link]
ConditionsExample: Committed-volume contract with a price cap; exit plan tested within 6 months
Review dateExample: At contract renewal or if search volume exceeds 3x forecast

Rationale: [Two to four sentences.] Example: OB leads the matrix and was the only option to meet every PoC target including merchandiser set-up. OD is a close second and remains the fallback; its lower lock-in did not outweigh the missing merchandiser tooling, which would cost an estimated 10 engineer-weeks to build.

9. Next steps

IDActionOwnerDue
ACT-01Example: Negotiate contract and DPA[@procurement]YYYY-MM-DD
ACT-02Example: Write ADR and update architecture description[@architect]YYYY-MM-DD
ACT-03Example: Plan migration of search UI behind a feature flag[@tech-lead]YYYY-MM-DD
[ACT-nn][Action][@name]YYYY-MM-DD

Standard and licence note

Based on common industry practice for technology selection and the weighted decision matrix (weighted-sum scoring). Original text written for this collection; example products, scores and figures are illustrative. Template licence: MIT.

Related templates