Technology Selection and Build vs Buy Template

Build, buy, open source or managed service evaluation with weighted decision matrix, PoC results, risks, cost and decision

Architektur & SystemdesignStandardtechnology selection mathAktualisiert 20.10.2026

Im Editor öffnen.md herunterladen

Anleitung: Use before committing to a product or platform for a capability. Fill the problem, must-haves and criteria weights first and agree them before scoring. The main pitfall is adjusting weights after seeing scores; run the sensitivity check and record the final decision in an ADR.

Vorschau

Technology Selection: [Capability or component name]

Technology selection and build-versus-buy evaluation for one capability: problem and requirements, candidate options (build, buy, open source, managed service), a weighted decision matrix, proof-of-concept results, risks, a cost summary and the decision with its ADR link. Based on the common weighted decision matrix practice. Replace every [bracketed] value and every row marked Example; the worked example evaluates a product search capability.

Field	Value
Document ID	[TS-001]
Capability	[e.g. Product search for the web shop]
Version	[0.1]
Status	[Draft / In review / Decided / Superseded]
Owner	[@architect]
Decision makers	[@cto], [@product-lead], [@platform-lead]
Evaluation period	YYYY-MM-DD to YYYY-MM-DD
Related	[ADR-NNN], [TCO analysis], [Architecture review request]

1. Problem and context

[Describe the problem in two or three paragraphs. Include the current state, the pain it causes and the trigger for this evaluation.]

Example: The web shop uses SQL LIKE queries for product search. Median search latency is 1.8 s at peak, typo tolerance is absent, and analytics show that 31% of searches return zero results. Search-led conversion is 40% below the industry benchmark our product team tracks. The catalogue will grow from 120,000 to 600,000 SKUs after the marketplace launch in Q3, which the current approach cannot support.

Constraint	Value
Decision needed by	YYYY-MM-DD
Budget envelope (3 years)	[e.g. USD 500,000]
Mandatory compliance	[e.g. GDPR, data stored in the EU]
Team capacity	[e.g. 2 engineers for 3 months, 0.5 FTE run]
Strategic principles	[e.g. "Buy before build for non-differentiating capabilities"]

2. Requirements and evaluation criteria

2.1 Must-have requirements (knock-out)

Any option that fails one of these is excluded and not scored.

ID	Requirement	Verification
TS-MH-01	Example: p95 search latency <= 200 ms at 300 queries per second	PoC load test
TS-MH-02	Example: Data processed and stored in the EU only	Contract and DPA review
TS-MH-03	Example: Typo tolerance and synonyms configurable by merchandisers	PoC demo
TS-MH-04	Example: Index update visible within 60 s of a price change	PoC test
[TS-MH-nn]	[Requirement]	[Method]

2.2 Scored criteria and weights

Weights are agreed before scoring and sum to 1.00. Scores use a 1 to 5 scale defined in the scale table.

ID	Criterion	What it measures	Weight $w_i$
C1	Functional fit	Relevance quality, facets, merchandising tools	0.30
C2	Cost (3-year TCO)	Lower cost scores higher	0.20
C3	Security and compliance	Certifications, data residency, access control	0.15
C4	Operability	Effort to run, upgrade, monitor and scale	0.15
C5	Team skills fit	Existing skills, learning curve, hiring market	0.10
C6	Exit and vendor risk	Lock-in, data portability, vendor viability	0.10
	Total		1.00

Score	Meaning
5	Exceeds the need with evidence
4	Fully meets the need
3	Meets the need with workarounds
2	Partly meets the need; significant gaps
1	Does not meet the need

3. Options considered

ID	Option	Archetype	Description	Must-haves met?
O0	Do nothing	Baseline	Example: Keep SQL `LIKE` search; add caching	No (TS-MH-01, TS-MH-03)
OA	Build in-house	Build	Example: Search service on PostgreSQL full-text search with custom ranking	Yes
OB	[SaaS search vendor]	Buy	Example: Hosted search API with merchandising console, EU region	Yes
OC	[Open-source engine], self-hosted	Open source	Example: Open-source search engine on our Kubernetes cluster	Yes
OD	[Managed engine service]	Managed service	Example: Cloud provider's managed open-source search engine	Yes

3.1 Option notes

For each option, record what was learned beyond the matrix.

OA Build: [Strengths, gaps, effort estimate.] Example: Full control and no licence fee; ranking and typo tolerance need about 14 engineer-weeks; merchandiser tooling would be built from scratch.
OB Buy: [Notes.] Example: Best relevance out of the box; pricing is per 1,000 searches, so cost rises with traffic; export of synonyms and rules is possible through the API.
OC Open source: [Notes.] Example: Mature engine and large community; we would own cluster upgrades and on-call; no merchandiser UI.
OD Managed service: [Notes.] Example: Same engine as OC with upgrades and snapshots handled by the provider; still no merchandiser UI.

4. Weighted decision matrix

The weighted score of option $j$ is the sum over criteria of weight times score:

S_j = \sum_{i=1}^{n} w_i \, s_{ij}, \qquad \sum_{i=1}^{n} w_i = 1

where $w_i$ is the weight of criterion $i$ and $s_{ij}$ is the score of option $j$ on criterion $i$ (1 to 5). The maximum possible score is 5.00.

Criterion	$w_i$	OA Build	OB Buy	OC Open source	OD Managed
C1 Functional fit	0.30	5	4	3	4
C2 Cost (3-year TCO)	0.20	2	3	4	3
C3 Security and compliance	0.15	3	4	3	4
C4 Operability	0.15	2	5	2	4
C5 Team skills fit	0.10	4	4	2	3
C6 Exit and vendor risk	0.10	5	2	4	3
Weighted score $S_j$	1.00	3.55	3.75	3.05	3.60
Rank		3	1	4	2

Example arithmetic for OB Buy: $S_B = 0.30(4) + 0.20(3) + 0.15(4) + 0.15(5) + 0.10(4) + 0.10(2) = 1.20 + 0.60 + 0.60 + 0.75 + 0.40 + 0.20 = 3.75$ .

Example arithmetic for OD Managed: $S_D = 1.20 + 0.60 + 0.60 + 0.60 + 0.30 + 0.30 = 3.60$ .

4.1 Sensitivity check

A difference of less than about 0.25 between the top two options is within scoring noise. Re-run the matrix with plausible alternative weights and record whether the ranking changes.

Scenario	Weight change	OA	OB	OC	OD	Leader
Baseline	As in 2.2	3.55	3.75	3.05	3.60	OB
Example: Exit risk matters more	C6 0.20, C1 0.20	3.55	3.55	3.15	3.50	OA and OB tie
[Scenario]	[Change]	[ ]	[ ]	[ ]	[ ]	[ ]

The second row is computed by moving 0.10 of weight from C1 to C6: for OB, $3.75 - 0.10(4) + 0.10(2) = 3.55$ ; for OA, $3.55 - 0.10(5) + 0.10(5) = 3.55$ .

5. Proof-of-concept results

PoC scope: [dataset, query set, load profile, duration, team]. Example: 120,000 real SKUs, 5,000 logged queries replayed, 300 QPS for 30 minutes, two engineers for two weeks per option.

Test	Target	OA Build	OB Buy	OC Open source	OD Managed
p95 latency at 300 QPS	<= 200 ms	Example: 240 ms	48 ms	95 ms	110 ms
Zero-result rate on replay	<= 10%	14%	6%	9%	9%
Index freshness after update	<= 60 s	5 s	20 s	2 s	2 s
Merchandiser rule set-up	< 1 day	Not built	2 hours	Not available	Not available
[Test]	[Target]	[Result]	[Result]	[Result]	[Result]

Findings: [Summarise what surprised the team and which scores changed after the PoC.] Example: OA missed the latency target until a dedicated read replica was added; this cost is reflected in C2.

6. Risks

ID	Option	Risk	Likelihood	Impact	Mitigation	Owner
RSK-01	OB	Example: Per-search pricing grows faster than revenue during sales peaks	Medium	High	Negotiate a committed-volume tier; cache popular queries	[@name]
RSK-02	OB	Example: Vendor acquisition changes the roadmap	Low	Medium	Keep rules and synonyms exportable; annual exit drill	[@name]
RSK-03	OA	Example: Relevance tuning depends on one engineer	High	High	Pair work; document ranking model	[@name]
[RSK-nn]	[Option]	[Risk]	[L/M/H]	[L/M/H]	[Mitigation]	[@name]

7. Cost summary

Detailed figures are in [TCO analysis link]. Currency: [USD], horizon [3] years, nominal values.

Cost item	OA Build	OB Buy	OC Open source	OD Managed
Implementation (one-off)	Example: 180,000	60,000	90,000	75,000
Licences or subscription (3 years)	0	270,000	0	0
Infrastructure (3 years)	108,000	0	126,000	198,000
People to run (3 years)	270,000	90,000	180,000	135,000
Exit allowance	0	30,000	10,000	20,000
Total (3 years)	558,000	450,000	406,000	428,000

8. Decision

Field	Value
Decision	Example: Adopt OB [SaaS search vendor] for product search
Decided by	[@cto] with [@product-lead], YYYY-MM-DD
ADR	[ADR-NNN: link]
Conditions	Example: Committed-volume contract with a price cap; exit plan tested within 6 months
Review date	Example: At contract renewal or if search volume exceeds 3x forecast

Rationale: [Two to four sentences.] Example: OB leads the matrix and was the only option to meet every PoC target including merchandiser set-up. OD is a close second and remains the fallback; its lower lock-in did not outweigh the missing merchandiser tooling, which would cost an estimated 10 engineer-weeks to build.

9. Next steps

ID	Action	Owner	Due
ACT-01	Example: Negotiate contract and DPA	[@procurement]	YYYY-MM-DD
ACT-02	Example: Write ADR and update architecture description	[@architect]	YYYY-MM-DD
ACT-03	Example: Plan migration of search UI behind a feature flag	[@tech-lead]	YYYY-MM-DD
[ACT-nn]	[Action]	[@name]	YYYY-MM-DD

Standard and licence note

Based on common industry practice for technology selection and the weighted decision matrix (weighted-sum scoring). Original text written for this collection; example products, scores and figures are illustrative. Template licence: MIT.