# Segmentation by Cluster Analysis Concept Card: [Market, survey or course] > A concept card on data-driven (post hoc) market segmentation for the MBA marketing analytics elective: choosing the > segmentation basis, standardising variables, k-means and Ward hierarchical clustering, choosing the number of > segments with the within-cluster sum of squares and the silhouette, testing stability, profiling and naming > segments, and turning them into a typing tool that classifies new customers. It is for second-year MBA students in > marketing analytics and for managers who commission or must act on a segmentation. It builds on the core card > `mba/mba-stp-positioning-card` (segmentation bases, the five criteria, targeting and positioning) and links to it > instead of re-teaching it. Source basis: Smith (1956), Ward (1963), MacQueen (1967), Lloyd (1982), Punj and Stewart > (1983), Hubert and Arabie (1985), Rousseeuw (1987) and Arthur and Vassilvitskii (2007), written up in original words > with fictional Northfield Outdoor Co. survey data. Every number was computed in Python (numpy, scipy) and is shown > with its Excel equivalent where Excel can compute it. > Replace every [bracketed] value and every row marked *Example*. ## Document control | Field | Value | |---|---| | Document ID | [MA-SEG-001] | | Course or project | [Marketing analytics elective / regional outdoor segmentation] | | Learner or analyst | [Your name] | | Decision owner | [@owner] | | Reviewed by | [Study group / research lead] | | Data set | [Survey of 400 regional outdoor buyers (fictional) / your data and source] | | Basis variables | [Five needs items, 1 to 7: weight saving, durability, price sensitivity, style, technical performance] | | Descriptor variables | [Age, annual category spend, channel, region] | | Method | [k-means on standardised items, 25 starts, k chosen from 2 to 7] | | Software | Excel (STANDARDIZE, SUMXMY2, MATCH, MIN); Python 3 with numpy and scipy | | Personal data check | [Privacy review reference / not applicable, anonymised data] | | Last updated | YYYY-MM-DD | | Version | [0.1] | | Classification | [Course work / internal] | ## Contents 1. When to use it 2. The idea in plain language 3. Formulas and notation 4. Assumptions and limits 5. How managers use it in practice 6. The segmentation process 7. Worked example A (fully solved): from nine respondents to 400 8. Worked example B (half solved): a bad start 9. Exercise C (unsolved): Ward's method and the typing tool 10. Connections to other cards 11. Common mistakes and misconceptions 12. Self-test questions 13. My notes and next steps 14. Sources and further reading 15. Answer key 16. Revision history 17. Standard and licence note ## When to use it **One-sentence rule:** use cluster analysis when you want the data to suggest groups of customers who are similar on the variables that drive their response to marketing, and you do not know the groups in advance. | Good uses | Poor uses | |---|---| | Finding needs-based segments from a survey of attitudes, benefits sought or conjoint part-worths | Proving that segments exist: k-means always returns k groups, even from structureless data | | Grouping customers by purchase behaviour (recency, frequency, value, categories) | Predicting a known outcome such as churn (use a supervised model, `academic/stats-logistic-regression`) | | Building a typing tool that assigns new customers to segments | Segmenting on demographics alone and hoping needs follow | | Checking whether an a priori segmentation (by region, by channel) matches how customers actually differ | Tiny samples (fewer than about 20 to 30 respondents per expected segment) | The core card `mba/mba-stp-positioning-card` treats segmentation as a choice of bases and a test against five criteria (measurable, substantial, accessible, differentiable, actionable). This card is about the analytics step in between: how to let the data propose segments, and how to judge whether the proposal deserves a place in the marketing plan. ## The idea in plain language Wendell Smith (1956) described segmentation as adjusting the offer to the demand of distinct groups, instead of bending all demand towards one product. Cluster analysis is the toolkit for finding those groups in data. Each customer is a point in a space whose axes are the **basis variables** (for example, how much they care about weight, durability, price, style and technical performance). Customers who are close together in that space want similar things; a cluster is a dense group of points that is far from other groups. **k-means** is the most used method. Pick a number of clusters $k$ and $k$ starting centres. Then repeat two steps until nothing changes: assign every customer to the nearest centre, and move every centre to the average of the customers assigned to it. Each repetition lowers the total squared distance of customers from their centres (the within-cluster sum of squares), so the algorithm always stops, but it can stop at a poor solution if the starting centres were badly placed. The method goes back to Lloyd's 1957 Bell Labs work (published in 1982) and to MacQueen (1967), who gave it its name. Running it from many starting points, with the "k-means++" seeding of Arthur and Vassilvitskii (2007), is the standard defence against bad starts. **Hierarchical clustering** works bottom-up instead: start with every customer as their own cluster and repeatedly merge the two clusters whose merger increases the within-cluster sum of squares the least (Ward 1963). The record of merges, drawn as a tree (dendrogram), shows solutions for every number of clusters at once. It is useful for small samples and for choosing $k$; k-means then refines the solution. Three decisions matter more than the algorithm. **Which variables** (needs and benefits for product and positioning decisions; behaviour for customer management; never a pile of everything available). **How to scale them**, because k-means measures distance and a variable with a large spread dominates. **How many segments**, which statistics can inform but managers must decide, because a segment the firm cannot reach or serve differently is useless however clean it looks. Punj and Stewart (1983) set out these choices for marketing research and their advice still holds: validate the solution, and do not trust a single run. ## Formulas and notation **Standardisation** of variable $v$ (so that every variable has mean 0 and standard deviation 1): $$z_{iv} = \frac{x_{iv} - \bar{x}_v}{s_v}$$ **Squared Euclidean distance** between respondent $i$ and centre $\mathbf{c}_k$: $$d^2(\mathbf{x}_i, \mathbf{c}_k) = \sum_{v=1}^{V} (x_{iv} - c_{kv})^2$$ **k-means objective** (within-cluster sum of squares) and the two alternating steps: $$\text{WSS} = \sum_{k=1}^{K} \sum_{i \in S_k} d^2(\mathbf{x}_i, \mathbf{c}_k), \qquad S_k = \{ i : d^2(\mathbf{x}_i, \mathbf{c}_k) \le d^2(\mathbf{x}_i, \mathbf{c}_j) \text{ for all } j \}, \qquad \mathbf{c}_k = \frac{1}{|S_k|} \sum_{i \in S_k} \mathbf{x}_i$$ **Share of variance explained** by the clusters, with TSS the total sum of squares around the overall mean: $$R^2_{\text{cluster}} = 1 - \frac{\text{WSS}}{\text{TSS}}$$ **Ward's merge cost** for clusters $A$ and $B$ with $n_A$, $n_B$ members and centres $\mathbf{c}_A$, $\mathbf{c}_B$: $$\Delta \text{WSS}(A, B) = \frac{n_A n_B}{n_A + n_B} \, \lVert \mathbf{c}_A - \mathbf{c}_B \rVert^2$$ **Silhouette** of respondent $i$ (Rousseeuw 1987), where $a(i)$ is the mean distance to the other members of its own cluster and $b(i)$ the smallest mean distance to the members of another cluster: $$s(i) = \frac{b(i) - a(i)}{\max\{a(i), b(i)\}}, \qquad -1 \le s(i) \le 1$$ | Symbol | Meaning | Northfield example | |---|---|---| | $x_{iv}$ | Score of respondent $i$ on basis variable $v$ | 6 on weight saving (1 to 7 scale) | | $\bar{x}_v$, $s_v$ | Sample mean and standard deviation of variable $v$ | 3.612 and 1.247 for weight saving | | $z_{iv}$ | Standardised score | 1.91 for a score of 6 on weight saving | | $\mathbf{c}_k$ | Centre (centroid) of cluster $k$: the mean of its members | (2.33, 6.00) for cluster 1 in Worked example A | | $S_k$ | The set of respondents assigned to cluster $k$ | R1, R2, R3 | | WSS | Within-cluster sum of squares | 6.6667 for the hand example with three clusters | | TSS | Total sum of squares around the overall mean | 55.5556 for the hand example | | $s(i)$ | Silhouette: near 1 well placed, near 0 on a border, below 0 probably misassigned | 0.6147 for R7 | **Excel equivalents.** Standardise: `=STANDARDIZE(B2,AVERAGE(B$2:B$401),STDEV.S(B$2:B$401))`. Squared distance from the respondent in `B2:C2` to the centre in `H2:I2`: `=SUMXMY2(B2:C2,$H$2:$I$2)`. Assign to the nearest of three centres with the distances in `D2:F2`: `=MATCH(MIN(D2:F2),D2:F2,0)`. New centre: `=AVERAGEIF($G$2:$G$10,1,B$2:B$10)`. WSS: `=SUMPRODUCT((G2:G10=1)*D2:D10)+SUMPRODUCT((G2:G10=2)*E2:E10)+SUMPRODUCT((G2:G10=3)*F2:F10)`. Iterate by copying the centres back, or with a short macro; for real data use statistical software. ## Assumptions and limits | Assumption or limit | What it means | What to do | |---|---|---| | Distance means similarity | Squared Euclidean distance treats every variable as equally important after scaling | Choose few, relevant, non-redundant basis variables; weight deliberately if at all | | Variables are on comparable scales | A variable with a larger spread dominates the distance | Standardise, or use items on the same scale and check their spreads | | Clusters are compact and roughly round | k-means favours groups of similar size and spread | Check silhouettes; consider Gaussian mixture or latent class models for elongated or very unequal groups | | Continuous variables | Means and Euclidean distances need numbers | Use other distances (Gower) or latent class models for categorical data | | The number of clusters is given | k-means does not choose $k$ | Compare $k$ = 2 to 7 on WSS, silhouette, stability and actionability | | Local optimum | Each run can stop at a poor solution | Many starts (25 or more), k-means++ seeding, keep the lowest WSS | | Outliers | A few extreme respondents can pull a centre or form their own cluster | Screen straight-liners and speeders; inspect tiny clusters before reporting them | | Structure exists | k-means always returns $k$ groups | Test stability across samples and starts; compare with a priori segments | A segmentation is a model, not a discovery of natural kinds. Its value is whether the segments respond differently to marketing and whether the firm can act on the difference. ## How managers use it in practice - **Targeting and positioning.** Segments feed the attractiveness scoring and perceptual map in `mba/mba-stp-positioning-card`; in the Northfield case the clusters below are the four segments scored there. - **Product and pricing.** Needs-based segments, or clusters of conjoint part-worths from `mba/mba-conjoint-analysis-card`, show which versions to build and which customers will pay for a premium. - **Customer management.** Behavioural clusters of the customer base, with CLV per cluster from `mba/mba-customer-lifetime-value-card`, decide service levels, offers and retention spend. - **Media and messaging.** Descriptors (age, region, channel) make segments reachable; the messaging framework (`marketing/messaging-framework`) gives each segment its own proof points. - **Typing tool.** A short set of questions, or a rule on CRM data, that assigns new customers to segments, so the segmentation lives on in operations and in every experiment (`mba/mba-marketing-experiments-card` reports results by segment). - **Privacy and fairness.** Segmenting identifiable customers is processing of personal data; in the EU and EEA the General Data Protection Regulation (Regulation (EU) 2016/679) applies, with its own rules on profiling. Check with your privacy team or counsel before you segment a customer database, and do not use sensitive characteristics as bases. This card is a learning aid, not legal advice. ## The segmentation process A segmentation project runs in a fixed order: decision, variables, preparation, clustering, choice of $k$, validation, profiling, and only then targeting. The loop back is normal: if the segments cannot be reached or served differently, change the variables or the number of segments, not the story. ```mermaid flowchart TD A["Decision the segments must serve"] --> B["Basis variables: needs, benefits or behaviour"] B --> C["Clean and standardise: screen speeders and straight-liners"] C --> D["Cluster for k = 2 to 7: k-means with many starts, Ward as a check"] D --> E["Compare k: WSS elbow, silhouette, stability"] E --> F{"Segments substantial, different and actionable?"} F -- "No" --> B F -- "Yes" --> G["Profile with descriptors and name the segments"] G --> H["Typing tool for new customers"] H --> I["Targeting and positioning: mba-stp-positioning-card"] ``` ## Worked example A (fully solved): from nine respondents to 400 **Situation (fictional).** Northfield Outdoor Co. surveyed 400 regional outdoor buyers before choosing a target for its lightweight trail range (the study behind `mba/mba-stp-positioning-card` and the Fastpack mini-case in `mba/mba-marketing-guide`). Each respondent rated five needs from 1 (unimportant) to 7 (essential): weight saving, durability, price sensitivity ("price is my main criterion"), style, and technical performance. The analytics lead first teaches the method on nine respondents and two of the items, then runs the full study. ### Step 1: nine respondents, two variables, three starting centres Both items are on the same 1 to 7 scale with similar spreads, so this small example uses the raw scores. | Respondent | Weight saving $W$ | Price sensitivity $P$ | |---|---|---| | R1 *(Example)* | 2 | 6 | | R2 *(Example)* | 3 | 7 | | R3 *(Example)* | 2 | 5 | | R4 *(Example)* | 6 | 3 | | R5 *(Example)* | 7 | 4 | | R6 *(Example)* | 6 | 2 | | R7 *(Example)* | 3 | 3 | | R8 *(Example)* | 4 | 2 | | R9 *(Example)* | 3 | 2 | Start with three centres at R1 (2, 6), R4 (6, 3) and R9 (3, 2). Squared distances, for example R7 to R4: $(3-6)^2 + (3-3)^2 = 9$ (`=SUMXMY2(B8:C8,$B$5:$C$5)`). | Respondent | to centre 1 (2, 6) | to centre 2 (6, 3) | to centre 3 (3, 2) | Assigned | |---|---|---|---|---| | R1 | 0 | 25 | 17 | 1 | | R2 | 2 | 25 | 25 | 1 | | R3 | 1 | 20 | 10 | 1 | | R4 | 25 | 0 | 10 | 2 | | R5 | 29 | 2 | 20 | 2 | | R6 | 32 | 1 | 9 | 2 | | R7 | 10 | 9 | 1 | 3 | | R8 | 20 | 5 | 1 | 3 | | R9 | 17 | 10 | 0 | 3 | WSS with the starting centres: $0 + 2 + 1 + 0 + 2 + 1 + 1 + 1 + 0 = 8$. ### Step 2: update the centres and reassign New centres are the means of the members: cluster 1 = ((2+3+2)/3, (6+7+5)/3) = (2.3333, 6.0000); cluster 2 = (6.3333, 3.0000); cluster 3 = (3.3333, 2.3333) (`=AVERAGEIF($G$2:$G$10,1,B$2:B$10)`). Recompute the distances: every respondent stays in its cluster (for example R7: 9.4444, 11.1111 and 0.5556 to the three new centres), so the algorithm has converged. Final WSS = 6.6667. The total sum of squares around the overall mean (4.0, 3.7778) is 55.5556, so the three clusters explain $1 - 6.6667/55.5556 = 88.0\%$ of the variation. ### Step 3: silhouettes For R7 in cluster 3, the mean distance to R8 and R9 is $a = (\sqrt{2} + 1)/2 = 1.2071$; the mean distance to cluster 1 is 3.1328 and to cluster 2 is 3.4285, so $b = 3.1328$ and $s = (3.1328 - 1.2071)/3.1328 = 0.6147$. The nine silhouettes range from 0.4053 (R6) to 0.7179 (R9), mean 0.5734: a reasonable structure. In words: cluster 1 cares little about weight and much about price (family campers), cluster 2 cares about weight and less about price (trail runners), cluster 3 cares about neither strongly. ### Step 4: the full survey in Python The block simulates the 400 responses with a fixed seed (in your own study, load the survey file instead), standardises the five items, runs k-means with 25 k-means++ starts for each $k$ from 2 to 7, and prints WSS and the mean silhouette. It then prints the sizes and item means of the four-cluster solution. ```python import numpy as np from scipy.cluster.vq import kmeans2 from scipy.spatial.distance import cdist rng = np.random.default_rng(400) # Needs-based items, 1 to 7: weight saving, durability, price sensitivity, style, technical performance means = {"S1": [3.0, 6.0, 6.0, 3.0, 3.0], "S2": [6.5, 5.5, 3.5, 3.5, 5.5], "S3": [3.5, 3.5, 4.5, 6.0, 2.5], "S4": [4.0, 6.5, 2.5, 2.5, 6.5]} sizes = {"S1": 152, "S2": 36, "S3": 164, "S4": 48} # 38%, 9%, 41%, 12% of 400 X = np.vstack([np.clip(np.rint(rng.normal(means[s], 0.8, (sizes[s], 5))), 1, 7) for s in means]) Z = (X - X.mean(0)) / X.std(0, ddof=1) # standardise each item def best_kmeans(Z, k, starts=25): best = None for seed in range(starts): # many starts, keep the lowest WSS C, lab = kmeans2(Z, k, minit="++", seed=seed) wss = ((Z - C[lab]) ** 2).sum() if best is None or wss < best[0]: best = (wss, C, lab) return best def silhouette(Z, lab): D = cdist(Z, Z) s = [] for i in range(len(Z)): same = lab == lab[i] a = D[i, same].sum() / max(same.sum() - 1, 1) b = min(D[i, lab == k].mean() for k in set(lab) if k != lab[i]) s.append((b - a) / max(a, b)) return np.mean(s) for k in range(2, 8): wss, C, lab = best_kmeans(Z, k) print(k, round(wss, 1), round(silhouette(Z, lab), 3)) wss, C, lab = best_kmeans(Z, 4) # the chosen solution for k in np.argsort(-np.bincount(lab)): print("cluster size", (lab == k).sum(), "item means", X[lab == k].mean(0).round(2)) ``` Output from the run: ```text 2 1257.6 0.406 3 719.9 0.432 4 620.5 0.417 5 571.1 0.321 6 523.9 0.221 7 499.1 0.213 cluster size 162 item means [3.49 3.48 4.57 5.99 2.4 ] cluster size 152 item means [2.93 5.9 5.9 3.07 2.89] cluster size 52 item means [4.08 6.42 2.75 2.65 6.29] cluster size 34 item means [6.53 5.68 3.56 3.18 5.47] ``` The item means used for standardising are 3.612, 4.970, 4.755, 4.205 and 3.355, with standard deviations 1.247, 1.480, 1.326, 1.713 and 1.633. The total sum of squares of the standardised data is $5 \times 399 = 1{,}995$. ### Step 5: how many segments? | $k$ | WSS | Share explained $1 - \text{WSS}/1{,}995$ | Mean silhouette | |---|---|---|---| | 2 | 1,257.6 | 37.0% | 0.406 | | 3 | 719.9 | 63.9% | 0.432 | | 4 | 620.5 | 68.9% | 0.417 | | 5 | 571.1 | 71.4% | 0.321 | | 6 | 523.9 | 73.7% | 0.221 | | 7 | 499.1 | 75.0% | 0.213 | ```vega-lite { "$schema": "https://vega.github.io/schema/vega-lite/v5.json", "description": "Northfield survey: share of variance explained and mean silhouette by number of clusters (fictional)", "width": 400, "height": 220, "data": {"values": [ {"k": 2, "explained": 37.0, "silhouette": 0.406}, {"k": 3, "explained": 63.9, "silhouette": 0.432}, {"k": 4, "explained": 68.9, "silhouette": 0.417}, {"k": 5, "explained": 71.4, "silhouette": 0.321}, {"k": 6, "explained": 73.7, "silhouette": 0.221}, {"k": 7, "explained": 75.0, "silhouette": 0.213} ]}, "encoding": {"x": {"field": "k", "type": "ordinal", "title": "Number of clusters k"}}, "layer": [ {"mark": {"type": "line", "point": true}, "encoding": {"y": {"field": "explained", "type": "quantitative", "title": "Variance explained (%)"}}}, {"mark": {"type": "line", "point": true, "strokeDash": [4, 3], "color": "#b5651d"}, "encoding": {"y": {"field": "silhouette", "type": "quantitative", "title": "Mean silhouette (dashed)"}}} ], "resolve": {"scale": {"y": "independent"}} } ``` The statistics do not settle the question. The big gain in fit comes from 2 to 3 clusters (from 37.0% to 63.9%); going to 4 adds 5.0 points and the silhouette falls only slightly (0.432 to 0.417); from 5 onwards the silhouette drops sharply, a sign that the extra clusters split real groups. So the candidates are 3 and 4. The three-cluster solution merges the two small technical segments (85 respondents) into one; the four-cluster solution separates them. Because the two differ on exactly the needs that drive Northfield's product decision (weight saving 6.53 against 4.08; durability and technical performance higher for the alpine group) and on spend, the four-cluster solution is the more **actionable** one. Choose $k$ = 4, and say in the report that $k$ = 3 fits almost as well. ### Step 6: profile and name the segments Descriptors were not used for clustering; they describe the clusters so that marketing can reach them. | Segment *(Example)* | Size | Share | Weight saving | Durability | Price sensitivity | Style | Technical | Mean age | Mean annual spend (USD) | |---|---|---|---|---|---|---|---|---|---| | Urban commuters (outdoor look) | 162 | 40.5% | 3.49 | 3.48 | 4.57 | 5.99 | 2.40 | 28.6 | 142.3 | | Weekend family campers | 152 | 38.0% | 2.93 | 5.90 | 5.90 | 3.07 | 2.89 | 39.8 | 165.9 | | Serious alpine hikers | 52 | 13.0% | 4.08 | 6.42 | 2.75 | 2.65 | 6.29 | 42.9 | 691.0 | | Ultralight trail runners | 34 | 8.5% | 6.53 | 5.68 | 3.56 | 3.18 | 5.47 | 32.2 | 505.5 | | All respondents | 400 | 100% | 3.61 | 4.97 | 4.76 | 4.21 | 3.36 | | 253.4 | Name each segment by its defining needs, not by a demographic. The runners are the smallest group but combine the highest need for weight saving with high durability needs, which is the Fastpack opportunity; the alpine hikers spend the most per head. These are the four segments the STP card scores; the marketing guide's Exhibit 1 reports their shares in the population (urban commuters 41%, family campers 38%, alpine hikers 12%, trail runners 9%), and the clusters recover them to within one percentage point (40.5%, 38.0%, 13.0% and 8.5%). ### Step 7: is it stable? Because the data were simulated, the true segment of each respondent is known, and the chosen solution recovers it for 390 of 400 respondents (97.5%; adjusted Rand index 0.932, Hubert and Arabie 1985). Real data have no answer key, so check stability instead: 20 single-start runs of k-means with different seeds gave WSS between 620.5 and 686.2, and agreed with the best solution with an adjusted Rand index between 0.717 and 1.000 (median 0.890; half of them above 0.95). One run is not enough; five further independent blocks of 25 starts each all found the same best solution (WSS 620.5). Also re-run on a random half of the sample and check that the segments and their profiles reappear. ### Step 8: the typing tool and the answer to the CMO To classify a new customer, standardise their five answers with the survey means and standard deviations and assign them to the nearest segment centre. On the hand example, a new respondent with $(W, P) = (5, 3)$ has squared distances 16.11, 1.78 and 3.22 to the three centres, so belongs to cluster 2 (`=MATCH(MIN(D12:F12),D12:F12,0)`). **Recommendation.** Report four needs-based segments with their sizes, profiles and the honest caveat that three segments fit almost as well. Use them for targeting in `mba/mba-stp-positioning-card`, put the five questions in the CRM onboarding survey as a typing tool, and report every future experiment by segment. ## Worked example B (half solved): a bad start Use the nine respondents from Step 1, but start with centres at R1 (2, 6), R2 (3, 7) and R4 (6, 3). | Respondent | to R1 (2, 6) | to R2 (3, 7) | to R4 (6, 3) | Assigned | |---|---|---|---|---| | R1 | 0 | 2 | 25 | 1 | | R2 | 2 | 0 | 25 | 2 | | R3 | 1 | 5 | 20 | [ ] | | R4 | 25 | 25 | 0 | 3 | | R5 | 29 | 25 | 2 | 3 | | R6 | 32 | 34 | 1 | 3 | | R7 | 10 | 16 | [ ] | 3 | | R8 | 20 | 26 | 5 | 3 | | R9 | 17 | 25 | 10 | 3 | Questions: 1. Fill the two gaps. Compute the new centres: cluster 1 = [ ], cluster 2 = (3, 7), cluster 3 = [ ]. 2. Reassign with the new centres. Does any respondent move? What is the final WSS? 3. Compare it with the WSS of 6.6667 in Step 2. Why did k-means stop at a worse solution, and how do the many-start and k-means++ defences in Step 4 prevent this? 4. What would a manager conclude from this solution about "price-focused" and "weight-focused" customers, and why would that conclusion be wrong? ## Exercise C (unsolved): Ward's method and the typing tool 1. Apply Ward's method to the nine respondents of Step 1. Using the merge cost formula, show that the first three merges are R1 with R3, R7 with R9 and R4 with R6, each with a cost of 0.5 (R8 with R9 ties at 0.5 too; either choice leads to the same result). Continue until three clusters remain. Are they the same as the k-means clusters? 2. What is the merge cost of going from three clusters to two, and from two to one? What does the size of these jumps say about the number of clusters? 3. Classify two new respondents, (3, 5) and (4, 4), with the final centres of Step 2. Which one is a borderline case, and how would you handle borderline customers in a CRM typing tool? 4. For your own course or company: list five basis variables you would use, the decision they serve, and two descriptors that would make the segments reachable. ## Connections to other cards | Card | Connection | |---|---| | `mba/mba-stp-positioning-card` | Core card this one builds on: bases, the five criteria, scoring segments, positioning | | `mba/mba-conjoint-analysis-card` | Individual part-worths are strong clustering variables for needs-based segments | | `mba/mba-customer-lifetime-value-card` | Behavioural segments and CLV per segment for customer management | | `mba/mba-marketing-experiments-card` | Testing whether segments really respond differently (heterogeneous treatment effects) | | `mba/mba-brand-equity-card` | Brand perceptions often differ sharply by segment | | `mba/mba-marketing-analytics-guide` | The elective as a whole and its session plan | | `academic/stats-reliability-factor-analysis` | Reducing many attitude items to a few factors before clustering, and its risks | | `academic/stats-eda-report`, `academic/stats-data-cleaning-log` | Screening and preparing survey data | ## Common mistakes and misconceptions 1. **Clustering on everything available.** Adding variables that do not drive response adds noise and hides real structure. Choose the basis for the decision. 2. **Forgetting to standardise.** Annual spend in dollars next to 1 to 7 ratings makes the segmentation a spend segmentation. 3. **Trusting one run.** k-means stops at local optima (Worked example B). Use many starts and keep the lowest WSS. 4. **Letting the statistics choose $k$ alone.** Fit indices often disagree and flatten out; actionability and stability decide. 5. **Treating segments as real types.** Segments are a simplification; many customers sit near borders. Report the share of low-silhouette respondents. 6. **Naming segments by demographics.** "Young urban women" is a descriptor, not a need. Name segments by what they want, describe them by who they are. 7. **Clustering demographics and hoping needs follow.** Demographic segments rarely differ in their response to product and price. 8. **Tandem analysis without checking.** Running factor analysis first and clustering the factor scores can discard exactly the dimensions that separate segments. Compare with clustering the items. 9. **No typing tool.** A segmentation that cannot classify new customers is used once, in a slide. 10. **Ignoring privacy.** Segmenting identifiable customers is profiling under data protection laws in many jurisdictions; involve your privacy team. ## Self-test questions 1. State the two alternating steps of k-means and explain why WSS can never increase from one iteration to the next. 2. Compute the squared distance from respondent (5, 3) to the centre (6.3333, 3.0000). 3. A respondent scores 6 on weight saving. With a mean of 3.612 and a standard deviation of 1.247, what is the standardised score, and what is the Excel formula? 4. Why does k-means need many starts, and what does k-means++ change? 5. Show how R7's silhouette of 0.6147 is computed. What would a negative silhouette mean? 6. What share of variance do four clusters explain in the 400-respondent survey, and why is that number not a reason to choose four? 7. The silhouette is highest at $k$ = 3 but the analyst chose $k$ = 4. Give the argument for and against. 8. What is the difference between basis variables and descriptor variables? Give one example of each from the Northfield survey. 9. Why should segment names describe needs rather than demographics? 10. Name two ways to test whether a segmentation is stable when there is no answer key. 11. Why is Ward's method a natural partner for k-means? ## My notes and next steps | Date | Note | Next step | |---|---|---| | YYYY-MM-DD | *Example:* I can run k-means by hand and explain why k = 4 was chosen | Build the typing tool in Excel | | YYYY-MM-DD | [What still confuses me] | [Question for office hours] | ## Sources and further reading - Smith, W. R. (1956). Product differentiation and market segmentation as alternative marketing strategies. *Journal of Marketing*, 21(1), 3-8. - Ward, J. H. (1963). Hierarchical grouping to optimize an objective function. *Journal of the American Statistical Association*, 58(301), 236-244. - MacQueen, J. B. (1967). Some methods for classification and analysis of multivariate observations. In *Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability*, 1, 281-297. University of California Press. - Lloyd, S. P. (1982). Least squares quantization in PCM. *IEEE Transactions on Information Theory*, 28(2), 129-137. - Punj, G. and Stewart, D. W. (1983). Cluster analysis in marketing research: review and suggestions for application. *Journal of Marketing Research*, 20(2), 134-148. - Hubert, L. and Arabie, P. (1985). Comparing partitions. *Journal of Classification*, 2(1), 193-218. - Rousseeuw, P. J. (1987). Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. *Journal of Computational and Applied Mathematics*, 20, 53-65. - Arthur, D. and Vassilvitskii, S. (2007). k-means++: the advantages of careful seeding. In *Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms*, 1027-1035. - Textbooks (further reading, by name only): Wedel and Kamakura, *Market Segmentation: Conceptual and Methodological Foundations*; Hair, Black, Babin and Anderson, *Multivariate Data Analysis*; James, Witten, Hastie and Tibshirani, *An Introduction to Statistical Learning*. - Open resources: *An Introduction to Statistical Learning*, whose authors offer the R and Python editions as free PDFs on the book's website (https://www.statlearning.com, checked October 2026; the site states no reuse licence, so treat it as all rights reserved), chapter on unsupervised learning; the SciPy documentation for `scipy.cluster.vq.kmeans2` and `scipy.cluster.hierarchy.linkage`. - Regulation (EU) 2016/679 (General Data Protection Regulation), EUR-Lex, for the rules on processing personal data and profiling; check the consolidated text and local guidance. ## Answer key **Worked example B.** (1) R3 goes to cluster 1 (distance 1); R7's distance to R4 is 9, so R7 goes to cluster 3. New centres: cluster 1 = (2.0, 5.5), cluster 2 = (3.0, 7.0), cluster 3 = (4.8333, 2.6667). WSS with the starting centres was 28. (2) Nobody moves; final WSS = 18.6667. (3) It is much worse than 6.6667: two of the three starting centres were in the same natural group, so that group was split in two and the other two groups were forced together. k-means only improves locally and cannot move a centre across the space. Many starts and k-means++ seeding (which picks starting centres far apart with high probability) make such a start unlikely to be the final answer. (4) It would see one large "everyone else" segment of six customers and two tiny price-focused segments, and conclude that weight saving is not a segment of its own; in fact the runners (R4 to R6) are a clear group. **Exercise C (parts 1 to 3).** (1) Merges and costs (increase in WSS): R1+R3 0.5, R7+R9 0.5, R4+R6 0.5, R8 joins {R7, R9} 0.8333, R2 joins {R1, R3} 2.1667, R5 joins {R4, R6} 2.1667. The three clusters {R1, R2, R3}, {R4, R5, R6}, {R7, R8, R9} are the k-means clusters, with WSS 6.6667. (2) Three to two: 14.1667 (the runners and the third group merge); two to one: 34.7222. The jump from 2.1667 to 14.1667 says that three clusters is a natural stopping point. (3) (3, 5): squared distances 1.4444, 15.1111 and 7.2222, cluster 1, clear. (4, 4): 6.7778, 6.4444 and 3.2222, cluster 3, but its squared distance to the second-nearest centre is only twice the nearest one; flag such customers as "borderline" and confirm with one or two extra questions or with behaviour. Part 4 is for your own work. **Self-test.** 1. Assign each point to its nearest centre; move each centre to the mean of its points. Reassignment cannot increase any point's distance to its centre, and the mean minimises the sum of squared distances within a group, so neither step can raise WSS. 2. $(5 - 6.3333)^2 + (3 - 3)^2 = 1.78$. 3. $(6 - 3.612)/1.247 = 1.91$; `=STANDARDIZE(6,3.612,1.247)`. 4. Each run converges to a local optimum that depends on the starting centres. k-means++ chooses each new starting centre with probability proportional to its squared distance from the centres already chosen, which spreads them out; many starts then keep the lowest WSS. 5. $a$ = mean distance to R8 and R9 = $(\sqrt{2} + 1)/2 = 1.2071$; $b$ = smallest mean distance to another cluster = 3.1328; $s = (3.1328 - 1.2071)/3.1328 = 0.6147$. A negative value means the respondent is on average closer to another cluster than to its own: probably misassigned. 6. $1 - 620.5/1{,}995 = 68.9\%$. More clusters always explain more variance (one cluster per respondent explains 100%), so the level alone cannot choose $k$. 7. For: the fourth cluster separates two groups with different needs and spend that matter for the decision, and the silhouette falls only from 0.432 to 0.417. Against: three is the best fit on silhouette and simpler to run; a small segment of 34 is costly to serve separately. 8. Basis variables define the clusters (the five needs items); descriptors describe them afterwards (age, annual spend). 9. Needs explain why customers respond differently to product, price and message; demographics only help to reach them. Two people of the same age can belong to different segments. 10. Any two of: many random starts compared with the adjusted Rand index; split-half replication; re-running with a different algorithm (Ward); checking the segments on a new sample later. 11. Ward's method minimises the same WSS criterion when it merges, gives a full tree for choosing $k$, and its solution makes a good set of starting centres for k-means. ## Revision history | Version | Date | Author | Change summary | |---|---|---|---| | 0.1 | YYYY-MM-DD | [Your name] | First draft from the template | | 0.2 | YYYY-MM-DD | [Your name] | *Example:* Added split-half stability check and the typing tool | ## Standard and licence note Based on the segmentation concept of Smith (1956), the clustering methods of Ward (1963), MacQueen (1967), Lloyd (1982) and Arthur and Vassilvitskii (2007), the validation measures of Hubert and Arabie (1985) and Rousseeuw (1987), and the marketing research practice described by Punj and Stewart (1983). The text, data, code and exercises are original and fictional; no textbook, published case or course material was copied or paraphrased. This template is not affiliated with or endorsed by any business school, publisher, author or software vendor. Data protection rules are named as facts only; check the current law and your organisation's policy. Template licence: MIT.