# Nonparametric Rank Tests Method Card: [Your study question or course] > A method card for the rank-based tests that replace t-tests and ANOVA when the data are ordinal, strongly skewed or > full of outliers: the Mann-Whitney U test (Wilcoxon rank-sum), the Wilcoxon signed-rank test, the sign test, the > Kruskal-Wallis test with Dunn post hoc comparisons (Holm adjustment) and the Friedman test. It is for students > preparing for an exam, master's and PhD students with small or awkward thesis samples, and analysts who need robust > comparisons. It speeds up learning by showing what each test really tests (not always the median), where R and Python > disagree by default, and how to report an effect size, with one example fully solved, one half solved and one left for > you. The worked examples use fictional data; replace them with your own. > Source basis: Wilcoxon (1945), Mann and Whitney (1947), Kruskal and Wallis (1952), Friedman (1937), Dunn (1964), > Holm (1979), the NIST/SEMATECH e-Handbook of Statistical Methods and the R and SciPy documentation, in original words. ## Document control | Field | Value | |---|---| | Course or project | [STAT 201 / thesis chapter 4 / client project] | | Learner or analyst | [Your name] | | Data set | [Brightwell Bakery waiting times, Harbour City Library loan durations, Lakemoor University ratings (all fictional) / your data set] | | Software and versions | R 4.5.1 (stats); Python 3.14 with numpy 2.5.4, scipy 1.18.1, statsmodels 0.15.0 | | Date | YYYY-MM-DD | | Version | [0.1] | ## When to use it **One-sentence rule:** use a rank test when the outcome is ordinal, or continuous but so skewed or outlier-prone that the mean is a poor summary in a small sample, and you are willing to ask "do values in one condition tend to be larger than in another?" rather than "do the means differ?". | Design | Parametric test | Rank-based alternative | |---|---|---| | Two independent groups | Welch t-test | Mann-Whitney U (Wilcoxon rank-sum) | | Two paired measurements | Paired t-test | Wilcoxon signed-rank; sign test (fewest assumptions) | | Three or more independent groups | One-way ANOVA or Welch ANOVA | Kruskal-Wallis, then Dunn post hoc with Holm adjustment | | Three or more repeated measurements (blocks) | Repeated-measures ANOVA | Friedman test | | Use when | Do not use when | |---|---| | Outcome is ordinal (ratings, ranks, Likert items) | Outcome is binary or a count of categories (use proportion or chi-square tests) | | Small samples with skew or outliers you cannot remove | You must estimate the difference in means (rank tests do not test means) | | You want a test that does not depend on the exact values, only their order | Groups differ mainly in spread, not location, and you want to test the centre | | Paired differences are asymmetric (sign test) | Observations are clustered or dependent within groups | **Alternatives:** a permutation test or bootstrap CI on the statistic you actually care about, such as the difference in means or medians (`academic/stats-resampling-methods`); a transformation followed by a t-test (`academic/stats-data-transformations`); ordinal logistic regression when you need covariates; Welch's t-test, which is often fine for moderately skewed data when each group has 30 or more observations (a rule of thumb). ## The question and the data **Research question for Worked example A1 (two independent groups):** Brightwell Bakery (fictional) has two branches. Do customers at the Harbour branch wait less time to be served than customers at the Station branch? Waiting times are right-skewed (most people wait a few minutes, a few wait much longer), and the samples are small. | Variable | Role | Measurement level | Values | |---|---|---|---| | `wait` | Outcome | Ratio (minutes), right-skewed | Time from joining the queue to being served | | `branch` | Grouping | Nominal, 2 levels | Harbour (n = 9), Station (n = 10) | | Design | Independent groups | Different customers at each branch, one visit each | Weekday lunchtime | | Branch | Waiting times (minutes) | n | Median | |---|---|---|---| | Harbour | 2.1, 3.4, 1.8, 4.0, 2.7, 3.1, 9.5, 2.4, 3.8 | 9 | 3.1 | | Station | 4.2, 5.1, 3.6, 6.8, 4.9, 12.3, 5.6, 4.4, 7.2, 3.9 | 10 | 5.0 | **Research question for Worked example A2 (three independent groups):** Harbour City Library (fictional) wants to know whether loan durations (days until a book is returned) differ between its Central, North and East branches. | Branch | Loan duration (days) | n | Median | |---|---|---|---| | Central | 7, 12, 9, 14, 10, 21, 8, 11 | 8 | 10.5 | | North | 14, 18, 16, 25, 13, 19, 15, 28 | 8 | 17.0 | | East | 10, 9, 12, 17, 8, 11, 13, 30 | 8 | 11.5 | The chart shows the waiting times. Each branch has one long wait (9.5 and 12.3 minutes) that would pull a mean upward; a rank test only cares that those values are the largest, not how large they are. ```vega-lite { "$schema": "https://vega.github.io/schema/vega-lite/v5.json", "description": "Brightwell Bakery waiting times by branch (fictional data)", "width": 380, "height": 140, "data": {"values": [ {"branch": "Harbour", "wait": 2.1}, {"branch": "Harbour", "wait": 3.4}, {"branch": "Harbour", "wait": 1.8}, {"branch": "Harbour", "wait": 4.0}, {"branch": "Harbour", "wait": 2.7}, {"branch": "Harbour", "wait": 3.1}, {"branch": "Harbour", "wait": 9.5}, {"branch": "Harbour", "wait": 2.4}, {"branch": "Harbour", "wait": 3.8}, {"branch": "Station", "wait": 4.2}, {"branch": "Station", "wait": 5.1}, {"branch": "Station", "wait": 3.6}, {"branch": "Station", "wait": 6.8}, {"branch": "Station", "wait": 4.9}, {"branch": "Station", "wait": 12.3}, {"branch": "Station", "wait": 5.6}, {"branch": "Station", "wait": 4.4}, {"branch": "Station", "wait": 7.2}, {"branch": "Station", "wait": 3.9} ]}, "encoding": { "y": {"field": "branch", "type": "nominal", "title": null}, "x": {"field": "wait", "type": "quantitative", "title": "Waiting time (minutes)", "scale": {"domain": [0, 13]}} }, "layer": [ {"mark": {"type": "point", "filled": true, "opacity": 0.6, "size": 60}}, {"mark": {"type": "tick", "thickness": 3, "size": 30, "color": "black"}, "encoding": {"x": {"aggregate": "median", "field": "wait", "type": "quantitative"}}} ] } ``` ## Assumptions and how to check each | Assumption | How to check | R | Python | If violated | |---|---|---|---|---| | Independence within and between groups (or of pairs) | Design: one observation per unit, no clustering | Read the protocol | Read the protocol | Mixed models or cluster-aware methods | | Outcome at least ordinal | Measurement level in the codebook | `str(df)` | `df.dtypes` | Use a test for categorical data | | Similar distribution shapes (only to read the result as a shift in medians) | Plot both groups; compare spread and skew | `boxplot(wait ~ branch)` | `df.boxplot("wait", by="branch")` | Report as stochastic dominance, P(X > Y), not "medians differ" | | Symmetric differences (signed-rank only) | Histogram or dot plot of the paired differences | `hist(after - before)` | `np.histogram(d)` | Use the sign test, which needs no symmetry | | Few ties (affects exact p-values) | Count tied values | `table(x)[table(x) > 1]` | `pd.Series(x).value_counts()` | Use the normal approximation with tie correction or a permutation p-value; report which | ## How it works **Plain language.** Put all observations in one line from smallest to largest and replace each value by its position (its rank). Tied values share the average of the ranks they occupy. If the groups come from the same distribution, the ranks should be shuffled evenly among them, so each group's average rank should be close to the overall average. If one group collects most of the low ranks, that is evidence its values tend to be smaller. Because only the order is used, an extreme value counts the same whether it is 9.5 or 95 minutes. **What Mann-Whitney actually tests.** The null hypothesis is that the two distributions are identical. The test is sensitive to the alternative that $P(X > Y) \neq 0.5$, that is, a randomly chosen value from one group tends to be larger than one from the other (stochastic dominance). It is a test of medians **only** if you add the assumption that the two distributions have the same shape and differ only by a shift. With different shapes, two groups can have equal medians and still give a significant Mann-Whitney test, or different medians and a non-significant one. **Mann-Whitney U.** With $R_1$ the sum of the ranks of group 1 in the combined sample: $$ U_1 = R_1 - \frac{n_1(n_1 + 1)}{2}, \qquad U_2 = n_1 n_2 - U_1, \qquad \hat{P}(X_1 > X_2) = \frac{U_1}{n_1 n_2} $$ For large samples (or with ties) the normal approximation uses $z = \dfrac{U_1 - n_1 n_2/2 \pm 0.5}{\sqrt{n_1 n_2 (N + 1)/12}}$, where the $\pm 0.5$ is the continuity correction and the variance is reduced when there are ties. **Rank-biserial correlation** (effect size, from $-1$ to $+1$): $$ r_{rb} = \frac{2U_1}{n_1 n_2} - 1 = 2\hat{P}(X_1 > X_2) - 1 $$ **Wilcoxon signed-rank.** Drop zero differences, rank the absolute differences, then add the ranks of the positive differences ($T^+$) and of the negative ones ($T^-$). The matched-pairs rank-biserial is $r = (T^+ - T^-)/(T^+ + T^-)$. **Sign test:** count positive differences among the non-zero ones and compare with a Binomial($n$, 0.5) distribution. **Kruskal-Wallis** with $k$ groups, $N$ observations, $R_j$ the rank sum of group $j$ (before tie correction): $$ H = \frac{12}{N(N + 1)} \sum_{j=1}^{k} \frac{R_j^2}{n_j} - 3(N + 1), \qquad df = k - 1, \qquad \eta^2_H = \frac{H - k + 1}{N - k} $$ **Dunn's post hoc test** compares mean ranks $\bar{R}_a$ and $\bar{R}_b$ from the same joint ranking used by Kruskal-Wallis: $z = (\bar{R}_a - \bar{R}_b)/\sqrt{\sigma^2 (1/n_a + 1/n_b)}$, with $\sigma^2 = \dfrac{N(N+1)}{12} - \dfrac{\sum (t^3 - t)}{12(N - 1)}$ ($t$ = size of each tie group). Adjust the p-values with Holm's method. **Friedman** with $n$ blocks (raters, subjects) and $k$ conditions, ranks within each block, $R_j$ the column rank sum: $$ \chi^2_F = \frac{12}{n k (k + 1)} \sum_{j=1}^{k} R_j^2 - 3n(k + 1), \qquad df = k - 1, \qquad W = \frac{\chi^2_F}{n(k - 1)} $$ where Kendall's $W$ (0 to 1) is the effect size. | Symbol | Meaning | |---|---| | $n_1$, $n_2$, $N$ | Sample size n of each group, total | | $R_1$, $R_j$ | Rank sum of group 1, of group (or condition) $j$ | | $U_1$, $U_2$ | Mann-Whitney statistic for group 1, for group 2 | | $T^+$, $T^-$ | Sum of ranks of positive, negative paired differences | | $r_{rb}$ | Rank-biserial correlation | | $H$, $\chi^2_F$ | Kruskal-Wallis and Friedman statistics (compared with chi-square, df = $k - 1$) | | $\eta^2_H$, $W$ | Kruskal-Wallis eta-squared, Kendall's W | | $t$ | Number of observations in one tie group | ```mermaid flowchart TD A["Ordinal or skewed outcome"] --> B{"Design?"} B -->|"Two independent groups"| C["Mann-Whitney U"] B -->|"Two paired measurements"| D{"Differences roughly symmetric?"} D -->|"Yes"| E["Wilcoxon signed-rank"] D -->|"No"| F["Sign test"] B -->|"Three or more independent groups"| G["Kruskal-Wallis"] G -->|"Significant"| H["Dunn post hoc with Holm adjustment"] B -->|"Three or more repeated measures"| I["Friedman test"] I -->|"Significant"| J["Pairwise signed-rank tests with Holm"] C --> K["Report statistic, exact or approximate p, effect size, Hodges-Lehmann shift"] E --> K F --> K H --> K J --> K ``` ### R versus Python defaults This table is the main practical lesson of this card. The same data can give different statistics and p-values in R and SciPy, and neither is wrong: they make different default choices. | Topic | R (`stats`) | Python (`scipy.stats`) | What to do | |---|---|---|---| | Mann-Whitney statistic | `wilcox.test` prints `W`, which is $U$ of the first sample (not a rank sum) | `mannwhitneyu` returns $U$ of the first sample (SciPy 1.7 and later; older versions returned the smaller U) | Swap the order of the groups and the statistic becomes $n_1 n_2 - U$; say which group it refers to | | Textbook "W" | Some textbooks report the Wilcoxon rank sum $R_1$, which is $U_1 + n_1(n_1+1)/2$; SAS reports the rank sum of the smaller sample; other packages and older SciPy versions report the smaller U | Same | Do not compare a "W" across sources without checking the definition | | Mann-Whitney p-value method | Exact when both n < 50 and no ties; otherwise normal approximation with continuity correction | `method="auto"`: exact only when one sample has n of 8 or less and there are no ties; otherwise normal approximation with continuity correction | Set `exact=` in R and `method=` in Python explicitly | | Signed-rank statistic | `V` = $T^+$, sum of ranks of positive differences | Two-sided: the smaller of $T^+$ and $T^-$ | Report $T^+$ or say which you report | | Signed-rank p-value method | No ties or zeros: exact when n < 50. With ties or zeros: normal approximation with continuity correction (with a warning); zeros dropped first | `method="auto"`: no ties or zeros, exact when n of 50 or less; with ties or zeros, exhaustive permutation p-value if n (all pairs, zeros included) is 13 or less, otherwise normal approximation; `correction=False` by default; zeros dropped (`zero_method="wilcox"`) | Set `method` and `correction` to match, or report which method was used | | Kruskal-Wallis, Friedman | Chi-square approximation with tie correction | Same (`kruskal`, `friedmanchisquare`) | Results agree | | Dunn post hoc | Not in base R (packages exist); the formula is short, see example A2 | Not in SciPy; compute as in example A2, Holm via `statsmodels` `multipletests` | Name the adjustment method | ## Worked example A (fully solved): Mann-Whitney U and Kruskal-Wallis with Dunn ### A1. Mann-Whitney U: waiting times at two bakery branches **Hypotheses.** $H_0$: waiting times at the two branches have the same distribution. $H_1$: waiting times at one branch tend to be longer, $P(X_H > X_S) \neq 0.5$ (two-sided), α = 0.05. **Checks.** Independent customers, one visit each. Both groups are right-skewed with one long wait each, which is why a rank test is chosen for samples this small. There are no tied values, so an exact p-value is possible. The shapes look broadly similar, so a cautious "shift in location" reading is reasonable. **R code and output.** ```r harbour <- c(2.1, 3.4, 1.8, 4.0, 2.7, 3.1, 9.5, 2.4, 3.8) station <- c(4.2, 5.1, 3.6, 6.8, 4.9, 12.3, 5.6, 4.4, 7.2, 3.9) median(harbour); median(station) r <- rank(c(harbour, station)); R1 <- sum(r[1:9]); R1 U_h <- R1 - 9 * 10 / 2; U_s <- 9 * 10 - U_h; c(U_h, U_s) wilcox.test(harbour, station, conf.int = TRUE) # exact: n < 50 and no ties wilcox.test(harbour, station, exact = FALSE)$p.value # normal approximation, continuity correction 2 * U_h / (9 * 10) - 1 # rank-biserial correlation ``` ```text [1] 3.1 [1] 5 [1] 57 [1] 12 78 Wilcoxon rank sum exact test data: harbour and station W = 12, p-value = 0.005672 alternative hypothesis: true location shift is not equal to 0 95 percent confidence interval: -3.5 -0.8 sample estimates: difference in location -2 [1] 0.007963489 [1] -0.7333333 ``` **Python code and output.** With n = 9 and n = 10, SciPy's `method="auto"` chooses the normal approximation (neither sample has 8 or fewer values), while R chose the exact test. Same U, different p-value: .008 versus .006. Ask for `method="exact"` and the two agree. ```python import numpy as np from scipy import stats harbour = np.array([2.1, 3.4, 1.8, 4.0, 2.7, 3.1, 9.5, 2.4, 3.8]) station = np.array([4.2, 5.1, 3.6, 6.8, 4.9, 12.3, 5.6, 4.4, 7.2, 3.9]) print(stats.mannwhitneyu(harbour, station)) # auto -> asymptotic here print(stats.mannwhitneyu(harbour, station, method="exact")) print(stats.mannwhitneyu(station, harbour)) # U of the first sample res = stats.mannwhitneyu(harbour, station, method="exact") print("rank-biserial:", 2 * res.statistic / (9 * 10) - 1) ``` ```text MannwhitneyuResult(statistic=np.float64(12.0), pvalue=np.float64(0.007963489206549997)) MannwhitneyuResult(statistic=np.float64(12.0), pvalue=np.float64(0.005672346229621772)) MannwhitneyuResult(statistic=np.float64(78.0), pvalue=np.float64(0.007963489206549997)) rank-biserial: -0.7333333333333334 ``` **Calculation by hand.** Sort all 19 values: 1.8, 2.1, 2.4, 2.7, 3.1, 3.4, 3.6, 3.8, 3.9, 4.0, 4.2, 4.4, 4.9, 5.1, 5.6, 6.8, 7.2, 9.5, 12.3. Harbour values sit at positions 1, 2, 3, 4, 5, 6, 8 (3.8), 10 (4.0) and 18 (9.5); Station takes 7, 9, 11 to 17 and 19. So $R_H = 1 + 2 + 3 + 4 + 5 + 6 + 8 + 10 + 18 = 57$, matching the software. Ranking by hand is error-prone (it is easy to misplace 3.6 or 9.5), so always check your rank sum against the code. 1. $U_H = 57 - 9 \times 10/2 = 57 - 45 = 12$ and $U_S = 90 - 12 = 78$. 2. $\hat{P}(X_H > X_S) = 12/90 = 0.133$: in only 13% of all Harbour-Station pairs did the Harbour customer wait longer. 3. Normal approximation: mean of U under $H_0$ is $90/2 = 45$, SD is $\sqrt{90 \times 20/12} = \sqrt{150} = 12.25$; $z = (12 - 45 + 0.5)/12.25 = -2.65$, two-sided p = .008, the value SciPy and `exact = FALSE` report. 4. Rank-biserial $r_{rb} = 2 \times 0.133 - 1 = -0.73$: a large effect, Harbour waits tend to be shorter. **Effect size with CI.** R's `conf.int = TRUE` gives the Hodges-Lehmann estimate of the shift, the median of all 90 Harbour-minus-Station differences: $-2.0$ minutes, 95% CI [$-3.5$, $-0.8$]. Note that this is not the difference of the two medians ($3.1 - 5.0 = -1.9$): they are different quantities. For a CI on $r_{rb}$, use a bootstrap (`academic/stats-resampling-methods`). **Interpretation and report.** Customers at the Harbour branch tended to wait less than customers at the Station branch (median 3.1 versus 5.0 minutes), Mann-Whitney *U* = 12, exact *p* = .006, rank-biserial *r* = -.73, Hodges-Lehmann shift = -2.0 minutes, 95% CI [-3.5, -0.8]. ### A2. Kruskal-Wallis with Dunn post hoc: loan durations at three library branches **Hypotheses.** $H_0$: loan durations have the same distribution at all three branches. $H_1$: at least one branch tends to have longer or shorter loans. α = 0.05; post hoc p-values adjusted by Holm. **R code and output.** Base R has `kruskal.test` but no Dunn test, so the code computes Dunn's z from the joint ranks, with the tie correction, and adjusts with `p.adjust(..., "holm")`. ```r central <- c(7, 12, 9, 14, 10, 21, 8, 11) north <- c(14, 18, 16, 25, 13, 19, 15, 28) east <- c(10, 9, 12, 17, 8, 11, 13, 30) days <- c(central, north, east) branch <- factor(rep(c("Central", "North", "East"), each = 8), levels = c("Central", "North", "East")) kt <- kruskal.test(days ~ branch); kt N <- length(days); k <- 3 (unname(kt$statistic) - k + 1) / (N - k) # eta-squared H rk <- rank(days); mr <- tapply(rk, branch, mean); n <- tapply(rk, branch, length); mr tt <- table(days); s2 <- N * (N + 1) / 12 - sum(tt^3 - tt) / (12 * (N - 1)) pairs <- combn(levels(branch), 2) z <- apply(pairs, 2, function(p) (mr[p[1]] - mr[p[2]]) / sqrt(s2 * (1 / n[p[1]] + 1 / n[p[2]]))) p <- 2 * pnorm(-abs(z)) data.frame(pair = paste(pairs[1, ], pairs[2, ], sep = "-"), z = round(z, 3), p = round(p, 4), p_holm = round(p.adjust(p, "holm"), 4)) ``` ```text Kruskal-Wallis rank sum test data: days by branch Kruskal-Wallis chi-squared = 7.6884, df = 2, p-value = 0.0214 [1] 0.2708762 Central North East 8.625 18.000 10.875 pair z p p_holm 1 Central-North -2.656 0.0079 0.0237 2 Central-East -0.637 0.5239 0.5239 3 North-East 2.018 0.0436 0.0871 ``` **Python code and output.** ```python import numpy as np from itertools import combinations from scipy import stats from statsmodels.stats.multitest import multipletests groups = {"Central": [7, 12, 9, 14, 10, 21, 8, 11], "North": [14, 18, 16, 25, 13, 19, 15, 28], "East": [10, 9, 12, 17, 8, 11, 13, 30]} print(stats.kruskal(*groups.values())) allv = np.concatenate(list(groups.values())); N = len(allv) ranks = stats.rankdata(allv) idx = np.cumsum([0] + [len(v) for v in groups.values()]) mr = {g: ranks[idx[i]:idx[i + 1]].mean() for i, g in enumerate(groups)} _, t = np.unique(allv, return_counts=True) s2 = N * (N + 1) / 12 - (t**3 - t).sum() / (12 * (N - 1)) rows = [] for a, b in combinations(groups, 2): z = (mr[a] - mr[b]) / np.sqrt(s2 * (1 / len(groups[a]) + 1 / len(groups[b]))) rows.append((f"{a}-{b}", round(z, 3), 2 * stats.norm.sf(abs(z)))) p_holm = multipletests([r[2] for r in rows], method="holm")[1] for r, ph in zip(rows, p_holm): print(r[0], r[1], round(r[2], 4), round(ph, 4)) ``` ```text KruskalResult(statistic=np.float64(7.688399476668127), pvalue=np.float64(0.02140352312845205)) Central-North -2.656 0.0079 0.0237 Central-East -0.637 0.5239 0.5239 North-East 2.018 0.0436 0.0871 ``` **Calculation by hand.** Mean ranks 8.625, 18.000 and 10.875 give rank sums 69, 144 and 87 (each times n = 8). $H = \dfrac{12}{24 \times 25}\left(\dfrac{69^2 + 144^2 + 87^2}{8}\right) - 3 \times 25 = 0.02 \times 4133.25 - 75 = 7.665$. The tie correction (each of the values 8 to 14 appears twice) raises it slightly to 7.688, with df = 2 and p = .021. Effect size $\eta^2_H = (7.688 - 3 + 1)/(24 - 3) = 0.27$. **Interpretation and report.** Loan durations differed between branches, Kruskal-Wallis *H*(2) = 7.69, *p* = .021, $\eta^2_H$ = .27. Dunn post hoc tests with Holm adjustment showed longer loans at North (median 17 days) than at Central (median 10.5 days), *z* = 2.66, adjusted *p* = .024. North versus East (*z* = 2.02, unadjusted *p* = .044) was not significant after adjustment (adjusted *p* = .087), and Central versus East did not differ (adjusted *p* = .524). ## Worked example B (half solved): Wilcoxon signed-rank and sign test **Setting.** Twelve Lakemoor University students rated their exam anxiety (0 to 50 scale) before and after a 20-minute breathing exercise session. Same students twice, so the design is paired. The scale is ordinal in nature and there are ties and one zero difference. | Student | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | |---|---|---|---|---|---|---|---|---|---|---|---|---| | Before | 32 | 28 | 41 | 25 | 36 | 30 | 27 | 39 | 33 | 29 | 35 | 31 | | After | 27 | 26 | 35 | 25 | 30 | 31 | 22 | 33 | 30 | 24 | 29 | 28 | | Difference (after minus before) | -5 | -2 | -6 | 0 | -6 | 1 | -5 | -6 | -3 | -5 | -6 | -3 | ```r before <- c(32, 28, 41, 25, 36, 30, 27, 39, 33, 29, 35, 31) after <- c(27, 26, 35, 25, 30, 31, 22, 33, 30, 24, 29, 28) d <- after - before; dn <- d[d != 0]; rank(abs(dn)) wilcox.test(after, before, paired = TRUE) binom.test(sum(dn > 0), length(dn)) ``` ```python import numpy as np from scipy import stats before = np.array([32, 28, 41, 25, 36, 30, 27, 39, 33, 29, 35, 31]) after = np.array([27, 26, 35, 25, 30, 31, 22, 33, 30, 24, 29, 28]) print(stats.wilcoxon(after, before)) # default print(stats.wilcoxon(after, before, method="asymptotic", correction=True)) d = after - before print(stats.binomtest(int((d > 0).sum()), int((d != 0).sum()))) ``` ```text # Trimmed output, R then Python [1] 6.0 2.0 9.5 9.5 1.0 6.0 9.5 3.5 6.0 9.5 3.5 V = 1, p-value = 0.004777 (Wilcoxon signed rank test with continuity correction; warnings: cannot compute exact p-value with ties / with zeroes) number of successes = 1, number of trials = 11, p-value = 0.01172 WilcoxonResult(statistic=np.float64(1.0), pvalue=np.float64(0.001953125)) WilcoxonResult(statistic=np.float64(1.0), pvalue=np.float64(0.004777261821459183)) BinomTestResult(k=1, n=11, alternative='two-sided', statistic=0.09090909090909091, pvalue=0.01171875) ``` Complete the steps: 1. Number of non-zero differences: n = [ ]. The zero difference (student 4) is [ ] by both programs' defaults. 2. The single positive difference (+1) has rank [ ], so $T^+ =$ [ ] and $T^- = 66 - T^+ =$ [ ]. 3. Matched-pairs rank-biserial: $r = (T^+ - T^-)/(T^+ + T^-) =$ [ ]. 4. R reports p = [ ] (normal approximation with continuity correction). SciPy's default reports p = [ ] because, with ties or zeros and n of 13 or less, it uses [ ]. Adding `method="asymptotic", correction=True` reproduces [ ]. 5. Sign test: [ ] positive out of [ ] non-zero differences, p = [ ]. 6. Report: Anxiety ratings fell after the session (median difference [ ] points), Wilcoxon signed-rank $T^+$ = [ ], *p* = [ ], *r* = [ ]. ## Exercise C (unsolved): Friedman test Brightwell Bakery asked eight tasters to rate three sourdough recipes (Classic, Rye, Spelt) on a 1 to 10 scale. Each taster rated all three, so tasters are blocks. | Taster | T1 | T2 | T3 | T4 | T5 | T6 | T7 | T8 | |---|---|---|---|---|---|---|---|---| | Classic | 7 | 6 | 8 | 5 | 7 | 6 | 8 | 7 | | Rye | 8 | 8 | 9 | 7 | 7 | 9 | 8 | 9 | | Spelt | 5 | 6 | 6 | 5 | 4 | 7 | 6 | 5 | ```r taste <- matrix(c(7,8,5, 6,8,6, 8,9,6, 5,7,5, 7,7,4, 6,9,7, 8,8,6, 7,9,5), ncol = 3, byrow = TRUE, dimnames = list(paste0("T", 1:8), c("Classic", "Rye", "Spelt"))) friedman.test(taste) ``` ```python from scipy import stats classic = [7, 6, 8, 5, 7, 6, 8, 7] rye = [8, 8, 9, 7, 7, 9, 8, 9] spelt = [5, 6, 6, 5, 4, 7, 6, 5] print(stats.friedmanchisquare(classic, rye, spelt)) ``` Tasks: 1. State $H_0$ and $H_1$. 2. Rank the three recipes within each taster (1 = lowest) and compute the column rank sums. 3. Compute $\chi^2_F$ with the formula above (no tie correction), then run the code. Why is the software value larger? 4. Compute Kendall's W from the software statistic and interpret it. 5. Which follow-up comparisons would you run, and how would you control the error rate across them? ## Reading software output | Output line | Meaning | Report it? | |---|---|---| | `Wilcoxon rank sum exact test` (R) | Mann-Whitney with exact p-value; "with continuity correction" means normal approximation | Name the test and say exact or approximate | | `W = 12` (R) or `statistic=12.0` (SciPy `mannwhitneyu`) | U for the first group named in the call | Yes, as *U*, with the group it refers to | | `95 percent confidence interval: -3.5 -0.8` and `difference in location -2` | Hodges-Lehmann shift and its CI (needs `conf.int = TRUE`) | Yes, as the effect in original units | | `V = 1` (R signed-rank) | $T^+$, sum of ranks of positive differences | Yes, as $T^+$ or *V* | | `statistic=1.0` (SciPy `wilcoxon`, two-sided) | The smaller of $T^+$ and $T^-$ | Say which one you report | | `cannot compute exact p-value with ties` | R switched to the normal approximation | Mention the method | | `Kruskal-Wallis chi-squared = 7.6884, df = 2` | H with tie correction, compared with chi-square on k - 1 df | Yes: *H*(2) = 7.69 | | `Friedman chi-squared = [value], df = 2` | Friedman statistic with tie correction | Yes: $\chi^2_F$(2) = [value] | | Holm-adjusted p (`p.adjust`, `multipletests`) | Post hoc p-values controlling the familywise error rate | Yes, adjusted p and the method name | ## Reporting template **Mann-Whitney:** [Outcome] in [group 1] (*Mdn* = [ ], *IQR* = [ ], *n* = [ ]) tended to be [lower / higher] than in [group 2] (*Mdn* = [ ], *IQR* = [ ], *n* = [ ]), Mann-Whitney *U* = [ ], [exact / normal approximation] *p* = [ ], rank-biserial *r* = [ ], Hodges-Lehmann shift = [ ], 95% CI [[ ], [ ]]. **Wilcoxon signed-rank:** [Outcome] [decreased / increased] from [time 1] (*Mdn* = [ ]) to [time 2] (*Mdn* = [ ]), Wilcoxon signed-rank $T^+$ = [ ], *n* = [non-zero pairs], *p* = [ ], *r* = [ ]. **Kruskal-Wallis:** [Outcome] differed between [groups], *H*([k - 1]) = [ ], *p* = [ ], $\eta^2_H$ = [ ]. Dunn post hoc tests (Holm adjusted) showed [pair] differed (*z* = [ ], adjusted *p* = [ ]). **Friedman:** Ratings differed between [conditions], $\chi^2_F$([k - 1]) = [ ], *p* = [ ], Kendall's *W* = [ ]. | Comparison | n | Medians (IQR) | Test statistic | p (method) | Effect size | |---|---|---|---|---|---| | *Example:* Harbour vs Station waits | 9 / 10 | 3.1 / 5.0 min | U = 12 | .006 (exact) | r = -.73; HL = -2.0 [-3.5, -0.8] | | *Example:* Loan days, 3 branches | 8 / 8 / 8 | 10.5 / 17.0 / 11.5 | H(2) = 7.69 | .021 (chi-square) | η²H = .27 | | [Your comparison] | [ ] | [ ] | [ ] | [ ] | [ ] | **Results checklist:** - [ ] The reason for a rank test is stated (ordinal outcome, skew, outliers, small n), decided before seeing p-values. - [ ] The design (independent, paired, blocks) matches the test. - [ ] Medians and IQRs (or full distributions) are reported for each group. - [ ] The statistic is named and its reference group or direction is stated (U of which group, $T^+$ or the smaller T). - [ ] The p-value method (exact, normal approximation with or without continuity correction, permutation) is stated. - [ ] Ties and zero differences are reported with how they were handled. - [ ] An effect size is reported (rank-biserial, $\eta^2_H$, Kendall's W, Hodges-Lehmann shift with CI). - [ ] Post hoc comparisons name the procedure (Dunn) and the adjustment (Holm). - [ ] The conclusion is phrased as "tend to be larger" unless equal shapes justify "medians differ". ## Common mistakes and misconceptions - **"Mann-Whitney compares medians."** Only under the extra assumption that the two distributions have the same shape. In general it tests whether values in one group tend to be larger, $P(X > Y) \neq 0.5$. - **"Nonparametric tests have no assumptions."** They still need independence, and the signed-rank test assumes symmetric differences. The sign test is the one with the fewest assumptions. - **Comparing W across programs.** R's W is U for the first group; some textbooks use the rank sum; other packages (and SciPy before 1.7) report the smaller U. A "W = 57" and a "W = 12" can describe the same data. - **Not noticing exact versus approximate p-values.** In Worked example A1, R gives .006 (exact) and SciPy's default gives .008 (normal approximation). In Worked example B, R gives .005 and SciPy's default gives .002 (permutation). State the method. - **Running pairwise Mann-Whitney tests after Kruskal-Wallis without adjustment.** Use Dunn's test (which uses the same joint ranking) or adjust pairwise tests with Holm. - **Treating the Hodges-Lehmann estimate as the difference of medians.** It is the median of all pairwise differences, which is close to, but not the same as, the difference of the sample medians (-2.0 versus -1.9 in example A1). - **Using a rank test because a t-test was not significant.** Choosing the test after seeing the p-value inflates the Type I error rate. Decide in the analysis plan (`academic/stats-analysis-plan`). - **Switching to ranks for large, mildly skewed samples by reflex.** With 50 or more per group the t-test is usually fine for means, and the mean may be the quantity you need (for example, total cost). - **"p = 0.02 means the groups differ with 98% probability."** No: it is the probability of a statistic this extreme if $H_0$ were true. ## Self-test questions 1. What null hypothesis does the Mann-Whitney test assess, and what extra assumption lets you describe a significant result as a difference in medians? 2. With the same two samples of sizes 9 and 10 and no ties, R's `wilcox.test` and SciPy's `mannwhitneyu` report the same U but different p-values. Why? 3. A student reports "W = 57" from a textbook formula and you get "W = 12" in R for the same data with $n_1 = 9$. How are the two numbers related? 4. When would you prefer the sign test over the Wilcoxon signed-rank test? 5. Interleaving check: name the test. (a) Satisfaction ratings (1 to 5) of the same 20 students for four campus cafes. (b) Repair times (skewed) for three Brightwell Bakery ovens, different repairs. (c) Daily visitor counts at Harbour City Library for 60 weekdays, comparing two branches open on the same days. 6. After a significant Kruskal-Wallis test with four groups, how many pairwise comparisons are there, and how do you control the familywise error rate? ## My notes and next steps | Item | My entry | |---|---| | Confidence with rank tests (1 = lost, 5 = could teach it) | [ ] | | Why my own data need (or do not need) a rank test | [ ] | | Software defaults I must set explicitly in my analysis | [ ] | | Open questions for my tutor or supervisor | [ ] | | Mistakes logged in `academic/stats-error-log` | [Date and short description] | | Next review date (spacing: +2 days, +1 week, +1 month) | YYYY-MM-DD | ## Sources and further reading Structure aligned with the NIST/SEMATECH e-Handbook of Statistical Methods and the reporting elements named in APA JARS-Quant; all wording, data and examples here are original. - Wilcoxon, F. (1945). Individual comparisons by ranking methods. *Biometrics Bulletin*, 1(6), 80–83. - Mann, H. B., and Whitney, D. R. (1947). On a test of whether one of two random variables is stochastically larger than the other. *Annals of Mathematical Statistics*, 18(1), 50–60. - Kruskal, W. H., and Wallis, W. A. (1952). Use of ranks in one-criterion variance analysis. *Journal of the American Statistical Association*, 47(260), 583–621. - Friedman, M. (1937). The use of ranks to avoid the assumption of normality implicit in the analysis of variance. *Journal of the American Statistical Association*, 32(200), 675–701. - Dunn, O. J. (1964). Multiple comparisons using rank sums. *Technometrics*, 6(3), 241–252. - Holm, S. (1979). A simple sequentially rejective multiple test procedure. *Scandinavian Journal of Statistics*, 6(2), 65–70. - Hodges, J. L., and Lehmann, E. L. (1963). Estimates of location based on rank tests. *Annals of Mathematical Statistics*, 34(2), 598–611. - Kerby, D. S. (2014). The simple difference formula: an approach to teaching nonparametric correlation. *Comprehensive Psychology*, 3, 11.IT.3.1. - NIST/SEMATECH e-Handbook of Statistical Methods, sections 7.3.5 Do two arbitrary processes have the same central tendency? and 7.4.1 How can we compare several populations with unknown distributions (the Kruskal-Wallis test)?, https://www.itl.nist.gov/div898/handbook/ (accessed 2026-10-25). Public domain. - R Core Team. `wilcox.test`, `kruskal.test`, `friedman.test`, `binom.test` documentation, R 4.5.1. SciPy documentation for `mannwhitneyu`, `wilcoxon`, `kruskal`, `friedmanchisquare` and `binomtest`, SciPy 1.18. - Further reading (by name only): de Smith, M. J. (2024). *Statistical Analysis Handbook*, 2024 edition. StatsRef, https://www.statsref.com/; OpenIntro Statistics; American Psychological Association, *Publication Manual* (7th ed.) and JARS-Quant; Hollander, Wolfe and Chicken, *Nonparametric Statistical Methods*. ## Answer key **Worked example B.** (1) n = 11; the zero difference is dropped by both defaults (R, and SciPy's `zero_method="wilcox"`). (2) The +1 difference is the smallest absolute value, rank 1, so $T^+ = 1$ and $T^- = 66 - 1 = 65$ (the ranks 1 to 11 sum to 66). (3) $r = (1 - 65)/66 = -0.97$. (4) R: p = .005; SciPy default: p = .002, because with ties or zeros and n of 13 or less it computes an exhaustive permutation p-value; adding `method="asymptotic", correction=True` reproduces R's .005. (5) 1 positive out of 11, p = .012. (6) Anxiety ratings fell after the session (median difference -5 points), Wilcoxon signed-rank $T^+$ = 1, *n* = 11, *p* = .005 (normal approximation with continuity correction), *r* = -.97. The sign test agrees (1 of 11 increased, *p* = .012). **Exercise C.** (1) $H_0$: the three recipes have the same distribution of ratings within tasters. $H_1$: at least one recipe tends to be rated higher or lower. (2) Column rank sums: Classic 15, Rye 23, Spelt 10 (ties within a taster get average ranks, for example T5 gives Classic 2.5, Rye 2.5, Spelt 1). (3) Uncorrected $\chi^2_F = \dfrac{12}{8 \times 3 \times 4}(15^2 + 23^2 + 10^2) - 3 \times 8 \times 4 = 0.125 \times 854 - 96 = 10.75$. R and SciPy both report 12.286, df = 2, p = .002, because they correct for the ties within tasters (T2, T4, T5, T7), which reduce the variance of the ranks and so increase the statistic. (4) $W = 12.286/(8 \times 2) = 0.77$: strong agreement among tasters in how they order the recipes (Rye first, Spelt last). (5) Three pairwise Wilcoxon signed-rank tests (Classic-Rye, Classic-Spelt, Rye-Spelt) with Holm-adjusted p-values, or a Conover or Nemenyi procedure, named in the report. **Self-test.** 1. $H_0$: the two distributions are identical; the test detects $P(X > Y) \neq 0.5$. If both distributions have the same shape and differ only by a shift, a significant result is a difference in medians (and in means). 2. U is the same. R uses the exact distribution (both n < 50, no ties); SciPy's `method="auto"` uses exact only when one sample has 8 or fewer values, so here it uses the normal approximation with continuity correction. 3. The textbook W is the rank sum $R_1$; R's W is $U_1 = R_1 - n_1(n_1+1)/2 = 57 - 45 = 12$. 4. When the paired differences are clearly asymmetric (the signed-rank test assumes symmetry), or when only the direction of each change is trustworthy (for example, coarse ratings). 5. (a) Friedman test (same students rate all four, ratings ordinal). (b) Kruskal-Wallis (three independent groups, skewed). (c) Paired by day: Wilcoxon signed-rank on daily differences (or a paired t-test on differences if roughly symmetric; n = 60 is large); also check for autocorrelation over time. 6. $4 \times 3/2 = 6$ comparisons; use Dunn's test with Holm adjustment (or Benjamini-Hochberg if controlling the false discovery rate is acceptable and stated). ## Revision history | Version | Date | Author | Change summary | |---|---|---|---| | 0.1 | YYYY-MM-DD | [Your name] | First draft from the template | | 0.2 | YYYY-MM-DD | [Your name] | *Example:* Replaced bakery example with thesis data; exact p-values set explicitly |