# Repeated-Measures ANOVA: [Your study question or course] > A method card for within-subject designs: the same people (or batches, plots, machines) measured under every > condition or at every time point. It covers long versus wide data, one-way repeated-measures ANOVA in R and > Python, sphericity with Mauchly's test, the Greenhouse-Geisser and Huynh-Feldt corrections, the Friedman test as a > nonparametric link, Holm-adjusted paired comparisons, an overview of mixed (split-plot) designs, and the point at > which you should switch to a mixed model. It is for students preparing for a course or exam, thesis writers with > pre-post or multi-wave data, and analysts running within-subject tests. It speeds up learning by giving one fixed > order of decisions, a fully worked example, a half-solved example, an exercise and a self-test with a separate > answer key. The worked examples use fictional data; replace them with your own. > Source basis: standard textbook methods (Fisher 1925; Greenhouse and Geisser 1959; Huynh and Feldt 1976; Mauchly > 1940; Friedman 1937; Holm 1979) and the NIST/SEMATECH e-Handbook, written in original words. ## Document control | Field | Value | |---|---| | Course or project | [Course code and name, or thesis chapter] | | Learner or analyst | [Your name] | | Data set | [Name, source, number of subjects n, number of conditions k] | | Software and versions | R 4.5.1 (stats); Python 3.14, numpy 2.5.4, pandas 3.1.0, scipy 1.18.1, statsmodels 0.15.0 | | Date | YYYY-MM-DD | | Version | [0.1] | ## When to use it **One-sentence rule:** use a one-way repeated-measures ANOVA when one continuous outcome is measured on the same subjects under three or more conditions (or time points) and you want to know whether the condition means differ. With exactly two conditions, the repeated-measures ANOVA gives the same answer as a paired t-test (F equals t squared), so use the paired t-test. The repeated-measures design is powerful because each subject acts as their own control: stable differences between people (some students are always more anxious than others) are removed from the error term, so the comparison between conditions is made against a much smaller amount of noise. | Use when | Do not use when | |---|---| | Every subject is measured under every condition (complete, balanced data) | Some subjects are missing one or more time points (use a mixed model) | | The outcome is continuous (interval or ratio) | The outcome is binary or a count (use a generalised mixed model or Cochran's Q for binary) | | The differences between conditions are roughly normal, with no extreme outliers | Data are ordinal ratings with few levels or strongly skewed (use Friedman) | | Conditions are fixed and of interest in themselves (three teaching methods, weeks 1, 6 and 12) | Time is continuous and unevenly spaced across subjects (use a mixed model with time as a number) | | Order of conditions was counterbalanced or is the natural time order | Carry-over effects are likely and order was not counterbalanced (redesign or model order) | **Alternatives:** paired t-test (two conditions); Friedman test (ordinal or non-normal); multivariate approach (MANOVA on difference scores, no sphericity assumption, needs at least k subjects and in practice many more); linear mixed model (missing data, unequal time points, extra covariates, see stats-mixed-models); mixed or split-plot ANOVA (one within and one between factor, see the overview below). ## The question and the data **Research question (Example A):** does statistics anxiety among Lakemoor University students change across the first term of an introductory statistics course? Ten students in STAT 101 at Lakemoor University (fictional) completed a 0 to 40 statistics anxiety scale in week 1, week 6 and week 12. Higher scores mean more anxiety. | Variable | Role | Measurement level | Values | |---|---|---|---| | id | Subject identifier | Nominal | S1 to S10 | | week | Within-subject factor (k = 3 levels) | Ordinal, treated as categorical | w1, w6, w12 | | anx | Outcome | Interval (scale score) | 0 to 40 | **Wide layout** (one row per subject, one column per condition). This is how most people type data, and it is what Mauchly's test and the multivariate approach need. | id | w1 | w6 | w12 | |---|---|---|---| | *Example:* S1 | 28 | 26 | 21 | | S2 | 32 | 29 | 30 | | S3 | 25 | 25 | 24 | | S4 | 30 | 31 | 25 | | S5 | 22 | 21 | 19 | | S6 | 35 | 30 | 27 | | S7 | 27 | 27 | 20 | | S8 | 31 | 27 | 29 | | S9 | 24 | 25 | 23 | | S10 | 29 | 26 | 22 | | **Mean** | **28.3** | **26.7** | **24.0** | | SD | 3.95 | 2.87 | 3.74 | **Long layout** (one row per subject per condition, 30 rows here). This is what `aov()` with an `Error()` term, `AnovaRM` and every mixed-model function expect. | id | week | anx | |---|---|---| | *Example:* S1 | w1 | 28 | | S1 | w6 | 26 | | S1 | w12 | 21 | | S2 | w1 | 32 | | ... | ... | ... | A quick test of whether you have the right layout: in long format the subject identifier repeats k times, and there is exactly one outcome column. If your outcome is spread over several columns, you are in wide format. ```vega-lite { "$schema": "https://vega.github.io/schema/vega-lite/v5.json", "description": "Statistics anxiety of ten Lakemoor students at weeks 1, 6 and 12, one line per student", "width": 360, "height": 240, "data": {"values": [ {"id": "S1", "week": 1, "anx": 28}, {"id": "S1", "week": 6, "anx": 26}, {"id": "S1", "week": 12, "anx": 21}, {"id": "S2", "week": 1, "anx": 32}, {"id": "S2", "week": 6, "anx": 29}, {"id": "S2", "week": 12, "anx": 30}, {"id": "S3", "week": 1, "anx": 25}, {"id": "S3", "week": 6, "anx": 25}, {"id": "S3", "week": 12, "anx": 24}, {"id": "S4", "week": 1, "anx": 30}, {"id": "S4", "week": 6, "anx": 31}, {"id": "S4", "week": 12, "anx": 25}, {"id": "S5", "week": 1, "anx": 22}, {"id": "S5", "week": 6, "anx": 21}, {"id": "S5", "week": 12, "anx": 19}, {"id": "S6", "week": 1, "anx": 35}, {"id": "S6", "week": 6, "anx": 30}, {"id": "S6", "week": 12, "anx": 27}, {"id": "S7", "week": 1, "anx": 27}, {"id": "S7", "week": 6, "anx": 27}, {"id": "S7", "week": 12, "anx": 20}, {"id": "S8", "week": 1, "anx": 31}, {"id": "S8", "week": 6, "anx": 27}, {"id": "S8", "week": 12, "anx": 29}, {"id": "S9", "week": 1, "anx": 24}, {"id": "S9", "week": 6, "anx": 25}, {"id": "S9", "week": 12, "anx": 23}, {"id": "S10", "week": 1, "anx": 29}, {"id": "S10", "week": 6, "anx": 26}, {"id": "S10", "week": 12, "anx": 22} ]}, "mark": {"type": "line", "point": true, "opacity": 0.6}, "encoding": { "x": {"field": "week", "type": "quantitative", "title": "Week of term", "scale": {"domain": [0, 13]}}, "y": {"field": "anx", "type": "quantitative", "title": "Anxiety score (0 to 40)", "scale": {"domain": [15, 36]}}, "detail": {"field": "id", "type": "nominal"}, "color": {"value": "#4c78a8"} } } ``` Read the chart before running anything. Most lines slope down, but the lines are spread far apart vertically: the between-student differences are much larger than the change over time. That is exactly the variation the repeated-measures design removes from the error term. ## Assumptions and how to check each | Assumption | How to check | R | Python | If violated | |---|---|---|---|---| | Independence between subjects | Design: one row set per subject, subjects not clustered | Study design | Study design | Clustered subjects (classes, families): mixed model | | Complete data: every subject in every condition | Count rows per subject | `table(long$id)` | `long.groupby("id").size()` | Mixed model (listwise deletion wastes data and can bias) | | Approximate normality of the differences (residuals) | Q-Q plot of residuals or pairwise differences; outliers | `qqnorm(wide$w1 - wide$w12)` | `stats.probplot(d, plot=None)` | Friedman test; transform; check the outlier is real | | Sphericity: equal variances of all pairwise differences | Mauchly's test and, more usefully, the GG epsilon | `mauchly.test(mlm, X = ~1)` | By hand (see Example A) | Report GG (or HF) corrected df and p; or the multivariate approach | | No carry-over or order effects | Design: counterbalancing, washout | Study design | Study design | Model order as a factor; redesign | **Sphericity in plain words.** Take every pair of conditions and compute each subject's difference score (w1 minus w6, w1 minus w12, w6 minus w12). Sphericity says those three columns of differences have the same variance. When it fails, the ordinary F test is too liberal: it rejects H0 more often than 5% of the time when H0 is true. With only two conditions there is only one difference, so sphericity holds automatically. **Mauchly's test is a weak guide.** It has low power in small samples (it misses real violations) and too much power in large ones (it flags trivial ones). Many analysts therefore skip the gate and always report the Greenhouse-Geisser corrected result, or decide from the size of epsilon. This card follows that practice: look at epsilon, not only at Mauchly's p. ## How it works **Plain language.** One-way ANOVA splits the total variation in scores into "between groups" and "within groups". Repeated-measures ANOVA splits it three ways. First, the variation between subjects: some students are simply more anxious than others at every time point. That part is removed and plays no role in the test. Second, the variation between conditions: how far the week means are from the grand mean. Third, what is left over: the subject-by-condition interaction, meaning how inconsistently individual subjects change. The F ratio compares the condition variation with this leftover variation. Because the stable subject differences are taken out first, the error term is small and the test is powerful. $$ SS_{\text{total}} = SS_{\text{subjects}} + SS_{\text{conditions}} + SS_{\text{error}} $$ $$ SS_{\text{conditions}} = n \sum_{j=1}^{k} (\bar{y}_{\cdot j} - \bar{y}_{\cdot\cdot})^2, \qquad SS_{\text{subjects}} = k \sum_{i=1}^{n} (\bar{y}_{i \cdot} - \bar{y}_{\cdot\cdot})^2 $$ $$ F = \frac{SS_{\text{conditions}} / (k-1)}{SS_{\text{error}} / [(n-1)(k-1)]} \sim F_{k-1,\,(n-1)(k-1)} \text{ under } H_0 $$ **Sphericity corrections.** Let $S$ be the $(k-1) \times (k-1)$ covariance matrix of the subjects' scores on a set of $k-1$ orthonormal contrasts (for example linear and quadratic trend). The Greenhouse-Geisser estimate is $$ \hat{\varepsilon}_{GG} = \frac{\left(\operatorname{tr} S\right)^2}{(k-1)\,\operatorname{tr}\!\left(S^2\right)} = \frac{\left(\sum \lambda_m\right)^2}{(k-1) \sum \lambda_m^2}, \qquad \frac{1}{k-1} \le \hat{\varepsilon}_{GG} \le 1 $$ Under perfect sphericity all eigenvalues $\lambda_m$ are equal and epsilon is 1. The correction keeps F as it is and multiplies both degrees of freedom by epsilon, which makes the critical value larger and the p-value larger. The Huynh-Feldt estimate reduces the downward bias of GG when the true epsilon is near 1: $$ \tilde{\varepsilon}_{HF} = \frac{n(k-1)\hat{\varepsilon}_{GG} - 2}{(k-1)\left[(n-1) - (k-1)\hat{\varepsilon}_{GG}\right]} $$ HF can come out above 1; software then caps it at 1 for the p-value. A widely taught rule: use GG when its epsilon is below about 0.75, otherwise HF (Girden 1992). The conservative choice is to always report GG. | Symbol | Meaning | Example A value | |---|---|---| | $n$ | Number of subjects | 10 | | $k$ | Number of conditions (levels of the within factor) | 3 | | $\bar{y}_{\cdot j}$ | Mean of condition $j$ over subjects | 28.3, 26.7, 24.0 | | $\bar{y}_{i \cdot}$ | Mean of subject $i$ over conditions | S1: 25.0 | | $\bar{y}_{\cdot\cdot}$ | Grand mean | 26.33 | | $SS_{\text{error}}$ | Subject-by-condition sum of squares | 62.20 | | $\hat{\varepsilon}_{GG}$ | Greenhouse-Geisser epsilon | 0.888 | | $\eta^2_p$ | Partial eta squared, $SS_{\text{cond}} / (SS_{\text{cond}} + SS_{\text{error}})$ | 0.60 | ```mermaid flowchart TD A["Same subjects in 3 or more conditions?"] -->|No| B["Between-subjects ANOVA or t-test"] A -->|Yes| C["Any subject missing a condition?"] C -->|Yes| D["Linear mixed model"] C -->|No| E["Reshape to long format"] E --> F["Differences roughly normal, no extreme outliers?"] F -->|No| G["Friedman test, then Wilcoxon signed-rank with Holm"] F -->|Yes| H["Run RM ANOVA, get Mauchly and epsilon"] H --> I["GG epsilon below 0.75?"] I -->|Yes| J["Report GG corrected df and p"] I -->|No| K["Report HF or GG corrected df and p"] J --> L["Paired post hoc tests with Holm"] K --> L L --> M["Effect size and APA report"] ``` ## Worked example A (full) **Hypotheses.** $H_0: \mu_{w1} = \mu_{w6} = \mu_{w12}$ (mean anxiety is the same at all three time points). $H_1$: at least one mean differs. α = 0.05. **Checks.** All ten students have all three scores (complete data). The pairwise differences have no extreme values (the largest absolute difference is 8 points, S6 from week 1 to 12, against a spread of about 3). Mauchly's test is not significant, but with n = 10 it has little power, so we look at epsilon as well. **R code (base R only).** ```r # Lakemoor University: statistics anxiety (0-40) at weeks 1, 6 and 12 wide <- data.frame( id = factor(paste0("S", 1:10)), w1 = c(28, 32, 25, 30, 22, 35, 27, 31, 24, 29), w6 = c(26, 29, 25, 31, 21, 30, 27, 27, 25, 26), w12 = c(21, 30, 24, 25, 19, 27, 20, 29, 23, 22) ) # wide -> long: one row per student per week long <- reshape(wide, direction = "long", varying = c("w1", "w6", "w12"), v.names = "anx", timevar = "week", times = c("w1", "w6", "w12"), idvar = "id") long$week <- factor(long$week, levels = c("w1", "w6", "w12")) long <- long[order(long$id, long$week), ] # 1. Repeated-measures ANOVA with an Error() stratum for subjects fit <- aov(anx ~ week + Error(id / week), data = long) summary(fit) # 2. Sphericity: Mauchly's test and GG / HF corrections (base R, multivariate lm) mlm <- lm(cbind(w1, w6, w12) ~ 1, data = wide) mauchly.test(mlm, X = ~1) anova(mlm, X = ~1, test = "Spherical") # 3. Greenhouse-Geisser epsilon by hand Y <- as.matrix(wide[, c("w1", "w6", "w12")]) C <- contr.poly(3) # orthonormal contrasts (k - 1 = 2 columns) Sc <- cov(Y %*% C) # covariance of the contrast scores eps_gg <- sum(diag(Sc))^2 / (2 * sum(Sc^2)) eps_gg # 4. Effect size: partial eta squared ss_week <- 94.467; ss_err <- 62.2 ss_week / (ss_week + ss_err) # 5. Friedman test (nonparametric link) and Holm-adjusted paired t-tests friedman.test(Y) pairwise.t.test(long$anx, long$week, paired = TRUE, p.adjust.method = "holm") ``` ```text (output abridged) Error: id Df Sum Sq Mean Sq F value Pr(>F) Residuals 9 278 30.89 Error: id:week Df Sum Sq Mean Sq F value Pr(>F) week 2 94.47 47.23 13.67 0.000245 *** Residuals 18 62.20 3.46 Mauchly's test of sphericity W = 0.87406, p-value = 0.5837 Greenhouse-Geisser epsilon: 0.8881 Huynh-Feldt epsilon: 1.0911 Df F num Df den Df Pr(>F) G-G Pr H-F Pr (Intercept) 1 13.669 2 18 0.00024509 0.00047646 0.00024509 [1] 0.8881471 # GG epsilon by hand: matches [1] 0.6029796 # partial eta squared Friedman chi-squared = 12, df = 2, p-value = 0.002479 w1 w6 w6 0.0407 - w12 0.0026 0.0332 P value adjustment method: holm ``` **Python code (statsmodels and scipy).** `AnovaRM` gives the uncorrected F only, so Mauchly's W and the GG epsilon are computed by hand from the contrast covariance matrix. The by-hand values match R exactly, which is a good check that you understand the formula. ```python import numpy as np import pandas as pd from scipy import stats from statsmodels.stats.anova import AnovaRM from statsmodels.stats.multitest import multipletests wide = pd.DataFrame({ "id": [f"S{i}" for i in range(1, 11)], "w1": [28, 32, 25, 30, 22, 35, 27, 31, 24, 29], "w6": [26, 29, 25, 31, 21, 30, 27, 27, 25, 26], "w12": [21, 30, 24, 25, 19, 27, 20, 29, 23, 22], }) long = wide.melt(id_vars="id", var_name="week", value_name="anx") # 1. Repeated-measures ANOVA (no sphericity correction in AnovaRM) print(AnovaRM(long, depvar="anx", subject="id", within=["week"]).fit()) # 2. Mauchly's W and Greenhouse-Geisser epsilon by hand Y = wide[["w1", "w6", "w12"]].to_numpy(float) n, k = Y.shape C = np.array([[-1, 0, 1], [1, -2, 1]], float).T C = C / np.linalg.norm(C, axis=0) # orthonormal polynomial contrasts Sc = np.cov(Y @ C, rowvar=False) p = k - 1 W = np.linalg.det(Sc) / (np.trace(Sc) / p) ** p f = 1 - (2 * p**2 + p + 2) / (6 * p * (n - 1)) chi2 = -(n - 1) * f * np.log(W) dfm = p * (p + 1) / 2 - 1 print(f"Mauchly W = {W:.4f}, chi2({dfm:.0f}) = {chi2:.3f}, p = {stats.chi2.sf(chi2, dfm):.4f}") eps = np.trace(Sc) ** 2 / (p * np.sum(Sc**2)) F, df1, df2 = 13.669, 2, 18 print(f"GG epsilon = {eps:.4f}; corrected df = {eps*df1:.2f}, {eps*df2:.2f}; " f"p = {stats.f.sf(F, eps*df1, eps*df2):.5f}") # 3. Friedman and Holm-adjusted paired t-tests print(stats.friedmanchisquare(wide.w1, wide.w6, wide.w12)) pairs = [("w1", "w6"), ("w1", "w12"), ("w6", "w12")] raw = [stats.ttest_rel(wide[a], wide[b]).pvalue for a, b in pairs] adj = multipletests(raw, method="holm")[1] for (a, b), r, h in zip(pairs, raw, adj): print(f"{a} vs {b}: raw p = {r:.4f}, Holm p = {h:.4f}") ``` ```text Anova ================================== F Value Num DF Den DF Pr > F ---------------------------------- week 13.6688 2.0000 18.0000 0.0002 ================================== Mauchly W = 0.8741, chi2(2) = 1.077, p = 0.5837 GG epsilon = 0.8881; corrected df = 1.78, 15.99; p = 0.00048 FriedmanchisquareResult(statistic=np.float64(12.000000000000007), pvalue=np.float64(0.0024787521766663494)) w1 vs w6: raw p = 0.0407, Holm p = 0.0407 w1 vs w12: raw p = 0.0009, Holm p = 0.0026 w6 vs w12: raw p = 0.0166, Holm p = 0.0332 ``` **R and Python agree** on F = 13.67, Mauchly's W = 0.874, GG epsilon = 0.888, the Friedman statistic and the Holm p-values. The difference is in what each tool prints by default: R's multivariate `anova(..., test = "Spherical")` reports both corrections, while `AnovaRM` reports only the uncorrected test. **Calculation by hand for the key statistic.** The grand mean is (28.3 + 26.7 + 24.0) / 3 = 26.333. - $SS_{\text{week}} = 10 \times [(28.3 - 26.333)^2 + (26.7 - 26.333)^2 + (24.0 - 26.333)^2] = 10 \times 9.447 = 94.47$. - $SS_{\text{error}} = 62.20$ (from the output: total minus subjects minus weeks). - $MS_{\text{week}} = 94.47 / 2 = 47.23$; $MS_{\text{error}} = 62.20 / 18 = 3.456$. - $F = 47.23 / 3.456 = 13.67$ on 2 and 18 df. - GG correction: df become $2 \times 0.888 = 1.78$ and $18 \times 0.888 = 15.99$; p rises from .000245 to .00048. - HF epsilon is 1.09, above 1, so it is capped at 1 and the HF p-value equals the uncorrected one. **Effect size.** Partial eta squared is $94.47 / (94.47 + 62.20) = 0.60$: 60% of the within-subject variation that is not due to stable student differences is explained by week. Note that this is large partly because the stable between-student variation (SS = 278) has been removed. Generalised eta squared, which keeps the subject variation in the denominator, is $94.47 / (94.47 + 62.20 + 278) = 0.22$ and is easier to compare with between-subjects studies (Olejnik and Algina 2003; Bakeman 2005). Report which one you used. **Post hoc comparisons.** Paired t-tests with Holm adjustment: week 1 to week 6, mean difference 1.6 points, 95% CI [0.08, 3.12], *t*(9) = 2.39, Holm *p* = .041, *d*z = 0.75; week 1 to week 12, mean difference 4.3 points, 95% CI [2.30, 6.30], *t*(9) = 4.87, Holm *p* = .003, *d*z = 1.54; week 6 to week 12, mean difference 2.7 points, 95% CI [0.62, 4.78], *t*(9) = 2.94, Holm *p* = .033, *d*z = 0.93. The CIs shown are unadjusted; say so in your report. **Interpretation.** Anxiety fell steadily across the term, by about 4.3 points on a 40-point scale from week 1 to week 12. All three adjacent and non-adjacent comparisons were significant after Holm adjustment. The Friedman test, which uses only within-student ranks, agrees. This design cannot say why anxiety fell: there is no control group, so time, the course, exam timing and regression to the mean are all possible explanations. **APA-style report.** A one-way repeated-measures ANOVA showed that statistics anxiety changed across the term, *F*(1.78, 15.99) = 13.67, *p* < .001, partial η² = .60, Greenhouse-Geisser ε = .89. Mean anxiety fell from 28.3 (*SD* = 3.95) in week 1 to 26.7 (*SD* = 2.87) in week 6 and 24.0 (*SD* = 3.74) in week 12; all pairwise differences were significant after Holm adjustment (largest: week 1 to week 12, *M*diff = 4.3, 95% CI [2.30, 6.30], Holm *p* = .003). ## Worked example B (half solved) **Setting.** Eight tasters at Brightwell Bakery (fictional) each rated three sourdough recipes (A, B, C) for crust quality on a 1 to 10 scale, in a counterbalanced order. Question: do mean ratings differ between recipes? | Taster | A | B | C | |---|---|---|---| | 1 | 6 | 7 | 8 | | 2 | 7 | 8 | 8 | | 3 | 5 | 7 | 6 | | 4 | 8 | 9 | 9 | | 5 | 6 | 6 | 9 | | 6 | 7 | 8 | 8 | | 7 | 5 | 7 | 6 | | 8 | 6 | 8 | 9 | ```r b <- data.frame(id = factor(1:8), A = c(6,7,5,8,6,7,5,6), B = c(7,8,7,9,6,8,7,8), C = c(8,8,6,9,9,8,6,9)) long <- reshape(b, direction = "long", varying = c("A", "B", "C"), v.names = "score", timevar = "recipe", times = c("A", "B", "C"), idvar = "id") long$recipe <- factor(long$recipe) long <- long[order(long$id, long$recipe), ] summary(aov(score ~ recipe + Error(id / recipe), data = long)) m <- lm(as.matrix(b[, -1]) ~ 1) mauchly.test(m, X = ~1) anova(m, X = ~1, test = "Spherical") friedman.test(as.matrix(b[, -1])) pairwise.t.test(long$score, long$recipe, paired = TRUE, p.adjust.method = "holm") ``` ```text (output abridged) Error: id Df Sum Sq Mean Sq F value Pr(>F) Residuals 7 17.29 2.47 Error: id:recipe Df Sum Sq Mean Sq F value Pr(>F) recipe 2 11.583 5.792 11.45 0.00113 ** Residuals 14 7.083 0.506 W = 0.50491, p-value = 0.1287 Greenhouse-Geisser epsilon: 0.6689 Huynh-Feldt epsilon: 0.7684 Df F num Df den Df Pr(>F) G-G Pr H-F Pr (Intercept) 1 11.447 2 14 0.0011329 0.0052681 0.0033075 Friedman chi-squared = 12.214, df = 2, p-value = 0.002227 A B B 0.0046 - C 0.0046 0.4423 ``` In Python the same F comes from `AnovaRM(long, depvar="score", subject="id", within=["recipe"]).fit()` (F = 11.4471 on 2 and 14 df), and the by-hand GG epsilon code from Example A returns 0.6689. 1. Recipe means: A = 6.25, B = 7.50, C = [ ]. 2. Mauchly's test is not significant (*p* = .129). Yet GG epsilon is [ ], below 0.75. Which correction do you report, and why does Mauchly's p not settle it? [ ] 3. GG-corrected df: $2 \times 0.669 =$ [ ] and $14 \times 0.669 =$ [ ]. GG-corrected *p* = [ ]. 4. Partial η² = 11.583 / (11.583 + [ ]) = [ ]. 5. Post hoc: A differs from B (Holm *p* = .005) and from C (Holm *p* = .005); B versus C: [ ]. 6. Report sentence: A one-way repeated-measures ANOVA showed that crust ratings differed between recipes, *F*([ ], [ ]) = 11.45, *p* = [ ], partial η² = [ ], Greenhouse-Geisser ε = .67. [Finish with the post hoc result.] ## Exercise C (unsolved) Six Lakemoor University students took four weekly practice quizzes (scored 0 to 25) in the STAT 101 learning lab. | Student | T1 | T2 | T3 | T4 | |---|---|---|---|---| | 1 | 12 | 14 | 15 | 15 | | 2 | 15 | 16 | 18 | 19 | | 3 | 11 | 13 | 13 | 14 | | 4 | 14 | 15 | 17 | 18 | | 5 | 13 | 15 | 16 | 16 | | 6 | 16 | 18 | 19 | 21 | Tasks: (a) reshape to long format; (b) run the repeated-measures ANOVA; (c) obtain Mauchly's test and the GG and HF epsilons; (d) choose and justify a correction; (e) compute partial η²; (f) run the Friedman test as a check; (g) write an APA-style sentence. What is the smallest possible value of epsilon with k = 4? ## Reading software output | Output line | What it means | What to report | |---|---|---| | `Error: id` stratum, Residuals | Between-subject variation (stable individual differences); no test here in a one-way design | Usually nothing; use it for generalised η² | | `Error: id:week`, week row | The within-subject effect: SS, df, MS, F and uncorrected p | F and the effect size; df corrected below | | `Error: id:week`, Residuals | Subject-by-condition error term, df = (n-1)(k-1) | Its df (before correction) | | Mauchly's W and p | Test of sphericity; W = 1 means perfect sphericity | W and p, with the caveat about power | | Greenhouse-Geisser epsilon | Degree of sphericity violation, from 1/(k-1) to 1 | ε, and corrected df = ε times the original df | | Huynh-Feldt epsilon | Less conservative estimate; may exceed 1 (capped) | Only if you chose HF | | G-G Pr, H-F Pr | p-values with corrected df | The p-value matching the correction you chose | | AnovaRM table (Python) | Uncorrected F, df and p only | Combine with your own epsilon | | Friedman chi-squared | Rank-based test of the same question | As a robustness check or primary test for ordinal data | | Pairwise table with Holm | Adjusted p-values for each pair | Holm p, mean difference and CI per pair | ## Reporting template **Fill-in sentence.** A one-way repeated-measures ANOVA showed that [outcome] [did / did not] differ across [conditions or time points], *F*([df1 × ε], [df2 × ε]) = [F], *p* = [p], partial η² = [value], [Greenhouse-Geisser / Huynh-Feldt] ε = [value]. [Means and SDs per condition.] Holm-adjusted paired comparisons showed [which pairs differed, mean differences with 95% CIs]. | Condition | n | Mean | SD | 95% CI of mean | |---|---|---|---|---| | *Example:* Week 1 | 10 | 28.3 | 3.95 | [value] | | *Example:* Week 6 | 10 | 26.7 | 2.87 | [value] | | *Example:* Week 12 | 10 | 24.0 | 3.74 | [value] | | Comparison | Mean difference | 95% CI | t(df) | Holm p | dz | |---|---|---|---|---|---| | *Example:* w1 vs w12 | 4.3 | [2.30, 6.30] | 4.87 (9) | .003 | 1.54 | | [pair] | [value] | [value] | [value] | [value] | [value] | **Results checklist** - [ ] Number of subjects and confirmation that data were complete (or how missing data were handled). - [ ] Data layout checked: k rows per subject in long format. - [ ] Normality of differences and outliers checked and described. - [ ] Mauchly's test and epsilon reported; correction named. - [ ] F with corrected df, exact p, effect size with its type named. - [ ] Post hoc method and multiplicity adjustment named. - [ ] Means and SDs (or a plot) per condition. - [ ] Limitations: order effects, no control group, regression to the mean. ## Common mistakes and misconceptions 1. **Running a between-subjects one-way ANOVA on repeated data.** The rows are not independent. The test treats between-student variation as error and usually loses power; worse, the p-value has the wrong meaning. 2. **Reshaping wrongly.** Running `aov(anx ~ week)` on long data without `Error(id/week)` gives a between-subjects ANOVA. Always check that the output has two error strata. 3. **Treating a non-significant Mauchly test as proof of sphericity.** In Example B Mauchly's *p* = .129, but GG ε = .67 shows a substantial departure. "Not significant" means "not detected", not "absent". 4. **Reporting the uncorrected df with a corrected p (or the reverse).** The df and p must come from the same correction: *F*(1.78, 15.99), not *F*(2, 18), next to the GG p. 5. **Unadjusted post hoc tests.** Three pairwise tests at α = .05 inflate the familywise error rate. Use Holm (more powerful than Bonferroni, with the same guarantee). 6. **Using pooled-error post hoc tests (such as Tukey HSD from a between-subjects model) after a repeated-measures ANOVA.** When sphericity is doubtful, pairwise paired t-tests, each with its own error term, are safer. 7. **Comparing partial η² from a repeated-measures design with η² from a between-subjects study.** The repeated design removes subject variation, so its partial η² looks larger. Use generalised η² for comparisons. 8. **Dropping subjects with a missing time point without saying so.** Listwise deletion silently reduces n and can bias results if dropout is related to the outcome. Switch to a mixed model. 9. **Claiming a cause from a time trend.** A fall in anxiety over a term has many possible causes. Without a control group, describe change, not effect. ## Mixed (split-plot) designs: overview A mixed or split-plot ANOVA has at least one between-subjects factor (each subject is in one group) and one within-subjects factor (each subject is measured at every level). The name comes from agriculture: whole plots receive one treatment, and each plot is split into subplots that receive the other. The variance is split into two strata. The between-subjects stratum tests the group main effect against the variation between subjects within groups. The within-subjects stratum tests time and the group-by-time interaction against the subject-by-time residual. Sphericity applies to the within-subject effects only. As an illustration, suppose the odd-numbered students in Example A (S1, S3, S5, S7, S9) attended a study-skills workshop and the even-numbered students did not. ```r # Run after the Example A code: it reuses the data frame `wide` wide$group <- factor(rep(c("workshop", "control"), times = 5)) long <- reshape(wide, direction = "long", varying = c("w1", "w6", "w12"), v.names = "anx", timevar = "week", times = c("w1", "w6", "w12"), idvar = "id") long$week <- factor(long$week, levels = c("w1", "w6", "w12")) summary(aov(anx ~ group * week + Error(id / week), data = long)) ``` ```text (output abridged) Error: id Df Sum Sq Mean Sq F value Pr(>F) group 1 192.53 192.53 18.02 0.00282 ** Residuals 8 85.47 10.68 Error: id:week Df Sum Sq Mean Sq F value Pr(>F) week 2 94.47 47.23 13.757 0.000334 *** group:week 2 7.27 3.63 1.058 0.370139 Residuals 16 54.93 3.43 ``` The group effect is significant, but look at the data: the workshop students were less anxious already in week 1 (mean 25.2 against 31.4). The group main effect mostly reflects that baseline gap, not the workshop. The question "did the workshop change the trajectory?" is answered by the group-by-week interaction, which here is not significant (*F*(2, 16) = 1.06, *p* = .370). In Python, `AnovaRM` handles within-subject factors only; fit a split-plot design as a mixed model (`statsmodels` MixedLM) instead. For pre-post designs with a control group, also consider ANCOVA on the post-test with the pre-test as covariate (see stats-ancova). ## When to move to mixed models Repeated-measures ANOVA needs a complete, balanced grid of subjects by conditions. Real longitudinal data rarely are. Move to a linear mixed model (stats-mixed-models) when: - **Any subject has a missing measurement.** RM ANOVA drops the whole subject. A mixed model uses every observed value and gives valid results when data are missing at random (MAR), a weaker condition than the missing completely at random (MCAR) that listwise deletion needs. - **Time points differ between subjects** (one student tested in weeks 1, 5, 12, another in weeks 2, 7, 11), or you want time as a continuous predictor with a slope. - **You want individual trajectories**: random slopes let each subject have their own rate of change. - **There are extra levels of clustering**, such as students within classes or tasters within bakery shifts. - **There are time-varying covariates** (hours of study each week). - **The outcome is not normal** (binary, count): use a generalised linear mixed model. With complete, balanced data and compound symmetry, a random-intercept mixed model reproduces the uncorrected repeated-measures F test, so moving to a mixed model does not change the answer when the ANOVA was valid; it extends the answer to the cases where the ANOVA was not. ## Self-test questions 1. Why is a repeated-measures ANOVA usually more powerful than a between-subjects ANOVA on the same number of observations? 2. State sphericity in one sentence using the words "pairwise differences". 3. With k = 5 conditions, what is the lower bound of the Greenhouse-Geisser epsilon? What does it mean if epsilon equals that bound? 4. A colleague reports *F*(2, 38) = 4.1, *p* = .046, with GG ε = .62. What is wrong, and what should the df be? 5. Name two situations where you would use a mixed model instead of repeated-measures ANOVA. 6. When does the Friedman test give a different answer from the repeated-measures ANOVA, and which would you trust for 1 to 5 ratings with many ties? ## My notes and next steps | Item | My entry | |---|---| | Confidence (1 to 5) for reshaping data | [ ] | | Confidence (1 to 5) for choosing a sphericity correction | [ ] | | Confidence (1 to 5) for writing the APA sentence | [ ] | | Open questions | [What I still do not understand] | | Mistakes logged in stats-error-log | [Link or entry numbers] | | Next review date (spacing: 2 days, 1 week, 3 weeks) | YYYY-MM-DD | | Next card to study | [stats-mixed-models / stats-ancova] | ## Sources and further reading Based on standard methods described in the sources below; all wording, data and examples on this card are original. - Fisher, R. A. (1925). *Statistical Methods for Research Workers*. Oliver and Boyd. - Mauchly, J. W. (1940). Significance test for sphericity of a normal n-variate distribution. *Annals of Mathematical Statistics*, 11(2), 204–209. - Greenhouse, S. W., and Geisser, S. (1959). On methods in the analysis of profile data. *Psychometrika*, 24(2), 95–112. - Huynh, H., and Feldt, L. S. (1976). Estimation of the Box correction for degrees of freedom from sample data in randomized block and split-plot designs. *Journal of Educational Statistics*, 1(1), 69–82. - Friedman, M. (1937). The use of ranks to avoid the assumption of normality implicit in the analysis of variance. *Journal of the American Statistical Association*, 32(200), 675–701. - Holm, S. (1979). A simple sequentially rejective multiple test procedure. *Scandinavian Journal of Statistics*, 6(2), 65–70. - Girden, E. R. (1992). *ANOVA: Repeated Measures*. Sage. - Olejnik, S., and Algina, J. (2003). Generalized eta and omega squared statistics: measures of effect size for some common research designs. *Psychological Methods*, 8(4), 434–447. - Bakeman, R. (2005). Recommended effect size statistics for repeated measures designs. *Behavior Research Methods*, 37(3), 379–384. - NIST/SEMATECH e-Handbook of Statistical Methods, section 7.4.3 Are the means equal? (ANOVA) and chapter 5 Process Improvement (experimental design), https://www.itl.nist.gov/div898/handbook/ (accessed 2026-10-25). Public domain. - Further reading (by name only): de Smith, M. J., *Statistical Analysis Handbook* (2024 edition, statsref.com); OpenIntro Statistics; APA *Publication Manual* (7th ed.) and APA JARS-Quant for reporting standards; Wasserstein and Lazar (2016), ASA Statement on Statistical Significance and P-Values. ## Answer key **Example B.** 1. C = 7.875. 2. GG ε = .669. Report the GG correction: epsilon is below 0.75, and with only eight tasters Mauchly's test has little power, so a non-significant result does not show that sphericity holds. 3. df = 1.34 and 9.36; GG *p* = .005 (0.0052681). 4. Partial η² = 11.583 / (11.583 + 7.083) = 0.62. 5. B versus C: Holm *p* = .442, not significant. 6. ... *F*(1.34, 9.36) = 11.45, *p* = .005, partial η² = .62, Greenhouse-Geisser ε = .67. Recipe A was rated lower than both B and C (Holm *p* = .005 for both), while B and C did not differ (Holm *p* = .442). **Exercise C.** Means: T1 = 13.50, T2 = 15.17, T3 = 16.33, T4 = 17.17. RM ANOVA: SS(time) = 45.46, SS(error) = 4.29, *F*(3, 15) = 52.96, *p* < .001. Mauchly's W = 0.228, *p* = .372 (not significant, but n = 6). GG ε = .571, HF ε = .840. Because GG ε is below 0.75, report GG: *F*(1.71, 8.57) = 52.96, *p* < .001 (G-G p = 0.0000203). Partial η² = 45.46 / (45.46 + 4.29) = .91. Friedman χ²(3) = 17.21, *p* < .001, which agrees. Report: "Quiz scores rose across the four practice weeks, *F*(1.71, 8.57) = 52.96, *p* < .001, partial η² = .91, Greenhouse-Geisser ε = .57." The smallest possible epsilon with k = 4 is 1/(4 - 1) = 0.333. **Self-test.** 1. Stable differences between subjects are removed from the error term, so the effect is tested against only the subject-by-condition variation, which is usually much smaller. 2. The variances of the differences between every pair of conditions are equal. 3. 1/(5 - 1) = 0.25. Epsilon at the lower bound means maximal violation: the corrected test is equivalent to testing with 1 and n - 1 df. 4. The df are uncorrected. With ε = .62 they should be 2 × .62 = 1.24 and 38 × .62 = 23.56, and the p-value must be recomputed with those df; it will be larger than .046 and may no longer be below .05. 5. Any two of: missing measurements; unequal or subject-specific time points; interest in individual slopes; extra clustering levels; time-varying covariates; non-normal outcomes (generalised mixed model). 6. When the data are skewed, have outliers or are ordinal: Friedman uses only within-subject ranks, ignoring how large the differences are. For 1 to 5 ratings with many ties, Friedman (with ties correction, as R and scipy apply) is the safer primary test, with the ANOVA as a sensitivity check. ## Revision history | Version | Date | Author | Change | |---|---|---|---| | 0.1 | YYYY-MM-DD | [Name] | First version from template | | [0.2] | YYYY-MM-DD | [Name] | [Replaced example data with own data] |