Markdown Editor

ANCOVA Method Card Template (Statistics, R and Python)

Analysis of covariance: covariate adjustment, homogeneity of slopes, adjusted means by hand and with emmeans, ANCOVA vs change scores, Lord's paradox, reporting

学术完整Fisher (1932); Lord (1967); Searle, Speed and Milliken (1980); Senn (2006); NIST/SEMATECH e-Handbook; APA JARS-Quant (name)statsancovamath

使用方法: Use when comparing group means on a continuous outcome with a covariate measured before treatment, such as a pre-test. Test the covariate-by-group interaction first, fit the ANCOVA as a linear model with the covariate entered first or type II sums of squares, and report adjusted means with CIs. Main pitfall: adjusting for a covariate measured after treatment, or reading adjusted means from non-randomised groups as if adjustment made them comparable.

预览

ANCOVA: [Your study question or course]

A method card for analysis of covariance (ANCOVA): comparing group means on an outcome while adjusting for a continuous covariate measured before treatment, such as a pre-test score. It covers adjusted means (by hand and with emmeans), the assumptions including homogeneity of regression slopes, ANCOVA written as a linear model in R (lm) and Python (statsmodels ols), pre-post designs (ANCOVA versus change scores), Lord's paradox, and reporting. It is for students preparing for a course or exam, thesis writers with baseline and follow-up data, and analysts evaluating interventions. It speeds up learning with one fixed order of decisions, a full worked example, a half-solved example, an exercise where the key assumption fails, and a self-test with a separate answer key. The worked examples use fictional data; replace them with your own. Source basis: standard linear-model methods (Fisher 1932; Lord 1967; Searle, Speed and Milliken 1980; Senn 2006) and the NIST/SEMATECH e-Handbook, written in original words.

Document control

FieldValue
Course or project[Course code and name, or thesis chapter]
Learner or analyst[Your name]
Data set[Name, source, n per group, covariate and when it was measured]
Software and versionsR 4.5.1 (stats), car 3.1.5, emmeans 2.0.4; Python 3.14, pandas 3.1.0, statsmodels 0.15.0
DateYYYY-MM-DD
Version[0.1]

When to use it

One-sentence rule: use ANCOVA when you compare the means of two or more groups on a continuous outcome and have a continuous covariate, measured before treatment, that predicts the outcome; it removes covariate-related noise (more power) and corrects for chance baseline imbalance (fairer comparison).

ANCOVA is simply a linear regression with one categorical predictor (the group) and one continuous predictor (the covariate). Once you see it that way, everything else follows: the group coefficient is the adjusted difference, the covariate coefficient is the common slope, and the assumptions are regression assumptions plus one extra, that the slope is the same in every group.

Use whenDo not use when
Groups were randomised and you have a baseline measure of the outcome (pre-test)The covariate was measured after treatment and could be affected by it
The covariate is related to the outcome (correlation above about 0.3)The covariate is unrelated to the outcome (no gain, lost df)
The relationship between covariate and outcome is roughly linearThe relationship is curved (add a quadratic term or transform)
Regression slopes are similar across groups (interaction not significant and small)Slopes clearly differ (report the interaction and simple slopes instead)
You want to "equate" randomised groups for chance baseline differencesGroups are intact, non-randomised and differ systematically on the covariate (adjustment cannot make them comparable)

Alternatives: one-way ANOVA on the outcome (no covariate, less power); Welch t-test on change scores (simpler, less efficient unless the slope is near 1); regression with an interaction (when slopes differ, Johnson-Neyman intervals); linear mixed model (several follow-up time points, see stats-mixed-models); propensity-score methods (observational group comparisons).

The question and the data

Research question (Example A): after adjusting for the pre-test, do STAT 101 students who received peer tutoring score higher on the post-test than students with standard support?

Twenty students at Lakemoor University (fictional) were randomly assigned to standard support or weekly peer tutoring (10 each). All took a 0 to 100 pre-test in week 1, before the assignment, and a post-test in week 10.

VariableRoleMeasurement levelValues
groupFactor (treatment)Nominal, 2 levelsstandard, tutoring
preCovariate (measured before assignment)Interval0 to 100
postOutcomeInterval0 to 100
Studentgroupprepost
Example: 1standard5256
2standard6167
3standard4850
4standard7072
5standard5762
6standard6670
7standard4549
8standard5964
9standard6366
10standard5560
11tutoring5060
12tutoring5864
13tutoring4756
14tutoring6470
15tutoring5361
16tutoring6877
17tutoring4453
18tutoring5663
19tutoring6071
20tutoring4958
GroupMean preMean postSD post
standard57.661.67.89
tutoring54.963.37.42

Read the chart first. The two lines are nearly parallel (homogeneity of slopes looks plausible), and the tutoring line sits about 4 points higher at every pre-test value. Yet the raw post-test means differ by only 1.7 points, because the tutoring group happened to start 2.7 points lower. ANCOVA turns that picture into a number.

Assumptions and how to check each

AssumptionHow to checkRPythonIf violated
Covariate measured before treatment, not affected by itDesign and timelineStudy designStudy designDo not adjust for it; report ANOVA
Independence of observationsDesign: one row per unit, no clusteringStudy designStudy designMixed model with a cluster random effect
Linear covariate-outcome relationship in each groupScatter plot with group lines; residuals vs covariateplot(d$pre, resid(fit))plt.scatter(d.pre, fit.resid)Add I(pre^2) or transform
Homogeneity of regression slopesTest the covariate by group interactionanova(lm(post ~ pre * group))anova_lm(ols("post ~ pre * group"), typ=2)Keep the interaction; compare groups at chosen covariate values
Normal residualsQ-Q plot, Shapiro-Wilk on residualsshapiro.test(resid(fit))stats.shapiro(fit.resid)Robust SEs, transformation, or a rank-based approach
Equal residual variance across groupsResidual plot by group; Levenecar::leveneTest(post ~ group)stats.levene(...)HC3 robust standard errors
Covariate measured reliablyReliability of the pre-testPrior evidencePrior evidenceMeasurement error biases the adjustment; say so
Groups do not differ systematically on the covariate (non-randomised data)Compare covariate meanst.test(pre ~ group)stats.ttest_ind(..., equal_var=False)Interpret adjusted means cautiously (Lord's paradox)

A note on testing baseline balance. In a randomised study any baseline difference is due to chance by definition, so a significance test of the pre-test is not needed to justify ANCOVA; adjust for the pre-specified covariate regardless. In Example A the check is shown only so you can see that the 2.7-point gap is well within chance (Welch t(18.0) = 0.78, p = .448).

How it works

Plain language. Imagine sliding every student along their group's regression line until they all have the same pre-test score, the overall average. The group means after that slide are the adjusted means: the means you would expect if both groups had started at the same level. A group that started lower is moved up a little; a group that started higher is moved down a little. How far each group moves depends on the slope (how much the outcome changes per point of covariate) and on how far the group's covariate mean is from the grand mean. The test of the group effect then compares adjusted means against the residual variation around the lines, which is much smaller than the raw variation in the outcome when the covariate is a good predictor. That is where the gain in power comes from.

The ANCOVA model with groups j=1,…,gj = 1, \dots, g and units ii:

yij=μ+τj+β (xij−xˉ)+εij,εij∼N(0,σ2)y_{ij} = \mu + \tau_j + \beta\,(x_{ij} - \bar{x}) + \varepsilon_{ij}, \qquad \varepsilon_{ij} \sim N(0, \sigma^2)

The common slope is estimated from the pooled within-group sums of squares and cross-products:

bw=SPxy,withinSSx,within=∑j∑i(xij−xˉj)(yij−yˉj)∑j∑i(xij−xˉj)2b_w = \frac{SP_{xy,\text{within}}}{SS_{x,\text{within}}} = \frac{\sum_j \sum_i (x_{ij} - \bar{x}_j)(y_{ij} - \bar{y}_j)}{\sum_j \sum_i (x_{ij} - \bar{x}_j)^2}

and the adjusted mean of group jj is

yˉj,adj=yˉj−bw (xˉj−xˉ)\bar{y}_{j,\text{adj}} = \bar{y}_j - b_w\,(\bar{x}_j - \bar{x})

The F test for groups compares a model with group and covariate against a model with the covariate only; with two groups it equals the squared t statistic of the group coefficient. Partial eta squared is SSgroup/(SSgroup+SSresidual)SS_{\text{group}} / (SS_{\text{group}} + SS_{\text{residual}}).

ANCOVA versus change scores. Analysing the change y−xy - x is the special case of ANCOVA with the slope fixed at

  1. ANCOVA estimates the slope from the data instead, so it is at least as precise when groups are randomised, and more precise when the true slope is far from 1 (Senn 2006; Vickers and Altman 2001). Analysing only the post-test is the special case with the slope fixed at 0.
SymbolMeaningExample A value
yijy_{ij}Outcome for unit ii in group jjpost-test
xijx_{ij}Covariatepre-test
xˉ\bar{x}Grand mean of the covariate56.25
xˉj\bar{x}_jCovariate mean in group jj57.6, 54.9
bwb_wPooled within-group slope1053.7 / 1091.3 = 0.966
yˉj,adj\bar{y}_{j,\text{adj}}Adjusted mean60.30, 64.60
τj\tau_jGroup effect (adjusted difference)4.31
ηp2\eta^2_pPartial eta squared for group0.70

Worked example A (full)

Hypotheses. H0H_0: the adjusted post-test means are equal (τtutoring=τstandard\tau_{\text{tutoring}} = \tau_{\text{standard}}). H1H_1: they differ. α = 0.05, two-sided.

R code.

# Lakemoor University: STAT 101 pre-test and post-test (0-100), standard vs peer tutoring
d <- data.frame(
  group = factor(rep(c("standard", "tutoring"), each = 10)),
  pre  = c(52, 61, 48, 70, 57, 66, 45, 59, 63, 55,
           50, 58, 47, 64, 53, 68, 44, 56, 60, 49),
  post = c(56, 67, 50, 72, 62, 70, 49, 64, 66, 60,
           60, 64, 56, 70, 61, 77, 53, 63, 71, 58)
)
aggregate(cbind(pre, post) ~ group, data = d, FUN = mean)

# 1. Homogeneity of regression slopes: test the pre x group interaction
anova(lm(post ~ pre * group, data = d))

# 2. ANCOVA as a linear model (covariate first, then group)
fit <- lm(post ~ pre + group, data = d)
summary(fit)
confint(fit)
anova(fit)                                  # sequential (type I) SS

# 3. Adjusted means by hand: mean_g - b * (mean_pre_g - grand mean_pre)
b <- coef(fit)["pre"]
m <- aggregate(cbind(pre, post) ~ group, data = d, FUN = mean)
m$adj <- m$post - b * (m$pre - mean(d$pre))
m

# 4. Adjusted means with emmeans (install.packages("emmeans") once)
library(emmeans)
emm <- emmeans(fit, ~ group)
emm
pairs(emm)

# 5. Effect size: partial eta squared for group
a <- anova(fit)
a["group", "Sum Sq"] / (a["group", "Sum Sq"] + a["Residuals", "Sum Sq"])

# 6. Comparison: change scores and post-test only
t.test(I(post - pre) ~ group, data = d)     # Welch on change scores
t.test(post ~ group, data = d)              # Welch on post-test only
(output abridged)
     group  pre post
1 standard 57.6 61.6
2 tutoring 54.9 63.3

Response: post                       # slopes check
          Df Sum Sq Mean Sq  F value    Pr(>F)
pre        1 942.09  942.09 391.8400 1.122e-12 ***
group      1  89.75   89.75  37.3302 1.506e-05 ***
pre:group  1   0.64    0.64   0.2645     0.614
Residuals 16  38.47    2.40

Coefficients:
              Estimate Std. Error t value Pr(>|t|)
(Intercept)    5.98457    2.68761   2.227   0.0398 *
pre            0.96555    0.04591  21.031 1.31e-13 ***
grouptutoring  4.30697    0.68951   6.246 8.86e-06 ***
Residual standard error: 1.517 on 17 degrees of freedom
Multiple R-squared:  0.9635,	Adjusted R-squared:  0.9592

                  2.5 %    97.5 %
(Intercept)   0.3141975 11.654940
pre           0.8686819  1.062409
grouptutoring 2.8522419  5.761705

Response: post                       # ANCOVA, type I SS
          Df Sum Sq Mean Sq F value    Pr(>F)
pre        1 942.09  942.09 409.558 2.467e-13 ***
group      1  89.75   89.75  39.018 8.859e-06 ***
Residuals 17  39.10    2.30

     group  pre post      adj
1 standard 57.6 61.6 60.29651
2 tutoring 54.9 63.3 64.60349

 group    emmean    SE df lower.CL upper.CL
 standard   60.3 0.484 17     59.3     61.3
 tutoring   64.6 0.484 17     63.6     65.6

 contrast            estimate   SE df t.ratio p.value
 standard - tutoring    -4.31 0.69 17  -6.246 <0.0001

[1] 0.6965274                        # partial eta squared

	Welch Two Sample t-test
data:  I(post - pre) by group
t = -6.5672, df = 17.254, p-value = 4.45e-06
95 percent confidence interval:
 -5.811975 -2.988025
mean in group standard mean in group tutoring
                   4.0                    8.4

	Welch Two Sample t-test
data:  post by group
t = -0.49618, df = 17.934, p-value = 0.6258
95 percent confidence interval:
 -8.900104  5.500104
mean in group standard mean in group tutoring
                  61.6                   63.3

Python code.

import pandas as pd
import statsmodels.formula.api as smf
from statsmodels.stats.anova import anova_lm

d = pd.DataFrame({
    "group": ["standard"] * 10 + ["tutoring"] * 10,
    "pre":  [52, 61, 48, 70, 57, 66, 45, 59, 63, 55,
             50, 58, 47, 64, 53, 68, 44, 56, 60, 49],
    "post": [56, 67, 50, 72, 62, 70, 49, 64, 66, 60,
             60, 64, 56, 70, 61, 77, 53, 63, 71, 58],
})

# 1. Homogeneity of slopes
print(anova_lm(smf.ols("post ~ pre * group", data=d).fit(), typ=2).round(4))

# 2. ANCOVA (type II SS: each term adjusted for the other)
fit = smf.ols("post ~ pre + C(group)", data=d).fit()
print(fit.summary().tables[1])
aov = anova_lm(fit, typ=2)
print(aov.round(4))
ss = aov["sum_sq"]
print("partial eta^2 (group):", round(ss["C(group)"] / (ss["C(group)"] + ss["Residual"]), 4))

# 3. Adjusted means: predict each group at the grand mean of the covariate
grid = pd.DataFrame({"group": ["standard", "tutoring"], "pre": d["pre"].mean()})
pred = fit.get_prediction(grid).summary_frame(alpha=0.05)
print(pd.concat([grid, pred[["mean", "mean_se", "mean_ci_lower", "mean_ci_upper"]]], axis=1).round(3))
              sum_sq    df         F  PR(>F)
group        89.7523   1.0   37.3302   0.000
pre        1017.3955   1.0  423.1601   0.000
pre:group     0.6360   1.0    0.2645   0.614
Residual     38.4685  16.0       NaN     NaN
========================================================================================
                           coef    std err          t      P>|t|      [0.025      0.975]
----------------------------------------------------------------------------------------
Intercept                5.9846      2.688      2.227      0.040       0.314      11.655
C(group)[T.tutoring]     4.3070      0.690      6.246      0.000       2.852       5.762
pre                      0.9655      0.046     21.031      0.000       0.869       1.062
========================================================================================
             sum_sq    df         F  PR(>F)
C(group)    89.7523   1.0   39.0182     0.0
pre       1017.3955   1.0  442.2948     0.0
Residual    39.1045  17.0       NaN     NaN
partial eta^2 (group): 0.6965
      group    pre    mean  mean_se  mean_ci_lower  mean_ci_upper
0  standard  56.25  60.297    0.484         59.276         61.317
1  tutoring  56.25  64.603    0.484         63.583         65.624

Where R and Python differ, and why. The coefficients, the group test and the adjusted means are identical. The sum of squares for the covariate is not: R's anova() uses sequential (type I) sums of squares, so pre, entered first, gets 942.09 without adjusting for group. Python's anova_lm(typ=2) adjusts each term for the other, so pre gets 1017.40. The group row is the same in both (89.75) because group was entered last in R. If you put group first in the R formula, its type I SS changes and no longer tests the adjusted difference. Rule: in ANCOVA, enter the covariate first, or use type II (car::Anova(fit, type = 2) gives 1017.40 and 89.75, as Python does).

Calculation by hand for the key statistic.

  • Pooled within-group slope: bw=1053.7/1091.3=0.9655b_w = 1053.7 / 1091.3 = 0.9655 (the same as the pre coefficient).
  • Grand mean of the pre-test: xˉ=56.25\bar{x} = 56.25.
  • Standard: 61.6−0.9655×(57.6−56.25)=61.6−1.30=60.3061.6 - 0.9655 \times (57.6 - 56.25) = 61.6 - 1.30 = 60.30.
  • Tutoring: 63.3−0.9655×(54.9−56.25)=63.3+1.30=64.6063.3 - 0.9655 \times (54.9 - 56.25) = 63.3 + 1.30 = 64.60.
  • Adjusted difference: 64.60−60.30=4.3164.60 - 60.30 = 4.31, the grouptutoring coefficient.
  • Test: t=4.307/0.690=6.25t = 4.307 / 0.690 = 6.25 on 17 df, and t2=39.0=Ft^2 = 39.0 = F.

Effect size with CI. The adjusted difference is 4.31 points, 95% CI [2.85, 5.76]. Partial η² for group is 89.75 / (89.75 + 39.10) = 0.70. Beware: partial η² is inflated by the covariate, because the covariate removes most of the residual variation. A standardised difference that is comparable with studies without a covariate divides the adjusted difference by the pooled raw SD of the outcome: 4.31/7.66=0.564.31 / 7.66 = 0.56. Report both and say which denominator you used.

Three analyses, three answers. The post-test-only comparison finds nothing (difference 1.7, p = .626), because the raw post-test SD of about 7.7 swamps the effect and the tutoring group started lower. The change-score analysis gives a difference of 4.4 (95% CI [2.99, 5.81]). ANCOVA gives 4.31 (95% CI [2.85, 5.76]). Because the estimated slope (0.97) is close to 1, change scores and ANCOVA agree closely here; with a slope far from 1 they would not, and ANCOVA would be the more precise of the two.

Interpretation. Adjusting for the pre-test, students who received peer tutoring scored about 4.3 points higher on the post-test than students with standard support. The covariate explained most of the variation in post-test scores, which is why the adjusted comparison is so much more precise than the raw one. Because assignment was random, the adjusted difference estimates the causal effect of tutoring in this sample.

APA-style report. An ANCOVA with pre-test score as covariate showed that post-test scores differed between groups, F(1, 17) = 39.02, p < .001, partial η² = .70. The covariate-by-group interaction was not significant, F(1, 16) = 0.26, p = .614, supporting homogeneity of slopes. Adjusted for pre-test (M = 56.25), the tutoring group scored higher (adjusted M = 64.60, 95% CI [63.58, 65.62]) than the standard group (adjusted M = 60.30, 95% CI [59.28, 61.32]), adjusted difference = 4.31, 95% CI [2.85, 5.76].

Worked example B (half solved)

Setting. Brightwell Bakery (fictional) tested three oven settings (A, B, C) on 15 sourdough loaves, five per setting. Dough weight (g) was recorded before baking, because heavier dough gives larger loaves. Outcome: loaf volume (mL).

OvenDough weight (g)Volume (mL)
A980, 1010, 995, 1020, 9702420, 2465, 2450, 2510, 2380
B1000, 990, 1030, 985, 10152505, 2500, 2570, 2495, 2560
C1005, 975, 1025, 990, 10102490, 2390, 2515, 2460, 2480
library(car); library(emmeans)
b <- data.frame(
  oven  = factor(rep(c("A", "B", "C"), each = 5)),
  dough = c(980, 1010, 995, 1020, 970, 1000, 990, 1030, 985, 1015, 1005, 975, 1025, 990, 1010),
  vol   = c(2420, 2465, 2450, 2510, 2380, 2505, 2500, 2570, 2495, 2560, 2490, 2390, 2515, 2460, 2480)
)
aggregate(cbind(dough, vol) ~ oven, data = b, FUN = mean)
anova(lm(vol ~ dough * oven, data = b))        # slopes check
fb <- lm(vol ~ dough + oven, data = b)
Anova(fb, type = 2)
coef(fb)
emm <- emmeans(fb, ~ oven)
emm
pairs(emm, adjust = "holm")
(output abridged)
  oven dough  vol
1    A   995 2445
2    B  1004 2526
3    C  1001 2467

           Df  Sum Sq Mean Sq  F value    Pr(>F)
dough       1 28573.2 28573.2 145.0563 7.464e-07 ***
oven        2 10645.6  5322.8  27.0220 0.0001569 ***
dough:oven  2   201.8   100.9   0.5121 0.6156739
Residuals   9  1772.8   197.0

Anova Table (Type II tests)
           Sum Sq Df F value    Pr(>F)
dough     21675.4  1 120.749 2.860e-07 ***
oven      10645.6  2  29.652 3.709e-05 ***
Residuals  1974.6 11

(Intercept)       dough       ovenB       ovenC
 270.903084    2.185022   61.334802    8.889868

 oven emmean   SE df lower.CL upper.CL
 A      2456 6.07 11     2443     2469
 B      2517 6.04 11     2504     2531
 C      2465 6.00 11     2452     2478

 contrast estimate   SE df t.ratio p.value
 A - B      -61.33 8.66 11  -7.082 <0.0001
 A - C       -8.89 8.56 11  -1.039  0.3212
 B - C       52.44 8.49 11   6.174  0.0001
P value adjustment: holm method for 3 tests

The Python equivalent is anova_lm(smf.ols("vol ~ dough + C(oven)", data=b).fit(), typ=2), which reproduces the type II table above.

  1. Slopes check: the dough-by-oven interaction gives F(2, 9) = [ ], p = [ ]. Conclusion: [ ].
  2. The grand mean of dough weight is 1000 g, and the common slope is 2.185 mL per g. Adjusted mean for oven A by hand: 2445−2.185×(995−1000)=2445 - 2.185 \times (995 - 1000) = [ ]. For oven B: 2526−2.185×(2526 - 2.185 \times ( [ ] −1000)=- 1000) = [ ].
  3. Why does oven A's mean go up after adjustment while oven B's goes down? [ ]
  4. Partial η² for oven = 10645.6 / (10645.6 + [ ]) = [ ].
  5. Which pairs differ after Holm adjustment? [ ]
  6. Report: An ANCOVA with dough weight as covariate showed that loaf volume differed between oven settings, F([ ], [ ]) = [ ], p < .001, partial η² = [ ]. [Adjusted means and pairwise result.]

Exercise C (unsolved)

Sixteen Lakemoor University students took a statistics lab in one of two formats, guided or open-ended (eight each). A maths placement score (0 to 100) was recorded at enrolment; the outcome is the lab score (0 to 100).

FormatMaths placementLab score
guided40, 45, 50, 55, 60, 65, 70, 7561, 64, 65, 67, 66, 70, 69, 73
open42, 47, 52, 57, 62, 67, 72, 7752, 54, 61, 64, 71, 74, 80, 86

Tasks: (a) plot lab score against maths by format; (b) test homogeneity of slopes; (c) decide whether a standard ANCOVA is appropriate; (d) if not, estimate the slope in each format and compare the formats at maths = 45 and at maths = 70 (hint: emtrends() and emmeans(..., at = list(maths = c(45, 70)))); (e) find the maths score at which the two lines cross; (f) write two sentences for a results section.

Reading software output

Output lineWhat it meansWhat to report
pre:group row in the interaction modelTest of homogeneity of slopesF, df and p, as an assumption check
pre coefficientCommon within-group slope of outcome on covariateOptional; shows the covariate mattered
grouptutoring coefficientAdjusted difference (tutoring minus reference) at any covariate valueEstimate with 95% CI
(Intercept)Predicted outcome for the reference group at covariate = 0Rarely meaningful; centre the covariate if you want it to be
Type I anova() tableSequential SS; depends on term orderOnly if the covariate was entered first
Type II table (car::Anova, anova_lm(typ=2))Each term adjusted for the otherF for group, df, p, partial η²
emmeans tableAdjusted means at the covariate mean, with SE and CIAdjusted means and CIs
pairs() tablePairwise adjusted differences with the chosen adjustmentEstimates, adjusted p
Multiple R-squaredVariance explained by covariate and group togetherOptional

Reporting template

Fill-in sentence. An ANCOVA with [covariate, measured at baseline] as covariate showed that [outcome] [did / did not] differ between [groups], F([df1], [df2]) = [F], p = [p], partial η² = [value]. The covariate-by-group interaction was not significant, F([df], [df]) = [F], p = [p]. Adjusted for [covariate] (M = [value]), [group 1] (adjusted M = [value], 95% CI [[lower], [upper]]) [scored higher / lower than] [group 2] (adjusted M = [value], 95% CI [[lower], [upper]]), adjusted difference = [value], 95% CI [[lower], [upper]].

GroupnCovariate meanRaw outcome mean (SD)Adjusted mean95% CI
Example: standard1057.661.6 (7.89)60.30[59.28, 61.32]
Example: tutoring1054.963.3 (7.42)64.60[63.58, 65.62]
[group][n][value][value][value][value]

Results checklist

  • Covariate named, with when it was measured (before treatment) and why it was chosen (pre-specified).
  • Slopes check reported (interaction F and p, plus a plot).
  • Residual checks (normality, equal variance) summarised.
  • Type of sums of squares stated.
  • F, df, p and an effect size, with the denominator of any standardised effect named.
  • Adjusted means with CIs and the covariate value at which they were computed.
  • Pairwise comparisons with adjustment method (three or more groups).
  • Raw means shown alongside adjusted means.

Common mistakes and misconceptions

  1. Adjusting for a covariate measured after treatment. If tutoring changed study hours, and you adjust for study hours, you remove part of the tutoring effect. Covariates must be fixed before treatment.
  2. "ANCOVA makes non-randomised groups comparable." It does not. It adjusts only for the measured covariate, only linearly, and with the measurement error the covariate carries. Intact groups that differ on the covariate can give misleading adjusted means (Miller and Chapman 2001).
  3. Skipping the slopes check. If slopes differ, the "adjusted difference" depends on where you look along the covariate, and a single number is misleading (see Exercise C).
  4. Reporting type I SS with the group entered first. Then the group test ignores the covariate. Enter the covariate first or use type II.
  5. Reporting raw means as if they were adjusted. In Example A the raw difference is 1.7 points and the adjusted difference is 4.3. Label both.
  6. Comparing partial η² from ANCOVA with η² from an ANOVA. The covariate removes residual variance, so partial η² looks much larger. Standardise by the raw outcome SD for comparisons.
  7. Choosing covariates after looking at the results. Picking the covariate that gives the smallest p-value is a form of p-hacking. Pre-specify covariates in the analysis plan.
  8. Assuming change scores and ANCOVA always agree. They agree when the slope is close to 1. In randomised trials ANCOVA is generally more precise; in non-randomised comparisons they can disagree sharply (Lord's paradox).

Pre-post designs and Lord's paradox

With a pre-test and a post-test there are three common analyses: post-test only (ANOVA or t-test), change scores (t-test or ANOVA on post minus pre), and ANCOVA (post-test adjusted for pre-test). In a randomised study all three estimate the same treatment effect, because randomisation makes the expected baseline difference zero. They differ in precision: post-test only ignores useful information; change scores assume a slope of exactly 1; ANCOVA estimates the slope and is the usual recommendation for randomised trials with one baseline and one follow-up.

In a non-randomised study the analyses answer different questions and can disagree. Lord (1967) described two statisticians analysing the same data on weight gain in two dining halls with different starting weights. One compared mean change and found no difference between halls; the other ran an ANCOVA and found a difference. Both calculations were correct. The paradox is that the two methods make different assumptions about what would have happened without the "treatment": change scores assume each group would have stayed at its own baseline level apart from a common change, while ANCOVA assumes each group would have regressed toward the common line. No statistical test can tell you which assumption is true; it is a question about the causal process, to be argued from subject knowledge (and today usually expressed with a causal diagram).

Practical rule: in randomised trials, use ANCOVA with the baseline as covariate. In observational pre-post comparisons of intact groups, state the causal assumption explicitly, consider both analyses as sensitivity checks, and avoid causal language unless the design supports it. With more than two time points, use a mixed model (stats-mixed-models) or a repeated-measures design (stats-repeated-measures).

Self-test questions

  1. In one sentence, what is an adjusted mean?
  2. A group has outcome mean 70, covariate mean 48, grand covariate mean 50 and common slope 0.8. What is its adjusted mean?
  3. Why must the covariate be measured before treatment? Give a concrete example of what goes wrong otherwise.
  4. What does a significant covariate-by-group interaction mean, and what do you report instead of a single adjusted difference?
  5. In R, anova(lm(post ~ group + pre)) and anova(lm(post ~ pre + group)) give different F values for group. Which one is the ANCOVA test, and why?
  6. Change scores are the special case of ANCOVA with which slope? And a post-test-only analysis?

My notes and next steps

ItemMy entry
Confidence (1 to 5) for computing adjusted means by hand[ ]
Confidence (1 to 5) for checking homogeneity of slopes[ ]
Confidence (1 to 5) for choosing ANCOVA vs change scores[ ]
Open questions[What I still do not understand]
Mistakes logged in stats-error-log[Link or entry numbers]
Next review date (spacing: 2 days, 1 week, 3 weeks)YYYY-MM-DD
Next card to study[stats-mixed-models / stats-factorial-anova]

Sources and further reading

Based on standard linear-model methods described in the sources below; all wording, data and examples on this card are original.

  • Fisher, R. A. (1932). Statistical Methods for Research Workers (4th ed.). Oliver and Boyd.
  • Lord, F. M. (1967). A paradox in the interpretation of group comparisons. Psychological Bulletin, 68(5), 304–305.
  • Searle, S. R., Speed, F. M., and Milliken, G. A. (1980). Population marginal means in the linear model: an alternative to least squares means. The American Statistician, 34(4), 216–221.
  • Miller, G. A., and Chapman, J. P. (2001). Misunderstanding analysis of covariance. Journal of Abnormal Psychology, 110(1), 40–48.
  • Vickers, A. J., and Altman, D. G. (2001). Analysing controlled trials with baseline and follow up measurements. BMJ, 323(7321), 1123–1124.
  • Senn, S. (2006). Change from baseline and analysis of covariance revisited. Statistics in Medicine, 25(24), 4334–4344.
  • Lenth, R. V., and Piaskowski, J. emmeans: Estimated Marginal Means, aka Least-Squares Means (R package, version 2.0.4).
  • Fox, J., Weisberg, S., and Price, B. car: Companion to Applied Regression (R package, version 3.1-5).
  • NIST/SEMATECH e-Handbook of Statistical Methods, section 3.2.3 Analysis of Variance Models (ANOVA), and chapter 4 Process Modeling, https://www.itl.nist.gov/div898/handbook/ (accessed 2026-10-25). Public domain.
  • Further reading (by name only): de Smith, M. J. (2024). Statistical Analysis Handbook (2024 edition, StatsRef); OpenIntro Statistics; APA Publication Manual (7th ed.) and APA JARS-Quant for reporting standards.

Answer key

Example B.

  1. F(2, 9) = 0.51, p = .616. The slopes are similar across ovens, so a common-slope ANCOVA is reasonable.
  2. A: 2445+10.93=2455.92445 + 10.93 = 2455.9. B: 2526−2.185×(1004−1000)=2526−8.74=2517.32526 - 2.185 \times (1004 - 1000) = 2526 - 8.74 = 2517.3. (C: 2467−2.19=2464.82467 - 2.19 = 2464.8.) These match the emmeans values 2456, 2517 and 2465.
  3. Oven A's loaves had lighter dough than average (995 g), so part of their low volume is due to the dough, not the oven; adjustment moves A up. Oven B had heavier dough (1004 g), so adjustment moves it down.
  4. 10645.6 / (10645.6 + 1974.6) = 0.84.
  5. B differs from A (adjusted difference 61.3 mL, Holm p < .001) and from C (52.4 mL, Holm p < .001); A and C do not differ (8.9 mL, Holm p = .321).
  6. ... F(2, 11) = 29.65, p < .001, partial η² = .84. Adjusted for dough weight (M = 1000 g), oven B produced larger loaves (adjusted M = 2517 mL) than ovens A (2456 mL) and C (2465 mL); A and C did not differ.

Exercise C. Run anova(lm(score ~ maths * lab, data = x)): the interaction is large and significant, F(1, 12) = 167.02, p < .001, so homogeneity of slopes fails and a single adjusted difference is misleading. Slopes (emtrends): guided 0.29 lab points per maths point (95% CI [0.21, 0.38]); open 0.99 ([0.90, 1.07]). At maths = 45 the guided format scores higher by 9.76 points (t(12) = 10.26, p < .001); at maths = 70 the open format scores higher by 7.56 points (t(12) = -8.68, p < .001). The lines cross where −40.94+0.693×maths=0-40.94 + 0.693 \times \text{maths} = 0, at maths ≈ 59.1 (from the coefficients: open intercept shift -40.936, slope difference 0.6929). Report: "The effect of lab format depended on maths placement, F(1, 12) = 167.02, p < .001. Students with low placement scores did better in the guided format (difference at 45: 9.8 points), while students with high scores did better in the open format (difference at 70: 7.6 points)."

Self-test.

  1. The group's mean outcome predicted at a common covariate value (usually the grand mean), using the common slope.
  2. 70−0.8×(48−50)=70+1.6=71.670 - 0.8 \times (48 - 50) = 70 + 1.6 = 71.6.
  3. Because a covariate affected by treatment carries part of the treatment effect, and adjusting for it removes that part. Example: adjusting a tutoring effect for hours studied during the term, when tutoring itself increased study hours.
  4. The covariate-outcome slope differs between groups, so the group difference depends on the covariate value. Report the slopes per group and compare groups at several meaningful covariate values (or give the Johnson-Neyman region).
  5. post ~ pre + group. With type I SS each term is adjusted only for the terms before it; group must come after the covariate to be tested as an adjusted effect. Type II SS avoids the ordering issue.
  6. Change scores: slope fixed at 1. Post-test only: slope fixed at 0.

Revision history

VersionDateAuthorChange
0.1YYYY-MM-DD[Name]First version from template
[0.2]YYYY-MM-DD[Name][Replaced example data with own data]

相关模板