t-Tests Method Card Template (Statistics, R and Python)

t-Tests Method Card: [Your study question or course]

A method card for the three t-tests you will meet in every statistics course and most thesis projects: the one-sample t-test, the independent two-sample t-test (Welch by default, pooled only when justified) and the paired t-test. It is for students preparing for an exam, master's and PhD students analysing thesis data, and analysts who need a quick, correct comparison of means. It speeds up learning by putting the decisions in order (which test, which checks, which effect size, which sentence to report), by showing one example fully solved, one half solved and one left for you, and by ending with self-test questions whose answers sit in a separate key. The worked examples use fictional data; replace them with your own. Source basis: Welch (1947), Student (1908), Cohen (1988), the NIST/SEMATECH e-Handbook of Statistical Methods, and the R and SciPy documentation, all written up in original words and examples.

Document control

When to use it

One-sentence rule: use a t-test when you want to compare one mean, or the difference between two means, for a continuous outcome, and the sample means can be treated as approximately normal (because the data are roughly normal or the samples are large enough).

There are three versions, and the design decides which one you use:

• One-sample t-test: one group, compared with a fixed reference value (a target, a published norm, a specification).
• Independent two-sample t-test: two separate groups of different people or units. Use Welch's t-test by default; it does not assume equal variances and loses almost nothing when the variances happen to be equal (Delacre, Lakens and Leys 2017). Use the pooled (Student) version only when equal variances are justified by design or prior knowledge, or when a course or protocol requires it.
• Paired t-test: two measurements on the same units (before and after, left and right, matched pairs). It is a one-sample t-test on the differences.

Alternatives: Mann-Whitney U or Wilcoxon signed-rank (academic/stats-nonparametric-tests); permutation test or bootstrap CI for the mean difference (academic/stats-resampling-methods); one-way ANOVA or Welch ANOVA for three or more groups (academic/stats-one-way-anova); linear regression when you need to adjust for covariates (academic/stats-linear-regression); mixed models for clustered or repeated data (academic/stats-mixed-models).

The question and the data

Research question (example): At Lakemoor University (fictional), STAT 101 ran two sections in one term. One section was taught by traditional lecture, the other with a flipped classroom (videos at home, problem solving in class). Do the final exam scores differ between the two sections, and by how much?

The worked example data set (fictional):

Look at the data before you test anything. The chart shows each student as a dot and each section mean as a thick tick. The flipped section sits higher, and it is also more spread out: that unequal spread is exactly why Welch's version is the safer default.

Assumptions and how to check each

The t-test is robust to mild non-normality, especially with balanced groups of moderate size. It is not robust to dependence between observations, and it is sensitive to extreme outliers. Formal normality tests have little power in small samples and flag trivial departures in large ones, so treat them as a supplement to a plot, never as the decision rule.

In Worked example A, Shapiro-Wilk gives p = .965 (lecture) and p = .829 (flipped), the dot plot shows no extreme values, and the variance ratio is 62.79 / 28.23 = 2.22 (F-test p = .240). An F-test that is "not significant" does not prove equal variances, which is one more reason not to let a pre-test switch you between the pooled and Welch versions. Use Welch and move on.

How it works

Every t-statistic is a signal-to-noise ratio. The signal is how far the observed mean (or difference in means) sits from the value claimed by the null hypothesis. The noise is the standard error SE: how much that mean would wobble from sample to sample by chance alone. If t is large in absolute value, the observed difference is large compared with the chance wobble, and the p-value is small. The t distribution (not the normal) is used because the SE itself is estimated from the data; with few degrees of freedom df, the t distribution has heavier tails to account for that extra uncertainty.

One-sample t-test (hypothesised mean $\mu_0$):

Paired t-test (differences $d_i = x_{i,\text{after}} - x_{i,\text{before}}$): a one-sample test on the $d_i$ with $\mu_0 = 0$:

Welch's independent t-test (default):

The df formula is the Welch-Satterthwaite approximation. It usually gives a non-integer df that lies between the smaller of $n_1 - 1$ and $n_2 - 1$ and the pooled value $n_1 + n_2 - 2$.

Pooled (Student) independent t-test (only when equal variances are justified):

Confidence interval for any of the three: estimate $\pm\ t_{0.975,\,df} \times SE$.

Effect sizes: Cohen's d for two independent groups uses the pooled SD; Hedges' g applies a small-sample correction; for paired data, $d_z$ divides the mean difference by the SD of the differences.

An approximate standard error for d (used for a normal-approximation 95% CI) is $SE_d = \sqrt{\dfrac{n_1 + n_2}{n_1 n_2} + \dfrac{d^2}{2(n_1 + n_2)}}$; multiply the CI limits by $J$ for g.

Worked example A (fully solved): Welch and pooled independent t-tests

Hypotheses. Let $\mu_F$ and $\mu_L$ be the population mean exam scores under the flipped and lecture formats. $H_0: \mu_F - \mu_L = 0$ versus $H_1: \mu_F - \mu_L \neq 0$ (two-sided), α = 0.05.

Checks. Independent groups by design (different students, one score each). No extreme outliers in the dot plot. Shapiro-Wilk p = .965 and p = .829 (no evidence against normality, but with n = 10 and 12 this test is weak). SDs 7.92 versus 5.31: a variance ratio of 2.22, so Welch is clearly the right default.

R code and output. R's t.test() uses Welch unless you ask for var.equal = TRUE.

# Lakemoor University (fictional): STAT 101 final exam scores
lecture <- c(62, 70, 58, 75, 66, 71, 64, 69, 60, 68)
flipped <- c(74, 81, 66, 90, 72, 85, 79, 68, 88, 76, 83, 70)
c(mean(lecture), sd(lecture), mean(flipped), sd(flipped))
shapiro.test(lecture)$p.value; shapiro.test(flipped)$p.value
var.test(flipped, lecture)$p.value
t.test(flipped, lecture)                    # Welch (R default, var.equal = FALSE)
t.test(flipped, lecture, var.equal = TRUE)  # pooled Student, for comparison
# Cohen's d (pooled SD), Hedges' g and an approximate 95% CI for g
n1 <- length(flipped); n2 <- length(lecture)
sp <- sqrt(((n1 - 1) * var(flipped) + (n2 - 1) * var(lecture)) / (n1 + n2 - 2))
d  <- (mean(flipped) - mean(lecture)) / sp
J  <- 1 - 3 / (4 * (n1 + n2) - 9)
g  <- J * d
se_d <- sqrt((n1 + n2) / (n1 * n2) + d^2 / (2 * (n1 + n2)))
round(c(sp = sp, d = d, g = g, lo = g - 1.96 * J * se_d, hi = g + 1.96 * J * se_d), 3)

[1] 66.300000  5.313505 77.666667  7.923880
[1] 0.9654593
[1] 0.8294417
[1] 0.2402227

	Welch Two Sample t-test

data:  flipped and lecture
t = 4.0048, df = 19.23, p-value = 0.0007426
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
  5.430958 17.302376
sample estimates:
mean of x mean of y
 77.66667  66.30000

	Two Sample t-test

data:  flipped and lecture
t = 3.8625, df = 20, p-value = 0.0009702
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
  5.227991 17.505342

   sp     d     g    lo    hi
6.873 1.654 1.591 0.657 2.525

Python code and output. Here is the key teaching point of this card: SciPy's ttest_ind() defaults to equal_var=True, the pooled test, while R's t.test() defaults to Welch. Run the same call in both languages with default arguments and you get two different answers (t = 3.86 with df = 20 in SciPy, t = 4.00 with df = 19.23 in R). Always set equal_var=False in Python unless you have a reason not to.

import numpy as np
from scipy import stats

lecture = np.array([62, 70, 58, 75, 66, 71, 64, 69, 60, 68])
flipped = np.array([74, 81, 66, 90, 72, 85, 79, 68, 88, 76, 83, 70])
print(stats.ttest_ind(flipped, lecture))                  # SciPy default: pooled
res = stats.ttest_ind(flipped, lecture, equal_var=False)  # Welch
print(res)
print(res.confidence_interval(0.95))

TtestResult(statistic=np.float64(3.8624700393746134), pvalue=np.float64(0.0009702033494507789), df=np.float64(20.0))
TtestResult(statistic=np.float64(4.004816788599783), pvalue=np.float64(0.0007426031571542732), df=np.float64(19.23042884691189))
ConfidenceInterval(low=np.float64(5.430957692222952), high=np.float64(17.302375641110395))

Calculation by hand (Welch). Group 1 is flipped ($n_1 = 12$, $\bar{x}_1 = 77.667$, $s_1^2 = 62.788$), group 2 is lecture ($n_2 = 10$, $\bar{x}_2 = 66.300$, $s_2^2 = 28.233$).

1. Variance of each mean: $s_1^2/n_1 = 62.788/12 = 5.2323$ and $s_2^2/n_2 = 28.233/10 = 2.8233$.
2. Standard error: $SE = \sqrt{5.2323 + 2.8233} = \sqrt{8.0556} = 2.8382$.
3. Statistic: $t = (77.667 - 66.300)/2.8382 = 11.367/2.8382 = 4.005$.
4. Welch-Satterthwaite df: numerator $8.0556^2 = 64.893$; denominator $5.2323^2/11 + 2.8233^2/9 = 2.4888 + 0.8857 = 3.3745$; so $df_W = 64.893/3.3745 = 19.23$.
5. Critical value $t_{0.975,\,19.23} = 2.091$, so the 95% CI is $11.367 \pm 2.091 \times 2.8382 = [5.43,\ 17.30]$.

The hand result matches both programs. Notice that the pooled test, with $df = 20$, gives a slightly smaller t here: the pooled variance (47.24) replaces both group variances, so the smaller lecture group (own variance 28.23, n = 10) is credited with a much larger variance, and the SE rises from 2.84 to 2.94. When the smaller group has the larger variance, the effect reverses and the pooled test becomes too liberal.

Effect size. Cohen's d = 11.367 / 6.873 = 1.65; Hedges' g = 1.59, approximate 95% CI [0.66, 2.53] (normal approximation with the SE formula above; exact intervals based on the noncentral t distribution differ slightly). That is a large difference by Cohen's rough benchmarks, but the CI is wide because the samples are small: the true effect could be moderate or very large.

Interpretation. The flipped section scored on average 11.4 points higher than the lecture section. A difference this large would be rare if the two formats had the same population mean (p < .001). Because students were not randomly assigned to sections, this is evidence of a difference between the two sections, not proof that the flipped format caused it.

APA-style report. Students in the flipped section (M = 77.67, SD = 7.92, n = 12) scored higher than students in the lecture section (M = 66.30, SD = 5.31, n = 10), Welch's t(19.23) = 4.00, p < .001, mean difference = 11.37 points, 95% CI [5.43, 17.30], Hedges' g = 1.59, 95% CI [0.66, 2.53].

Worked example B (half solved): paired t-test

Setting. Ten Lakemoor University students took a 20-point statistics quiz before and after a two-hour revision workshop run by the campus learning centre. The same students were measured twice, so the design is paired.

before <- c(12, 15, 9, 14, 11, 13, 10, 16, 12, 8)
after  <- c(14, 17, 10, 15, 14, 13, 13, 18, 13, 11)
d <- after - before
mean(d); sd(d); shapiro.test(d)$p.value
t.test(after, before, paired = TRUE)

import numpy as np
from scipy import stats
before = np.array([12, 15, 9, 14, 11, 13, 10, 16, 12, 8])
after  = np.array([14, 17, 10, 15, 14, 13, 13, 18, 13, 11])
r = stats.ttest_rel(after, before)
print(r, r.confidence_interval())

# Trimmed and combined output from the two blocks above
R:      mean(d) = 1.8   sd(d) = 1.032796   Shapiro-Wilk p = 0.191
        t = 5.5114, df = 9, p-value = 0.0003746
        95 percent confidence interval: 1.061183 2.538817
Python: TtestResult(statistic=np.float64(5.511351921262151), pvalue=np.float64(0.0003746456530901087), df=np.int64(9))
        ConfidenceInterval(low=np.float64(1.0611825644216402), high=np.float64(2.53881743557836))

Complete the steps:

1. Hypotheses: $H_0: \mu_d =$ [ ] versus $H_1:$ [ ].
2. Standard error of the mean difference: $SE = s_d/\sqrt{n} = 1.0328/\sqrt{10} =$ [ ].
3. Statistic: $t = \bar{d}/SE = 1.8/$[ ] $=$ [ ], with df = [ ].
4. 95% CI: $1.8 \pm t_{0.975,\,9} \times SE = 1.8 \pm 2.262 \times$ [ ] $=$ [ ].
5. Effect size: $d_z = \bar{d}/s_d =$ [ ].
6. Report: Quiz scores rose after the workshop, mean difference = [ ] points, t([ ]) = [ ], p [ ], 95% CI [ ], $d_z$ = [ ].

Now the trap. If you analyse these data with an independent test by mistake (t.test(after, before)), you get t = 1.60 and p = .127: "not significant". The pairing removes the large between-student differences (the before and after scores correlate at r = 0.92), which makes the SE of the paired difference much smaller. Using the wrong design throws that information away.

Exercise C (unsolved): one-sample t-test

Lakemoor University recommends that first-year students study 10 hours per week outside class for STAT 101. Twelve students logged their study hours for one typical week:

Tasks:

1. State $H_0$ and $H_1$ (two-sided, α = 0.05).
2. Compute the mean, SD, SE, t and df by hand. ($t_{0.975,\,11} = 2.201$.)
3. Compute the 95% CI for the mean and the effect size $d = (\bar{x} - \mu_0)/s$.
4. Run t.test(hours, mu = 10) in R and stats.ttest_1samp(hours, 10) in Python to check your answers.
5. Write a one-sentence APA-style result and a one-sentence plain-language conclusion. Does a non-significant result show that students study 10 hours on average?

Reading software output

Reporting template

Fill-in sentence (independent, Welch): [Group 1] (M = [ ], SD = [ ], n = [ ]) scored [higher / lower] than [Group 2] (M = [ ], SD = [ ], n = [ ]) on [outcome], Welch's t([df]) = [t], p = [p], mean difference = [ ] [units], 95% CI [[lower], [upper]], Hedges' g = [ ], 95% CI [[ ], [ ]].

Fill-in sentence (paired): [Outcome] changed from M = [ ] (SD = [ ]) at [time 1] to M = [ ] (SD = [ ]) at [time 2], mean change = [ ], 95% CI [[ ], [ ]], t([n - 1]) = [ ], p = [ ], $d_z$ = [ ].

Fill-in sentence (one-sample): The mean [outcome] (M = [ ], SD = [ ], n = [ ]) [did / did not] differ from the reference value of [μ0], t([n - 1]) = [ ], p = [ ], mean difference = [ ], 95% CI [[ ], [ ]], d = [ ].

Results checklist:

• The design (independent, paired, one-sample) is stated and matches how the data were collected.
• The test version is named (Welch or pooled) and the software default was checked.
• Means, SDs and n for each group or time point are reported.
• t, df, exact p (or p < .001) and the direction of the difference are reported.
• The mean difference with its 95% CI is reported in the original units.
• An effect size with a CI is reported and its formula (d, g or $d_z$) is named.
• Assumption checks and any outliers or exclusions are described.
• One-sided tests, if used, were planned in advance and are labelled.
• The conclusion matches the design (association versus causation).

Common mistakes and misconceptions

• Trusting software defaults across languages. R t.test() defaults to Welch (var.equal = FALSE); SciPy ttest_ind() defaults to the pooled test (equal_var=True). The same data give different t, df and p. Always set the argument explicitly and say which test you ran.
• Using a variance pre-test to choose between pooled and Welch. Two-stage procedures (Levene first, then pick a t-test) distort the Type I error rate. Use Welch unconditionally.
• Analysing paired data as independent. In Worked example B this changes p from < .001 to .127. Pairing is decided by the design, not by which test gives the smaller p-value.
• Running many t-tests for three or more groups. With four groups there are six pairwise tests; the chance of at least one false positive grows well above 5%. Use ANOVA with planned contrasts or a post hoc procedure.
• "p = 0.03 means a 3% chance that H0 is true." No. The p-value is the probability of data at least this extreme if H0 were true. It says nothing directly about the probability of H0 (see the ASA statement on p-values).
• "Not significant means no difference." A non-significant result with a wide CI is inconclusive; the CI may contain both zero and large effects. Absence of evidence is not evidence of absence (see equivalence testing in further reading).
• Testing normality of the raw pooled data. For two groups, check each group separately; for a paired test, check the differences, not the before and after scores.
• Reporting d without saying which d. Cohen's d with pooled SD, Hedges' g and $d_z$ for paired data can differ a lot for the same data. Name the formula.
• Ignoring clustering. Students nested in tutorial groups are not independent. A t-test on individual students then understates the SE; use a mixed model (academic/stats-mixed-models).
• Claiming causation from non-randomised groups. Sections that students choose themselves can differ in prior ability. Only random assignment supports a causal reading.

Self-test questions

1. A colleague runs scipy.stats.ttest_ind(a, b) and you run t.test(a, b) in R on the same data. Why might your results differ, and how do you make them agree on Welch?
2. Why does Welch's test usually give a non-integer df, and between which two values does it lie?
3. A study measures blood pressure in 15 patients before and after a diet. Which t-test applies, and on what variable do you check normality?
4. Two groups give t(38) = 2.10, p = .042, d = 0.66, 95% CI for d [0.02, 1.30]. What does the CI tell you that the p-value does not?
5. Interleaving check: for each design, name the test. (a) Exam scores of 40 students compared with a national mean of 65. (b) Reaction times of 18 participants under two lighting conditions, each person tested in both. (c) Weekly sales of Brightwell Bakery's two shops over 20 weeks, where the question is which shop sells more on average.
6. Why is Hedges' g preferred over Cohen's d in small samples?

My notes and next steps

Sources and further reading

Structure aligned with the NIST/SEMATECH e-Handbook of Statistical Methods (sections on two-sample t-tests) and with the reporting elements named in APA JARS-Quant; all wording, data and examples here are original.

• Student (1908). The probable error of a mean. Biometrika, 6(1), 1–25.
• Welch, B. L. (1947). The generalization of "Student's" problem when several different population variances are involved. Biometrika, 34(1–2), 28–35.
• Satterthwaite, F. E. (1946). An approximate distribution of estimates of variance components. Biometrics Bulletin, 2(6), 110–114.
• Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Lawrence Erlbaum.
• Hedges, L. V. (1981). Distribution theory for Glass's estimator of effect size and related estimators. Journal of Educational Statistics, 6(2), 107–128.
• Delacre, M., Lakens, D., and Leys, C. (2017). Why psychologists should by default use Welch's t-test instead of Student's t-test. International Review of Social Psychology, 30(1), 92–101.
• Wasserstein, R. L., and Lazar, N. A. (2016). The ASA statement on p-values: context, process, and purpose. The American Statistician, 70(2), 129–133.
• NIST/SEMATECH e-Handbook of Statistical Methods, section 1.3.5.3 Two-Sample t-Test for Equal Means, https://www.itl.nist.gov/div898/handbook/ (accessed 2026-10-25). Public domain.
• R Core Team. t.test documentation, package stats, R 4.5.1. SciPy documentation for scipy.stats.ttest_ind, ttest_rel and ttest_1samp, SciPy 1.18.
• Further reading (by name only): de Smith, M. J., Statistical Analysis Handbook (2024 edition, statsref.com); OpenIntro Statistics; American Psychological Association, Publication Manual (7th ed.) and JARS-Quant; Lakens, D. (2017), equivalence tests (TOST), Social Psychological and Personality Science, 8(4), 355–362.

Answer key

Worked example B. (1) $H_0: \mu_d = 0$ versus $H_1: \mu_d \neq 0$. (2) $SE = 1.0328/3.1623 = 0.3266$. (3) $t = 1.8/0.3266 = 5.51$, df = 9. (4) $1.8 \pm 2.262 \times 0.3266 = 1.8 \pm 0.739$, so 95% CI [1.06, 2.54]. (5) $d_z = 1.8/1.0328 = 1.74$. (6) Quiz scores rose after the workshop, mean difference = 1.80 points, t(9) = 5.51, p < .001, 95% CI [1.06, 2.54], $d_z$ = 1.74. The differences look roughly normal (Shapiro-Wilk p = .191) and there are no extreme values.

Exercise C. (1) $H_0: \mu = 10$ versus $H_1: \mu \neq 10$. (2) $\bar{x} = 9.083$, $s = 1.917$, $SE = 1.917/\sqrt{12} = 0.553$, $t = (9.083 - 10)/0.553 = -1.66$, df = 11. (3) 95% CI $9.083 \pm 2.201 \times 0.553 = [7.87,\ 10.30]$; $d = -0.917/1.917 = -0.48$. (4) R and Python both give t = -1.6566, df = 11, p = 0.1258, 95% CI [7.865, 10.301]. (5) "Mean weekly study time (M = 9.08 hours, SD = 1.92, n = 12) did not differ significantly from the recommended 10 hours, t(11) = -1.66, p = .126, 95% CI [7.87, 10.30], d = -0.48." In plain words: the sample studied a little under 10 hours, but with 12 students we cannot rule out that the true average is 10. No, the result does not show that students study 10 hours: the CI runs from about 7.9 to 10.3 hours, so an average well below the target is also plausible. A larger sample (see academic/stats-power-sample-size) or an equivalence test would be needed.

Self-test.

1. SciPy's ttest_ind defaults to the pooled test (equal_var=True); R's t.test defaults to Welch (var.equal = FALSE). Use ttest_ind(a, b, equal_var=False) in Python, or keep the R default.
2. The Welch-Satterthwaite formula weights each group's variance of the mean, so df is a continuous value. It lies between $\min(n_1 - 1, n_2 - 1)$ and $n_1 + n_2 - 2$; it reaches the upper bound when both the sample variances and the sample sizes are equal.
3. Paired t-test (same patients measured twice). Check normality of the 15 differences, not of the raw before and after values.
4. The CI shows the effect is estimated very imprecisely: the data are consistent with a negligible effect (0.02) and with a very large one (1.30). The p-value only says zero is (just) outside the 95% interval.
5. (a) One-sample t-test against 65. (b) Paired t-test (each person in both conditions). (c) Two independent groups (Welch), but check whether the weeks are paired: if both shops are observed in the same 20 weeks, a paired test on weekly differences removes shared week-to-week variation, and autocorrelation over time should also be checked.
6. Cohen's d overestimates the population effect in small samples; Hedges' correction factor $J$ (slightly below 1) removes most of that bias. With n = 22 in Worked example A, d = 1.65 becomes g = 1.59.

Revision history