Power Analysis and Sample Size Template (Statistics, R and Python)

Power Analysis and Sample Size: [Your study question or course]

A worksheet for planning how many participants, units or observations a study needs: the inputs you must decide (α, power, effect size, design, allocation ratio, attrition), analytic power calculations for t-tests, two proportions, one-way ANOVA, correlation and multiple regression in R and Python, simulation-based power for designs without a formula, a sensitivity analysis table, and a reporting sentence. It is for statistics students learning what power means, master's and PhD students who must justify a sample size to a committee or ethics board, and analysts sizing an experiment. It teaches by structure: decide the inputs in order, compute, check with a second tool, then test yourself. The worked examples use fictional data; replace them with your own. Source basis: Cohen (1988) for the effect size conventions and power framework; Lakens (2022) for sample size justification; all text and examples are original.

Document control

This table records the study the calculation belongs to and the software that produced each number. Rows marked Example: show how to fill a field.

What power is and why it matters

Statistical power is the probability that a study will detect an effect of a given size, if that effect really exists. "Detect" means the test gives p below α. Power depends on four quantities, and if you fix any three, the fourth is determined: the significance level α, the effect size, the sample size n, and the power itself (1 - β, where β is the Type II error rate, the chance of missing a real effect). A sample size calculation fixes α, power and the effect size and solves for n. A sensitivity analysis fixes α, power and n and solves for the smallest effect the study can detect.

Why it matters. An underpowered study is not just "less certain". It wastes participants' time, it usually ends in a non-significant result that is hard to interpret, and when it does reach significance the estimate is likely to be exaggerated, because only the luckiest overestimates cross the threshold. An overpowered study spends resources that could have gone elsewhere and can make trivially small effects significant. Planning power is how you match the study to the question.

Power is a property of the design and the assumed effect, not of the data you later collect. Computing "observed power" from your own results after the study adds nothing: it is a direct function of the p-value you already have (Hoenig and Heisey 2001).

The planning workflow

The diagram shows the order of decisions. The design determines the test, the test determines which effect size the software needs, and the effect size must be justified before any calculation.

Input worksheet

The formulas behind the software

The idea is simple. Under the null hypothesis the test statistic is centred at zero; under the alternative it is shifted by an amount that grows with the effect size and with the square root of n. Power is the share of the shifted distribution that lies beyond the critical value. For two independent groups of equal size n, the normal approximation gives:

With α = 0.05 (z = 1.960) and power 0.80 (z = 0.842), $(1.960 + 0.842)^2 = 7.85$, so $n \approx 15.7 / d^2$: for d = 0.5 that is 62.8. The exact calculation uses the noncentral t distribution and gives 63.77, so 64 per group. Halving the effect size quadruples n: that single fact explains why small effects need very large studies.

The exact power for the two-sample t-test is the probability that a noncentral t variable exceeds the critical value:

The other designs use the same idea with a different effect size metric:

Attrition and allocation adjustments:

Analytic power in R and Python

The scripts below solve for n in six standard designs at α = 0.05 (two-sided) and power 0.80: two independent groups (d = 0.5), paired (d_z = 0.4), two proportions (60% vs 45%), one-way ANOVA with three groups (f = 0.25), correlation (r = 0.30) and multiple regression with three predictors (f² = 0.15). The R script also checks a 2:1 allocation and prints a sensitivity table.

library(pwr)   # install.packages("pwr") once
t_two    <- pwr.t.test(d = 0.5, sig.level = 0.05, power = 0.80, type = "two.sample")$n
t_paired <- pwr.t.test(d = 0.4, power = 0.80, type = "paired")$n
h        <- ES.h(0.60, 0.45)                       # Cohen's h for two proportions
two_prop <- pwr.2p.test(h = h, power = 0.80)$n
anova_n  <- pwr.anova.test(k = 3, f = 0.25, power = 0.80)$n     # per group
corr_n   <- pwr.r.test(r = 0.30, power = 0.80)$n
reg_v    <- pwr.f2.test(u = 3, f2 = 0.15, power = 0.80)$v       # denominator df
round(c(t_two = t_two, t_paired = t_paired, h = h, two_prop = two_prop,
        anova_per_group = anova_n, corr = corr_n, reg_v = reg_v, reg_N = reg_v + 3 + 1), 2)
# unequal allocation 2:1, check the power of n1 = 48, n2 = 96
round(pwr.t2n.test(n1 = 48, n2 = 96, d = 0.5)$power, 3)
# base R cross-checks (no package)
round(c(power.t.test(delta = 5, sd = 10, power = 0.80)$n,
        power.prop.test(p1 = 0.60, p2 = 0.45, power = 0.80)$n), 2)
# sensitivity: n = 30 per group
round(pwr.t.test(n = 30, power = 0.80)$d, 3)
round(sapply(c(0.3, 0.5, 0.7, 0.9), function(d) pwr.t.test(n = 30, d = d)$power), 3)
# required n per group for a grid of d and power
grid <- outer(c(0.3, 0.4, 0.5, 0.6, 0.8), c(0.80, 0.90),
              Vectorize(function(d, p) ceiling(pwr.t.test(d = d, power = p)$n)))
dimnames(grid) <- list(d = c(0.3, 0.4, 0.5, 0.6, 0.8), power = c("0.80", "0.90")); grid

          t_two        t_paired               h        two_prop anova_per_group
          63.77           51.01            0.30          172.66           52.40
           corr           reg_v           reg_N
          84.07           72.71           76.71
[1] 0.802
[1]  63.77 172.80
[1] 0.736
[1] 0.208 0.478 0.760 0.929
     power
d     0.80 0.90
  0.3  176  235
  0.4  100  133
  0.5   64   86
  0.6   45   60
  0.8   26   34

import numpy as np
from scipy import stats
from statsmodels.stats.power import (TTestIndPower, TTestPower, NormalIndPower,
                                     FTestAnovaPower, FTestPowerF2)
from statsmodels.stats.proportion import proportion_effectsize

t_two = TTestIndPower().solve_power(effect_size=0.5, alpha=0.05, power=0.80)
t_ratio = TTestIndPower().solve_power(effect_size=0.5, alpha=0.05, power=0.80, ratio=2)
t_paired = TTestPower().solve_power(effect_size=0.4, alpha=0.05, power=0.80)
h = proportion_effectsize(0.60, 0.45)
two_prop = NormalIndPower().solve_power(effect_size=h, alpha=0.05, power=0.80)
anova_total = FTestAnovaPower().solve_power(effect_size=0.25, k_groups=3, alpha=0.05, power=0.80)
reg = FTestPowerF2().solve_power(effect_size=0.15, df_num=3, df_denom=None, alpha=0.05, power=0.80)
# correlation: Fisher z approximation (statsmodels has no correlation power class)
z_a, z_b = stats.norm.ppf(0.975), stats.norm.ppf(0.80)
corr = ((z_a + z_b) / np.arctanh(0.30)) ** 2 + 3
print(f"t two-sample n/group={t_two:.2f}; ratio 2:1 n1={t_ratio:.2f} n2={2 * t_ratio:.2f}")
print(f"paired n={t_paired:.2f}; h={h:.3f}; two proportions n/group={two_prop:.2f}")
print(f"ANOVA total N={anova_total:.2f} ({anova_total / 3:.2f} per group)")
print(f"regression df_denom={reg:.2f} -> N={reg + 3 + 1:.2f}; correlation n={corr:.2f}")
print(f"sensitivity d (n=30/group)={TTestIndPower().solve_power(nobs1=30, alpha=0.05, power=0.80):.3f}")

t two-sample n/group=63.77; ratio 2:1 n1=47.74 n2=95.48
paired n=51.01; h=0.302; two proportions n/group=172.66
ANOVA total N=157.19 (52.40 per group)
regression df_denom=72.71 -> N=76.71; correlation n=84.93
sensitivity d (n=30/group)=0.736

Results after rounding up:

Why the tools differ, and why it matters.

• ANOVA n per group vs total. pwr.anova.test() returns n per group; FTestAnovaPower returns the total number of observations. 52.40 · 3 = 157.19. Mixing the two up gives a study three times too large or too small.
• Correlation. pwr.r.test() uses a refined approximation and gives 84.07; the plain Fisher z formula in the Python code gives 84.93. Both round up to 85, but in general state which method you used.
• Two proportions. pwr.2p.test() and statsmodels use Cohen's h with the arcsine transformation (172.66); base R power.prop.test() uses a different normal approximation (172.80). Both give 173 here.
• Regression. Both tools return the denominator df v, not N. For u tested predictors and no others, N = v + u + 1; if the model also has covariates that are not tested, add them as well.
• Welch vs pooled. The t-test power functions assume equal variances (pooled t). With equal group sizes and similar SDs the Welch test has nearly the same power, so the result transfers; with unequal sizes and unequal SDs, simulate.
• Unequal allocation costs participants. A 2:1 split needs 144 people for the same power that a 1:1 split reaches with 128. Use unequal allocation only for a reason such as cost or ethics.

The chart shows how power grows with n per group for three effect sizes, computed with pwr.t.test(). The dashed line marks 0.80.

Simulation-based power

When no formula fits your planned analysis (a rank test on skewed data, a mixed model, a count model with overdispersion, a test after data-dependent exclusions), estimate power by simulation:

1. Write down a data-generating model that matches what you expect: distributions, effect, variability.
2. Generate one fake data set of the planned size and run exactly the analysis you will run on the real data.
3. Record whether p < α. Repeat thousands of times with a fixed seed.
4. Power is the proportion of replications with p < α. Its Monte Carlo standard error is $\sqrt{\hat{p}(1-\hat{p})/R}$ for R replications; with R = 5,000 and power near 0.80, that is about 0.006.
5. Repeat over a grid of n and pick the smallest n that reaches the target.

Example: Lakemoor wants to compare the time students take to complete an online quiz with an old and a new interface. Times are right-skewed, so the planned test is the Wilcoxon rank-sum (Mann-Whitney) test. The model: log-normal times with median 20 minutes under the old interface, a 30% shorter median under the new one (log-scale shift 0.35), and log-scale SD 0.6.

# Simulated power: skewed task times (minutes), Mann-Whitney / Wilcoxon rank-sum test
set.seed(2026)
sim_power <- function(n, reps = 5000, shift = 0.35) {
  p_w <- p_t <- numeric(reps)
  for (i in seq_len(reps)) {
    old <- rlnorm(n, meanlog = log(20), sdlog = 0.6)
    new <- rlnorm(n, meanlog = log(20) - shift, sdlog = 0.6)
    p_w[i] <- wilcox.test(old, new)$p.value
    p_t[i] <- t.test(old, new)$p.value          # Welch, for comparison
  }
  c(n = n, wilcoxon = mean(p_w < 0.05), welch = mean(p_t < 0.05))
}
round(t(sapply(c(30, 40, 50, 55, 60), sim_power)), 3)

      n wilcoxon welch
[1,] 30    0.588 0.531
[2,] 40    0.707 0.656
[3,] 50    0.790 0.738
[4,] 55    0.837 0.797
[5,] 60    0.873 0.828

import numpy as np
from scipy import stats

rng = np.random.default_rng(2026)

def sim_power(n, reps=5000, shift=0.35):
    p_w = np.empty(reps)
    p_t = np.empty(reps)
    for i in range(reps):
        old = rng.lognormal(mean=np.log(20), sigma=0.6, size=n)
        new = rng.lognormal(mean=np.log(20) - shift, sigma=0.6, size=n)
        p_w[i] = stats.mannwhitneyu(old, new).pvalue
        p_t[i] = stats.ttest_ind(old, new, equal_var=False).pvalue
    return n, (p_w < 0.05).mean(), (p_t < 0.05).mean()

for n in (30, 40, 50, 55, 60):
    print("n=%d  wilcoxon=%.3f  welch=%.3f" % sim_power(n))

n=30  wilcoxon=0.581  welch=0.541
n=40  wilcoxon=0.711  welch=0.658
n=50  wilcoxon=0.795  welch=0.747
n=55  wilcoxon=0.835  welch=0.787
n=60  wilcoxon=0.873  welch=0.835

R and Python use different random number generators, so the same seed gives different streams and the estimates differ by Monte Carlo error (0.790 vs 0.795 at n = 50). Both agree that 50 per group falls just short and 55 per group reaches about 0.84 for the rank test. They also show a design lesson: on skewed data the rank test has more power than the Welch t-test on the raw times. Report: "Power was estimated by simulation (5,000 replications, seed 2026, R 4.5.1) under a log-normal model ..."

Justifying the effect size

The effect size is the input that drives n the most and the one most often chosen badly. Three legitimate routes, in order of preference (Lakens 2022):

1. Smallest effect size of interest (SESOI). Ask what effect would matter in practice or for theory, then power the study to detect that. Example: the Lakemoor exam board considers 5 points (half a grade band) the smallest improvement worth a change in teaching; the score SD in past cohorts is about 10, so d = 0.5. A SESOI is the most defensible choice because it does not depend on guessing the true effect.
2. Prior literature, adjusted for bias. Published effects are inflated by publication bias and by the winner's curse. Use a meta-analytic estimate if one exists, prefer the lower part of its CI, and say how you adjusted.
3. Pilot data, with caution. A pilot effect size is very imprecise. The Lakemoor pilot of 10 + 10 students gave g = 1.06, 95% CI [0.14, 1.95] (see stats-effect-sizes). Planning with d = 1.10 gives 14.0, so 15 per group; if the true effect is 0.5, those 15 per group have power 0.26. Use a pilot mainly to test procedures. If you must plan on a pilot effect, use a conservative value: "safeguard power" (Perugini, Gallucci and Costantini 2014) plans on the lower limit of a two-sided 60% CI, here d = 0.68, which gives 35 per group. Pilot SDs are imprecise too (next paragraph), and when only promising pilots go ahead the follow-up study is planned on an inflated effect (Albers and Lakens 2018).

Where the SD comes from. When the SESOI is set in raw units (5 exam points), the SD you divide by decides d, so it needs its own source. Prefer an SD from a large, similar population: past cohorts, registry or administrative data, or the control arms of earlier studies. An SD estimated from a pilot of 10 per group is itself uncertain: with 9 df, the 95% CI for the population SD runs from about 0.69 to 1.83 times the sample SD, and a small, homogeneous pilot group often understates the spread of the full population. If the only SD you have is from a pilot, plan on a larger value (for example the upper limit of its CI) and say so.

Avoid "we expect a medium effect (d = 0.5) following Cohen (1988)" as the only justification. It says nothing about your study. If you use a benchmark, say why it is a sensible minimum in your field.

Sensitivity analysis

When n is fixed by circumstances, report what the study can detect. With 30 students per group, α = 0.05 two-sided, the smallest d detectable with 80% power is 0.736, and power at other effect sizes is:

And the reverse view, n per group needed for each combination of d and target power:

Worked example A (fully solved): two independent groups with attrition

Question. Lakemoor plans a full trial of flashcard revision against re-reading. How many students should be recruited?

1. Design and test. Two independent groups, randomised; Welch t-test on exam score; α = 0.05, two-sided.
2. Effect size. SESOI of 5 points (set by the exam board). Planning SD 10, taken from the exam records of the past five full cohorts, so d = 5 / 10 = 0.5 (not the pilot's g = 1.06). The pilot SDs (5.5 and 4.2) come from 10 students per group in one seminar, so they are imprecise and likely too small for a whole cohort. Had the plan used SD 5, the same 5-point SESOI would become d = 1.0 and about 17 per group; if the true SD is 10, that study would have power of only about 0.29.
3. Power. 0.80.
4. Compute. pwr.t.test(d = 0.5, power = 0.80) and TTestIndPower().solve_power(effect_size=0.5, ...) both give 63.77, so 64 per group. Base R power.t.test(delta = 5, sd = 10) agrees in raw units.
5. Hand check. Normal approximation: 2 · 7.85 / 0.25 = 62.8, a little below the exact t-based value, as expected.
6. Attrition. 15% of students usually miss the exam: 64 / 0.85 = 75.3, so recruit 76 per group, 152 in total.
7. Check. With 76 per group and d = 0.5, power is 0.865 if nobody drops out, and about 0.80 if 15% drop out.
8. Report sentence. "We will recruit 152 students (76 per group). With 64 students per group after an expected 15% attrition, a two-sided Welch t-test at α = .05 has 80% power to detect a difference of 5 exam points (d = 0.5), the smallest improvement the exam board considers worth a change in teaching. Calculations used the pwr package 1.3.0 in R 4.5.1 and were checked with statsmodels 0.15.0."

Worked example B (half solved): two proportions

Question. Lakemoor wants to know whether a weekly reminder email raises the pass rate in an introductory course from 45% to 60%.

1. Design and test: two independent groups, binary outcome, test of two proportions; α = 0.05, two-sided, power 0.80.
2. Effect size metric needed by pwr.2p.test(): Cohen's [ ].
3. h = 2 arcsin √0.60 - 2 arcsin √0.45 = [ ] (to three decimals).
4. n per group from the code output: 172.66, so [ ] per group.
5. Expected attrition 10%: n to recruit per group = [ ] / 0.90 = [ ], rounded up to [ ]; total [ ].
6. Base R power.prop.test() gives 172.80. Why does it differ slightly from pwr.2p.test()? [ ]
7. Write the report sentence: [ ].

Exercise C (unsolved)

(a) A Harbour City Library analyst wants to detect a correlation of r = 0.25 between weekly opening hours and visits across branch-weeks, with α = 0.05 two-sided and power 0.90. How many observations are needed?

(b) A Brightwell Bakery quality study will regress loaf volume on four process variables (proofing time, oven temperature, hydration, flour protein). The team wants 80% power to detect f² = 0.10 for the set of four predictors. How many loaves must be measured?

Reporting template and checklist

Fill-in sentence:

A sample size of [N total] ([n per group] per group, [allocation]) was planned to give [power]% power to detect [effect size and metric, with raw units] using a [two-sided / one-sided] [test] at α = [ ]. The effect size was based on [SESOI / meta-analysis / pilot, with reason]. The target was inflated by [attrition]% for expected attrition. Calculations used [software and version][; simulation with R replications and seed].

Checklist:

• The primary outcome and the test are named.
• α, sidedness and power are stated.
• The effect size is in the metric the software needs and in raw units.
• The effect size is justified (SESOI, literature or pilot) with a source.
• n is rounded up, inflated for attrition and rounded up again.
• Software and version (and seed and replications for simulation) are given.
• A sensitivity analysis is included if n was constrained.
• Multiplicity is handled if there is more than one primary test.
• No "observed power" calculated after the study.

Common mistakes and misconceptions

Self-test questions

1. Name the four quantities linked in a power analysis. If you know three, what can you compute?
2. Using the normal approximation, roughly how many students per group do you need for d = 0.25 at 80% power?
3. A study planned 64 per group for d = 0.5, but only 40 per group finished. What is the sensitivity analysis question you should now answer?
4. Why is a SESOI usually a better basis for the effect size than a pilot study?
5. statsmodels' FTestAnovaPower returned 157.19 for three groups. How many participants per group do you recruit?
6. In a simulation with 2,000 replications, estimated power is 0.80. What is its Monte Carlo standard error?

My notes and next steps

Sources and further reading

Based on the following sources, cited by name; all text, notation and examples here are original.

• Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Lawrence Erlbaum Associates.
• Lakens, D. (2022). Sample size justification. Collabra: Psychology, 8(1), 33267.
• Hoenig, J. M., and Heisey, D. M. (2001). The abuse of power: the pervasive fallacy of power calculations for data analysis. The American Statistician, 55(1), 19–24.
• Perugini, M., Gallucci, M., and Costantini, G. (2014). Safeguard power as a protection against imprecise power estimates. Perspectives on Psychological Science, 9(3), 319–332.
• Albers, C., and Lakens, D. (2018). When power analyses based on pilot data are biased: inaccurate effect size estimators and follow-up bias. Journal of Experimental Social Psychology, 74, 187–195.
• Champely, S. pwr: Basic Functions for Power Analysis, R package version 1.3-0.
• Seabold, S., and Perktold, J. (2010). statsmodels: econometric and statistical modeling with Python. Proceedings of the 9th Python in Science Conference, 92–96.
• Mann, H. B., and Whitney, D. R. (1947). On a test of whether one of two random variables is stochastically larger than the other. Annals of Mathematical Statistics, 18(1), 50–60.
• NIST/SEMATECH e-Handbook of Statistical Methods, section 7.2.2.2 Sample sizes required, https://www.itl.nist.gov/div898/handbook/ (accessed 2026-10-25). Public domain.
• Further reading: de Smith, M. J. (2024). Statistical Analysis Handbook, 2024 edition. StatsRef, https://www.statsref.com/; OpenIntro Statistics.

Answer key

Example B. (2) Cohen's h. (3) 2 · arcsin(0.7746) - 2 · arcsin(0.6708) = 2 · 0.8861 - 2 · 0.7353 = 0.302. (4) 173 per group. (5) 173 / 0.90 = 192.2, rounded up to 193 per group; 386 total. (6) power.prop.test() uses a normal approximation on the raw proportion scale with pooled and unpooled variances, while pwr.2p.test() works on the arcsine scale where the variance does not depend on p; the methods differ slightly, and both round to 173. (7) "We will recruit 386 students (193 per group). With 173 per group after 10% attrition, a two-sided test of two proportions at α = .05 has 80% power to detect an increase in the pass rate from 45% to 60% (h = 0.30)."

Exercise C. (a) pwr.r.test(r = 0.25, power = 0.90) gives n = 163.18; the Fisher z formula gives 164.07. Both round up to 164 observations. (b) pwr.f2.test(u = 4, f2 = 0.10, power = 0.80) gives v = 119.19, and statsmodels FTestPowerF2 gives the same denominator df; v rounds up to 120, so N = 120 + 4 + 1 = 125 loaves. The trap is reporting 120 as the sample size.

Self-test.

1. α, power, effect size and n. Any three determine the fourth: n (sample size calculation), the detectable effect (sensitivity), power (power calculation for a fixed design) or the α needed.
2. n ≈ 15.7 / 0.0625 = 251.2 per group; the exact t-based value is 252.13, so 253 per group.
3. "What effect could 40 per group detect with 80% power?" Here d = 0.634 (pwr.t.test(n = 40, power = 0.80); between 0.6, which needs 45, and 0.8, which needs 26), so an effect of 0.5 is now likely to be missed and a null result is weak evidence of no effect.
4. A SESOI is fixed by what matters, not estimated from a tiny noisy sample; pilot effect sizes are imprecise and, when only promising pilots go ahead, biased upward, which leads to underpowered main studies.
5. 157.19 is the total. 157.19 / 3 = 52.40, so 53 per group, 159 total (plus attrition).
6. √(0.80 · 0.20 / 2000) = √0.00008 = 0.0089, about 0.009.

Revision history