Sampling Plan Template (Statistics, Survey Design, R and Python)

Sampling Plan: [Your study population and question]

A working sampling plan for your own survey, study or quality check: it defines the target population, the sampling frame and the unit, chooses a probability or non-probability design, lists the bias risks, and sizes the sample with a margin-of-error calculation, with seeded R and Python code that draws a stratified sample. It is for methods students, thesis writers planning data collection and analysts who must defend how their data were chosen. It speeds up learning by making you answer the same five questions in the same order for every study, so the logic of "who could have been in my data" becomes automatic. The worked examples use fictional data; replace them with your own. Source basis: classic survey sampling texts (Cochran 1977, Kish 1965, Lohr 2021) and the NIST/SEMATECH e-Handbook, written in original words.

Document control

What a sampling plan decides

Almost every statistical method assumes that the data are a random sample from some population. A sampling plan is where you make that assumption true, or at least honest. It answers five questions:

1. Who or what do I want to describe? (target population)
2. Which list or procedure can I actually draw from? (sampling frame)
3. What is one "case"? (sampling unit and observation unit)
4. How are cases chosen? (design: probability or not, and which one)
5. How many do I need, and how many will I invite? (sample size and expected response)

The diagram shows where each kind of error enters. Each step loses or distorts part of the population, and only one of these errors (sampling error) is measured by a standard error or a confidence interval. The others must be handled by design and described in your limitations.

Population, frame and unit

A frame is good when every member of the target population appears exactly once and you have the information needed to stratify or contact them. Duplicates (one person listed twice) give that person a double chance of selection; missing groups get zero chance, and no sample size can fix a zero.

Probability sampling designs

In a probability sample, every unit in the frame has a known, non-zero chance of being selected, and chance alone (a seeded random number generator, not the researcher) makes the selection. That known chance is what lets you compute standard errors, confidence intervals and design weights.

Two technical terms appear in every design. The inclusion probability $\pi_i$ is the chance that unit $i$ is selected. The design weight is its inverse, $w_i = 1/\pi_i$: the number of population units each sampled unit "stands for". In SRS all weights are equal, so they can be ignored for means and proportions. In a stratified design with proportional allocation (each stratum sampled at the same fraction), weights are also equal. With disproportionate allocation (for example, oversampling a small faculty so it can be analysed on its own), the weights differ and must be used for whole-population estimates. Neyman (1934) showed that, for a fixed total n, the most precise allocation samples more heavily from strata that are larger and more variable.

For cluster designs, the precision loss is summarised by the design effect. With clusters of average size $m$ and intraclass correlation $\rho$ (how alike people in the same cluster are):

$\text{deff} \approx 1 + (m - 1)\rho$

If $m = 21$ students per seminar group and $\rho = 0.05$, then deff $= 1 + 20 \times 0.05 = 2$: a cluster sample of 400 students carries about as much information as a simple random sample of 200.

Non-probability designs and their limits

Non-probability samples are chosen by convenience, judgement or self-selection. They are fast and sometimes the only option (pilot studies, hard-to-reach groups, early product research), but the inclusion probabilities are unknown, so the usual standard errors describe only chance variation within an unknown selection process.

The honest rule for reporting a non-probability sample: describe who took part, compare their profile with known population figures where possible, and present uncertainty statements with an explicit caveat. A large non-probability sample can be precisely wrong: more cases shrink the standard error but do not remove selection bias.

Choosing a design

Start at the top and answer each question about your frame. The first question is the most important: without a complete list of units, simple random, systematic and stratified designs are not possible, and the choice is between sampling groups (clusters) and a non-probability design with stated limits. When several designs are possible, prefer the one that guarantees the subgroups you must report and keeps the analysis as simple as the question allows.

Bias sources to plan against

Bias is a systematic difference between what the sample tells you and the population truth. It does not shrink when the sample grows, which is why it matters more than random error in large studies.

Survivorship bias has a classic story: in the Second World War, the statistician Abraham Wald pointed out that the bullet holes on returning aircraft showed where planes could be hit and still return, so armour belonged where the returning planes had no holes. The planes that did not return were the missing data.

Response rate. Plan for it. If you expect 40% of invited students to respond and need 351 completes, invite $351 / 0.40 \approx 878$. A higher invitation number does not cure nonresponse bias; it only protects the sample size.

Sample size and margin of error for a proportion

Plain language first. The margin of error (MOE) is half the width of a 95% confidence interval. It says how far the sample percentage is likely to be from the population percentage because of chance alone, assuming a probability sample and no other bias. It shrinks with the square root of the sample size n: to halve the MOE you need about four times as many cases. When the sample is a large share of a finite population (more than about 5%), a finite population correction (FPC) makes the MOE smaller, because you have already seen much of the population.

Formula. For a proportion $p$ estimated from a simple random sample of size $n$ from a population of size $N$:

$\text{MOE} = z_{1-\alpha/2}\sqrt{\frac{p(1-p)}{n}}\;\sqrt{\frac{N-n}{N-1}}$

To plan the sample size for a target margin $e$, first ignore the population size, then correct:

$n_0 = \frac{z_{1-\alpha/2}^{2}\,p(1-p)}{e^{2}}, \qquad n = \frac{n_0}{1 + \dfrac{n_0 - 1}{N}}$

The full planning chain is: $n_0$, then FPC, then multiply by the design effect, then divide by the expected response rate. For hypothesis tests and comparisons between groups, the question is power rather than margin of error: see academic/stats-power-sample-size.

The chart shows how the MOE for $p = 0.5$ falls as the sample grows from a population of 4,000, with and without the FPC. The curves are close for small samples and separate once n is a large fraction of N.

Worked example A: a stratified sample of Lakemoor students

Setting. The Lakemoor University (fictional) student union wants to pilot a library-use questionnaire with 40 students from a frame of 400 students living in the two residence halls. The frame lists each student's faculty: ARTS 120, SCI 160, BUS 80, ENG 40. Faculties are expected to use the library differently, and the union wants every faculty present, so a stratified design with proportional allocation is chosen.

Step 1: allocation by hand. The sampling fraction is $40/400 = 0.10$. Proportional allocation gives each stratum 10% of its size: ARTS $120 \times 0.10 = 12$, SCI 16, BUS 8, ENG 4. Total 40. Each sampled student represents $1/0.10 = 10$ students, so every design weight is 10.

Step 2: draw the sample with a seed. A seed makes the random draw reproducible: anyone running the script gets the same students. Record the seed in the plan before drawing, so nobody can re-draw until a "nicer" sample appears.

# Sampling frame: 400 Lakemoor University students (fictional)
frame <- data.frame(
  student_id = sprintf("LU%03d", 1:400),
  faculty    = rep(c("ARTS", "SCI", "BUS", "ENG"), times = c(120, 160, 80, 40))
)
n_total <- 40
N_h <- table(frame$faculty)
n_h <- round(n_total * N_h / sum(N_h))     # proportional allocation
print(n_h)
set.seed(2026)
idx <- unlist(lapply(names(n_h), function(h) {
  rows <- which(frame$faculty == h)
  rows[sample.int(length(rows), n_h[[h]])]
}))
smp <- frame[idx, ]
smp$weight <- as.numeric(N_h[smp$faculty] / n_h[smp$faculty])
print(table(smp$faculty))
print(head(smp[smp$faculty == "ENG", ], 4), row.names = FALSE)
# Sample size for a proportion, margin 0.05, with finite population correction
z <- qnorm(0.975); p <- 0.5; e <- 0.05; N <- 4000
n0 <- z^2 * p * (1 - p) / e^2
n_fpc <- n0 / (1 + (n0 - 1) / N)
cat(sprintf("n0 = %.1f, n with FPC = %.1f -> %d\n", n0, n_fpc, ceiling(n_fpc)))
moe <- function(n, N, p = 0.5) z * sqrt(p * (1 - p) / n) * sqrt((N - n) / (N - 1))
cat(sprintf("MOE at n = 200: %.4f (no FPC %.4f)\n", moe(200, N), z * sqrt(0.25 / 200)))

ARTS  BUS  ENG  SCI 
  12    8    4   16 

ARTS  BUS  ENG  SCI 
  12    8    4   16 
 student_id faculty weight
      LU397     ENG     10
      LU395     ENG     10
      LU372     ENG     10
      LU363     ENG     10
n0 = 384.1, n with FPC = 350.6 -> 351
MOE at n = 200: 0.0675 (no FPC 0.0693)

import numpy as np, pandas as pd
from scipy import stats
frame = pd.DataFrame({
    "student_id": [f"LU{i:03d}" for i in range(1, 401)],
    "faculty": np.repeat(["ARTS", "SCI", "BUS", "ENG"], [120, 160, 80, 40]),
})
n_total = 40
N_h = frame["faculty"].value_counts().sort_index()
n_h = (n_total * N_h / N_h.sum()).round().astype(int)
print(n_h.to_dict())
rng = np.random.default_rng(2026)
parts = [g.sample(n=n_h[h], random_state=rng) for h, g in frame.groupby("faculty")]
smp = pd.concat(parts)
smp["weight"] = smp["faculty"].map(N_h / n_h)
print(smp["faculty"].value_counts().sort_index().to_dict())
print(smp[smp["faculty"] == "ENG"].head(4).to_string(index=False))
z = stats.norm.ppf(0.975); p = 0.5; e = 0.05; N = 4000
n0 = z**2 * p * (1 - p) / e**2
n_fpc = n0 / (1 + (n0 - 1) / N)
print(f"n0 = {n0:.1f}, n with FPC = {n_fpc:.1f} -> {int(np.ceil(n_fpc))}")
moe = lambda n: z * np.sqrt(p * (1 - p) / n) * np.sqrt((N - n) / (N - 1))
print(f"MOE at n = 200: {moe(200):.4f} (no FPC {z*np.sqrt(0.25/200):.4f})")

{'ARTS': 12, 'BUS': 8, 'ENG': 4, 'SCI': 16}
{'ARTS': 12, 'BUS': 8, 'ENG': 4, 'SCI': 16}
student_id faculty  weight
     LU394     ENG    10.0
     LU382     ENG    10.0
     LU369     ENG    10.0
     LU366     ENG    10.0
n0 = 384.1, n with FPC = 350.6 -> 351
MOE at n = 200: 0.0675 (no FPC 0.0693)

Step 3: read the output. Both languages allocate 12, 8, 4 and 16 students and give every student a weight of 10. The selected IDs differ (R picks LU397, LU395, LU372, LU363 among engineers; Python picks LU394, LU382, LU369, LU366). That is not an error: R and numpy use different random number generators, so the same seed number gives different streams. The design is the same; the draw is reproducible only within one language and version. Record the language, version and seed in the plan.

Step 4: size the main survey by hand. For the full survey of all 4,000 students, the union wants a 95% MOE of 5 percentage points with $p$ unknown, so $p = 0.5$:

$n_0 = 1.96^2 \times 0.25 / 0.05^2 = 3.8415 \times 0.25 / 0.0025 = 384.1$.

FPC: $n = 384.1 / (1 + 383.1/4000) = 384.1 / 1.0958 = 350.6$, rounded up to 351 completed questionnaires. With an expected response rate of 40%, invite $351 / 0.40 \approx 878$ students.

The second output line shows the gain from the FPC at $n = 200$: the MOE is 6.75 points instead of 6.93 points. At this sampling fraction (5%) the correction is small; it matters more for small populations such as one residence hall or one bakery's daily output.

Interpretation and report sentence. "A stratified random sample of 40 students (proportional allocation by faculty, seed 2026, R 4.5.1) was drawn from the residence-hall list of 400 students; all design weights equal 10. The main survey requires 351 completed responses for a 95% margin of error of ±5 percentage points (N = 4,000, finite population correction applied), and 878 invitations at an expected 40% response rate."

Worked example B: Brightwell Bakery loaf inspection (half solved)

Setting. Brightwell Bakery (fictional) bakes 2,400 loaves per day on one line. Quality control will weigh 60 loaves per day. The loaves come off the line in trays of 8, and position 1 of every tray sits nearest the oven door, where loaves bake slightly lighter.

Exercise C: Harbour City Library user survey

Harbour City Library (fictional) wants to estimate the percentage of its 12,000 registered card holders who would use an evening opening, with a 95% margin of error of ±4 percentage points. Nothing is known about the likely percentage. Past surveys had a 40% response rate. The frame is the card-holder database.

Tasks: (a) compute $n_0$ and the FPC-corrected $n$; (b) how many card holders should be invited? (c) name one group in the target population "people who might use an evening opening" that is not in the frame, and the type of bias it causes; (d) if the library instead hands questionnaires to visitors at the desk during the day, name the design and its main bias.

Plan table to fill in

Common mistakes and misconceptions

Self-test questions

1. What is the difference between the target population and the sampling frame? Give a Lakemoor example of a gap.
2. Why does a stratified sample with proportional allocation not need weights for a whole-population proportion, while an equal allocation of 10 per faculty (ARTS 120, SCI 160, BUS 80, ENG 40) does? Give the four weights.
3. A cluster sample uses seminar groups of 26 students with $\rho = 0.04$. What is the design effect, and what effective sample size do 520 students give?
4. Halving the margin of error requires multiplying the sample size by about how much? Why?
5. Name the bias: a study of successful bakeries' habits based only on bakeries still open after five years.
6. A probability survey of n = 351 reports 42% weekly library use with a 95% MOE of ±5 points. A student says, "there is a 95% chance the true value is between 37% and 47%." What is the more accurate statement?

My notes and next steps

Sources and further reading

Based on standard survey sampling theory as presented in the texts below, written in original words; the margin of error and finite population correction formulas are standard results.

• Cochran, W. G. (1977). Sampling Techniques (3rd ed.). Wiley.
• Kish, L. (1965). Survey Sampling. Wiley (design effect).
• Lohr, S. L. (2021). Sampling: Design and Analysis (3rd ed.). CRC Press.
• Neyman, J. (1934). On the two different aspects of the representative method: the method of stratified sampling and the method of purposive selection. Journal of the Royal Statistical Society, 97(4), 558–625 (with discussion; optimal allocation).
• Horvitz, D. G., and Thompson, D. J. (1952). A generalization of sampling without replacement from a finite universe. Journal of the American Statistical Association, 47(260), 663–685 (inverse-probability weights).
• Groves, R. M., Fowler, F. J., Couper, M. P., Lepkowski, J. M., Singer, E., and Tourangeau, R. (2009). Survey Methodology (2nd ed.). Wiley (total survey error).
• NIST/SEMATECH e-Handbook of Statistical Methods, sections 7.2.2.2 and 7.2.4.2 Sample sizes required, https://www.itl.nist.gov/div898/handbook/ (accessed 2026-10-25). Public domain.
• Further reading: de Smith, M. J., Statistical Analysis Handbook (2024 edition, statsref.com); Diez, Çetinkaya-Rundel and Barr, OpenIntro Statistics (chapter on sampling principles).

Answer key

Worked example B. Sampling unit: one loaf. $k = 2400/60 = 40$. Random start between 1 and 40. Yes, 40 is a multiple of 8 ($40 = 5 \times 8$), so every sampled loaf has the same tray position and the sample is biased (all light, if the start falls on position 1, or never including the light position otherwise). Fix: a simple random sample of 60 positions, a stratified sample with tray position as strata, or an interval that is not a multiple of 8 (for example $k = 39$ with a check that it covers the day). FPC: $\sqrt{2340/2399} = \sqrt{0.97541} = 0.988$.

Exercise C. (a) $n_0 = 1.96^2 \times 0.25 / 0.04^2 = 600.2$; $n = 600.2 / (1 + 599.2/12000) = 571.7$, round up to 572. (b) $572 / 0.40 = 1430$ invitations. (c) People who do not hold a card (for example residents who stopped visiting because of the current hours): coverage bias, likely missing exactly those who want evening hours. (d) A convenience sample of daytime visitors: selection bias, because people who can visit in the day are less likely to need evening hours; a margin of error would not be meaningful.

Self-test.

1. The target population is who you want to describe; the frame is the list you draw from. Example: target = all enrolled students; frame = residence-hall list, which misses commuters (undercoverage).
2. With proportional allocation every student has the same inclusion probability (0.10), so all weights are equal and cancel. With 10 per faculty, inclusion probabilities differ, giving weights ARTS 12, SCI 16, BUS 8, ENG 4.
3. deff $= 1 + (26 - 1) \times 0.04 = 2.0$; effective sample size $520 / 2.0 = 260$.
4. About four times, because the MOE is proportional to $1/\sqrt{n}$, and $1/\sqrt{4n} = \tfrac{1}{2} \cdot 1/\sqrt{n}$.
5. Survivorship bias.
6. "The procedure used produces intervals that contain the true value in about 95% of repeated samples; this interval is 37% to 47%." The 95% describes the method, not a probability for this one interval, and it covers sampling error only, not nonresponse or coverage bias (see academic/stats-confidence-intervals).

Revision history