Which Test Practice Set Template (Statistics, 30 Interleaved Scenarios)

Which Test? Interleaved Practice Set: [Your course or exam]

An interleaved practice set for the skill that most statistics exams and most thesis supervisors test first: choosing the right method. It contains 30 short, realistic scenarios mixed across every method family (t-tests, rank tests, ANOVA designs, correlation, linear and generalised regression, mixed models, contingency tables, proportions, time series, reliability and resampling). For each scenario you name the method, the effect size you would report and one assumption you would check, then score yourself with the rubric and read the reasons in the answer key. It is for students preparing for an exam, for master's and PhD students who want to check their judgement before planning a thesis analysis, and for analysts who want a quick refresher. It speeds up learning by mixing problem types (interleaving), so you practise telling methods apart instead of repeating one method in a block, and by making you retrieve the answer before you see it. The scenarios use fictional settings and data; replace them with your own. Source basis: the learning-science reviews of Dunlosky et al. (2013) and Weinstein and Sumeracki (2018), and the method papers cited in each method card, written up in original words and scenarios.

Document control

How this practice set works

Most textbook exercises come in blocks: a chapter on t-tests ends with ten t-test problems. Blocked practice feels smooth, because you already know which method to use before you read the question. The exam, the thesis and the job do not tell you which chapter you are in. The hard part is recognising the problem type, and blocked practice never trains it.

This set is interleaved: consecutive scenarios come from different method families, so each one forces you to decide what kind of problem it is. Interleaving feels harder and slower than blocked practice, and learners often think it is working less well; controlled studies of mathematics and category learning find the opposite on delayed tests. The difficulty is the point. The set also uses retrieval practice (you answer before you see the key), immediate feedback (brief reasons in the key) and spacing (three sessions on different days).

For each scenario write three things:

1. Method: the specific test or model, including the variant (Welch, not just "t-test"; McNemar, not just "chi-square").
2. Effect size: what you would report alongside the p-value (Hedges' g, Cramér's V, odds ratio, rate ratio, R²...).
3. One assumption to check: the one most likely to fail or to matter in that design, and how you would check it.

Structure aligned with the evidence summaries of Dunlosky et al. (2013) and Weinstein and Sumeracki (2018) on retrieval practice, interleaving and spacing, and with the five-question routine in academic/stats-test-selector.

The answer routine

Read each scenario with a pen. Underline the outcome variable and write its type next to it. Circle the words that reveal the design: "the same", "before and after", "each judge", "matched", "within", "nested in". Count the groups. Only then name a method. If two methods seem possible, look them up in the confusable pairs table.

Outcome types and their first families, as a memory hook:

Worked example A (fully solved)

Scenario. Lakemoor University (fictional) offers two versions of an online statistics module. Forty-two students take version A and thirty-nine take version B; each student takes only one version. The outcome is the final module score out of 100. Is the mean score different between versions?

• Outcome: final score, numeric (0 to 100).
• Design words: "each student takes only one version", so the groups are independent.
• Groups: two.
• Method: Welch's t-test (the default for two independent groups; it does not assume equal variances).
• Effect size: Hedges' g (or Cohen's d) with a 95% CI, plus the mean difference in points with its CI.
• Assumption: no extreme outliers and approximately normal scores per group; with 39 or more per group, the means are close to normal anyway, so the main check is a dot plot for outliers (for example a score of 0 from a student who never logged in, which is a different question).
• Score: 4 out of 4 points.

Worked example B (half solved)

Scenario. Brightwell Bakery (fictional) asks 15 tasters to rate both a rye loaf and a spelt loaf on a single 1 to 7 liking scale. Each taster rates both loaves, in random order. Do tasters like one loaf more?

• Outcome: liking rating, [ ] (single 7-point item).
• Design words: "each taster rates both", so the ratings are [ ].
• Method: [ ] signed-rank test.
• Effect size: [ ].
• Assumption: the differences between the two ratings are roughly [ ] around their median; if not, use the sign test.

Exercise C (unsolved)

Scenario. Harbour City Library (fictional) records whether each of 250 visitors used the self-service kiosk (yes or no) and the visitor's age group (under 30, 30 to 59, 60 and over). Does kiosk use depend on age group?

Write the method, the effect size and one assumption to check.

Answer sheet

The 30 scenarios

All settings are fictional. Assume α = 0.05, two-sided tests, and independent units unless the scenario says otherwise.

Set 1 (scenarios 1 to 10)

1. Two seminar groups in Lakemoor University's economics course (35 and 38 students, different students in each) sat the same mid-term. Compare the mean mid-term mark of the two groups.
2. Lakemoor surveys 300 students: faculty (Arts, Science, Business, Engineering) and preferred study place (library, home, café). Is preferred study place related to faculty?
3. Twelve Brightwell Bakery bakers each shaped a batch of loaves with the old bench layout and, a week later, with the new layout. Shaping time per batch (minutes) was recorded for both. Did the new layout change the mean time?
4. Harbour City Library records the number of e-book loans per day for 180 days, along with whether a promotion banner was shown that day and the day of the week. Does the banner change the number of loans?
5. A tasting panel ranks 15 Brightwell products from best to worst; the bakery also ranks the same 15 products by last month's sales. How strongly do taste rank and sales rank agree?
6. Eighty Lakemoor students answer "Do you understand what a p-value means?" (yes or no) before and after a one-hour workshop. Did the proportion answering yes change?
7. Brightwell tests three flour suppliers. Thirty loaves are randomly assigned, ten per supplier, and each loaf's weight (grams) is measured. Residuals look normal and the group SDs are similar. Do mean weights differ?
8. Lakemoor wants to predict whether first-year students pass the statistics exam (pass or fail) from lecture attendance (%) and prior maths grade, using records of 410 students.
9. Customers of two campus canteens rate satisfaction on a single 1 to 7 item (25 and 31 respondents, different people). Is satisfaction different between canteens?
10. Harbour City Library compares the number of days books are returned late at three branches (about 20 late returns per branch). The distributions are strongly right-skewed with a few returns more than 90 days late. Do the branches differ?

Set 2 (scenarios 11 to 20)

11. Twenty Lakemoor students complete a 10-item statistics anxiety scale (sum score 10 to 50) in weeks 1, 5 and 10. No data are missing. Does mean anxiety change over the term?
12. Harbour City Library logs 600 loans by weekday (Monday to Saturday). Are loans spread equally across the six days?
13. A Lakemoor housing study models monthly rent (in local currency) of 220 student rooms from distance to campus (km), room size (m²) and whether the room is furnished.
14. Brightwell tests a natural preservative: 1 of 12 treated loaves and 6 of 10 untreated loaves showed mould after five days. Is mould related to treatment?
15. Brightwell's sourdough is labelled 800 g. A trading-standards check weighs 25 loaves. Is the mean weight different from 800 g?
16. Forty Lakemoor students report average nightly sleep (hours) and complete a reaction-time test (milliseconds). The scatter plot looks roughly linear with no extreme points. How strongly are sleep and reaction time associated?
17. Brightwell bakes loaves at two oven temperatures and three proofing times, five loaves per combination (30 loaves), measuring loaf volume. Does the effect of proofing time depend on oven temperature?
18. Fourteen students rate their stress on a single 0 to 10 item the week before exams and the week after. Did stress change?
19. Lakemoor tests a new feedback method assigned to 8 of 15 tutorial groups (about 20 students per group, 300 in total). The outcome is each student's final quiz average. Does the feedback method raise scores?
20. Lakemoor sends two versions of an assignment reminder email to two random halves of 2,400 students (1,200 each) and records whether each student submitted on time. Do the on-time proportions differ?

Set 3 (scenarios 21 to 30)

21. Sixty Lakemoor students are randomly assigned to two teaching methods. All take a pre-test and a post-test. Compare post-test scores between methods, making use of the pre-test.
22. Harbour City Library models the number of items each member borrowed last year (0 to about 200; the variance is many times the mean) from age group and membership type.
23. Ten judges at a bread festival each rank four sourdough recipes from 1 (best) to 4. Do the recipes differ in rank?
24. In a Brightwell taste test, 18 of 20 customers prefer the new croissant recipe to the old one. Is the preference different from 50%?
25. Harbour City Library has daily visitor counts for three years and wants to forecast daily visitors for the next 30 days.
26. Lakemoor develops an 8-item questionnaire on study engagement (each item 1 to 5) and gives it to 250 students. Do the items form a reliable scale?
27. Thirty students attempt one probability question on each of three quizzes (weeks 3, 6 and 9) and are scored correct or incorrect each time. Does the proportion correct change across quizzes?
28. Lakemoor records the number of days until 18 new library members first borrow a book. The data are strongly skewed. The library wants a 95% CI for the median.
29. Four campus shops are compared on transaction amount (n = 45, 30, 60 and 25 transactions). The SDs differ by a factor of three across shops; the distributions are only mildly skewed. Do mean transaction amounts differ?
30. Brightwell has bought a second oven and wants to show that loaves from the two ovens have the same mean weight, within ±10 g, using 40 loaves from each oven.

Scoring rubric

Error tally by family (fill in after each session; the family with the most misses is your next study topic):

The chart below is an Example of a learner's total score across three spaced sessions. This learner rose steadily; your own scores may stay flat or dip after a gap before rising, and that is normal: a dip after a gap shows that retrieval is effortful, which is what makes it work.

Confusable pairs

Common mistakes and misconceptions

• Naming only the family. "A t-test" or "chi-square" is not enough; the variant (Welch, paired, McNemar, Fisher's exact) is often the whole point of the question.
• Deciding pairing from the numbers. Two columns of equal length are not necessarily paired. Pairing comes from the design: the same unit, or units deliberately matched.
• Choosing a rank test because n is small. Small n alone is not a reason; ordinal data, strong skew or outliers are. A rank test on a small sample also has little power.
• Using a chi-square test on paired yes or no data. The same person counted twice breaks independence.
• Treating counts as normal. Counts are bounded at zero and often skewed; model them with Poisson or negative binomial regression.
• Forgetting clustering. Students in tutorial groups, loaves in batches, visits by the same member: if the treatment is assigned to groups, analyse at the group level or with a mixed model.
• Reading "not significant" as "the same". To show equivalence you need an equivalence test with a margin set in advance.
• Reporting only p. Every answer needs an effect size; "p < .05" says nothing about size or importance.

Self-test questions

1. Why does this set mix method families instead of grouping all t-test scenarios together?
2. Scenario 21 could be answered with "ANCOVA" or with "multiple regression with group and pre-test". Are both correct, and what would you report?
3. Which single word or phrase in scenario 27 rules out a chi-square test of independence?
4. A learner scores 2 out of 4 on scenario 22 by answering "Poisson regression, rate ratio, independence". What did they miss, and which phrase in the scenario was the tell?
5. Why is a spaced second session (two days later) more useful than redoing the set immediately?

My notes and next steps

Sources and further reading

• Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., and Willingham, D. T. (2013). Improving students' learning with effective learning techniques. Psychological Science in the Public Interest, 14(1), 4–58.
• Weinstein, Y., and Sumeracki, M. (2018). Understanding How We Learn: A Visual Guide (illustrated by O. Caviglioli). Routledge.
• Rohrer, D., and Taylor, K. (2007). The shuffling of mathematics problems improves learning. Instructional Science, 35(6), 481–498.
• Roediger, H. L., and Karpicke, J. D. (2006). Test-enhanced learning: taking memory tests improves long-term retention. Psychological Science, 17(3), 249–255.
• American Statistical Association (2016). Guidelines for Assessment and Instruction in Statistics Education (GAISE) College Report.
• NIST/SEMATECH e-Handbook of Statistical Methods, section 1.3.5 Quantitative Techniques and chapter 7 Product and Process Comparisons, https://www.itl.nist.gov/div898/handbook/ (accessed 2026-10-25). Public domain.
• Further reading (by name only): de Smith, M. J., Statistical Analysis Handbook (2024 edition); Diez, Çetinkaya-Rundel and Barr, OpenIntro Statistics.

Answer key

Example B. Outcome: ordinal (single 7-point item). Ratings are paired (each taster rates both). Method: Wilcoxon signed-rank test. Effect size: matched-pairs rank-biserial r (or the median difference with a CI). Assumption: differences roughly symmetric around their median.

Exercise C. Chi-square test of independence on the 2 x 3 table (kiosk use by age group). Effect size: Cramér's V (and the kiosk-use proportion per age group). Assumption: expected counts of at least 5 in every cell (with 250 visitors this is likely unless one age group is very small or kiosk use is very rare), and each visitor counted once. If expected counts are small, use Fisher's exact test.

Scenarios 1 to 30.

Defensible alternatives that earn full method credit: scenario 1, a bootstrap CI for the mean difference; scenario 5, Kendall's τ; scenario 10, a negative binomial or log-linear model for days late if the data are counts of days and covariates matter; scenario 11, a linear mixed model with a random intercept per student; scenario 13, regression with HC3 standard errors or a log of rent; scenario 18, the sign test; scenario 19, a Welch t-test on the 15 tutorial-group means (the group is the unit of analysis); scenario 21, multiple regression with group and pre-test (the same model); scenario 27, a mixed logistic model; scenario 28, the exact distribution-free CI for the median based on order statistics.

Self-test.

1. Mixing forces you to identify the problem type before choosing a method, which is the skill exams and real analyses need; blocked practice lets you skip that step because the chapter tells you the method.
2. Yes, both are correct: ANCOVA is a linear regression with a group factor and a numeric covariate. Report the adjusted mean difference between methods with its CI (and partial η²), and the check for equal slopes.
3. "Thirty students ... on each of three quizzes": the same thirty students are scored three times, so the observations are not independent; use Cochran's Q or a mixed logistic model.
4. They missed the variant: the scenario says the variance is many times the mean (overdispersion), so negative binomial regression is needed (1 of 2 method points), and the key assumption is overdispersion, not independence (0 of 1). Effect size (rate ratio) earns 1.
5. Retrieval after a delay is harder, and that effort strengthens memory for longer; an immediate repeat mostly tests short-term memory of the answers you just read.

Revision history