Statistics Error Log Template (Mistake Journal and 25 Misconceptions)

Statistics mistake journal – log table, six-type error taxonomy, 25 classic misconceptions with the correct idea and fixing template, fix-it drills, weekly review routine

AkademischStandardGreenland et al. (2016) misinterpretation guide; ASA p-value statement (Wasserstein and Lazar 2016); Roediger and Karpicke (2006); Cepeda et al. (2006)stats error logAktualisiert 25.10.2026

Im Editor öffnen.md herunterladen

Anleitung: Log each mistake within a day with your exact wrong answer, classify it with the six-type taxonomy, write a one-sentence fix rule and set re-test dates. Review weekly: close errors only after two correct re-tests on new problems, and turn every fix rule into a flashcard. The main pitfall is correcting a mistake without naming its root cause, so the same misconception returns.

Vorschau

Statistics Error Log: [Your course, exam or thesis]

A mistake journal for learning statistics: a log table for every error you make in homework, practice exams or analysis, a six-type error taxonomy to classify each one, a catalogue of 25 classic statistics misconceptions with the correct idea and the template that fixes it, fix-it drills, and a weekly review routine. It is for students, thesis writers and analysts who want each mistake to happen only once. It speeds up learning because errors you classify, explain and re-test on a schedule turn into rules you remember, while errors you only correct tend to come back. The worked examples use fictional data; replace them with your own. Source basis: retrieval and spacing research (Roediger and Karpicke 2006; Cepeda et al. 2006) and the misinterpretation catalogue of Greenland et al. (2016), written in our own words.

Document control

Field	Value
Course or project	[STAT 201, Lakemoor University (fictional)]
Learner or analyst	[Your name]
Period covered	YYYY-MM-DD to YYYY-MM-DD
Software and versions	[R 4.5.1 / Python 3.14 with scipy 1.18.1, statsmodels 0.15.0]
Date	YYYY-MM-DD
Version	[0.1]

Why a mistake journal works

Every learner makes a small number of mistakes again and again. A mistake that you correct and forget will usually return, because the wrong idea is still the first thing memory offers. Writing the mistake down, naming its type and its root cause, and turning the fix into a short rule that you later test yourself on, replaces the wrong idea with the right one through retrieval practice and spacing.

The log also shows patterns. If most of your errors are "reading the question", more statistics study will not help: a slower first read will. If most are "concept" errors about p-values and intervals, the concept cards (academic/stats-hypothesis-testing, academic/stats-confidence-intervals) are the best use of your next week.

Error-type taxonomy

Type	What went wrong	Typical example	Usual fix
Reading the question	misread what was asked or given	used the SD when the question gave the SE	underline what is given and what is asked
Concept	wrong idea about what a quantity means	"p = .03 means a 3% chance $H_0$ is true"	re-learn the concept card, explain it in plain words
Procedure	wrong method or wrong steps for the design	analysed paired data with an independent-samples test	test selector, worked-problem steps 2 and 3
Calculation	right method, arithmetic or formula slip	divided by $n$ instead of $n-1$	hand check against software, estimate size first
Software	wrong function, option or default	scipy `ttest_ind` without `equal_var=False`	read the output heading, set options explicitly
Interpretation	right number, wrong conclusion	causal claim from an observational study	report template, Feynman check

Classify in this order, stopping at the first "no":

Error log

Date	Problem (source, short)	My answer	Correct answer	Error type	Root cause	Fix rule	Re-test date
Example: YYYY-MM-DD	PS3 Q2: 95% CI [4.1, 7.9] for mean sleep hours	"95% of students sleep 4.1 to 7.9 h"	the interval estimates the mean, not individual values	Concept	confused a CI with a reference range	"A CI is about a parameter, not about individual data values"	YYYY-MM-DD
Example: YYYY-MM-DD	Practice exam Q5: before/after scores, 12 students	Welch t, p = .21	paired t, p = .004	Procedure	did not check whether the same students appear twice	"Same units measured twice: compute differences first"	YYYY-MM-DD
Example: YYYY-MM-DD	Lab 4: two-group comparison in Python	pooled p = .031	Welch p = .048	Software	scipy default is pooled	"In scipy write `equal_var=False` every time"	YYYY-MM-DD
Example: YYYY-MM-DD	PS4 Q1: SE of mean, s = 12, n = 36	0.33	2	Calculation	divided by $n$ , not $\sqrt{n}$	"SE uses the square root of n"	YYYY-MM-DD
Example: YYYY-MM-DD	Thesis survey: tutoring use and grades, r = .41	"tutoring raises grades"	"tutoring use is associated with higher grades"	Interpretation	causal wording for observational data	"No randomisation, no causal verbs"	YYYY-MM-DD
Example: YYYY-MM-DD	Quiz 2 Q3: "at least 2 of 5"	P(X = 2)	P(X ≥ 2)	Reading the question	skipped "at least"	"Circle every inequality word before computing"	YYYY-MM-DD
YYYY-MM-DD	[Problem]	[My answer]	[Correct answer]	[Type]	[Root cause]	[Fix rule]	YYYY-MM-DD
YYYY-MM-DD	[Problem]	[My answer]	[Correct answer]	[Type]	[Root cause]	[Fix rule]	YYYY-MM-DD

Re-test record:

Error (date)	Re-test 1 (date, new problem, right/wrong)	Re-test 2 (date, right/wrong)	Closed
Example: paired vs independent (YYYY-MM-DD)	YYYY-MM-DD, branch visits before/after, right	YYYY-MM-DD, right	Y
[Error]	YYYY-MM-DD, [ ]	YYYY-MM-DD, [ ]	[ ]

Error profile

Counting errors by type tells you where your next study hour pays most. In the fictional month below, concept and procedure errors dominate: the learner should spend the next week on the concept cards and the test selector, not on more arithmetic practice.

Catalogue of 25 classic misconceptions

#	Misconception	Correct idea	Template that fixes it
1	The p-value is the probability that $H_0$ is true	it is the probability of data at least this extreme, assuming $H_0$ is true	academic/stats-hypothesis-testing
2	p > .05 shows there is no effect	absence of evidence is not evidence of absence; look at the CI, which may include large effects	academic/stats-confidence-intervals
3	A significant result is an important result	significance depends on n; importance depends on the effect size and context	academic/stats-effect-sizes
4	A computed 95% CI has a 95% probability of containing $\mu$	95% describes the long-run procedure; a Bayesian credible interval carries the probability statement	academic/stats-confidence-intervals
5	A 95% CI contains 95% of the data	it is a range for a parameter; individual values spread much wider	academic/stats-confidence-intervals
6	Overlapping CIs mean no significant difference	two 95% CIs can overlap while the difference is significant; build a CI for the difference	academic/stats-confidence-intervals
7	A bigger sample makes the SD smaller	the SD estimates the population spread and does not shrink; the SE shrinks with $\sqrt{n}$	academic/stats-sampling-distributions
8	The central limit theorem makes the data normal	it makes the sampling distribution of the mean approximately normal	academic/stats-sampling-distributions
9	$P(A \mid B) = P(B \mid A)$	they differ unless $P(A) = P(B)$ (or the events never occur together); use Bayes' theorem with the base rate	academic/stats-probability-basics
10	Correlation shows causation	confounding and reverse causation are possible without randomisation	academic/stats-correlation
11	r = 0 means no relationship	r measures only linear association; a curved relation can have r near 0	academic/stats-correlation
12	A regression slope in observational data is a causal effect	it is an adjusted association; causal claims need design or strong assumptions	academic/stats-linear-regression
13	A high $R^2$ means the model is correct	$R^2$ can be high with wrong functional form or violated assumptions; check residuals	academic/stats-regression-model-report
14	Regression requires the outcome to be normal	the assumption is about the residuals (errors), and mainly matters for small-sample inference	academic/stats-assumption-checklist
15	Always run a normality test and obey its p-value	small samples: low power; large samples: trivial departures flagged; use plots and design	academic/stats-assumption-checklist
16	Before-and-after data are two independent groups	the same units measured twice are paired; analyse the differences	academic/stats-t-tests
17	Many pairwise t-tests are fine instead of ANOVA	the family-wise Type I error grows; use ANOVA with Tukey or Games-Howell	academic/stats-one-way-anova
18	A significant ANOVA shows which groups differ	it shows at least one mean differs; post hoc tests show which	academic/stats-one-way-anova
19	Nonparametric tests have no assumptions and always compare medians	they still assume independent observations; Mann-Whitney tests whether one group tends to have larger values, which equals a median comparison only if the two shapes are the same	academic/stats-nonparametric-tests
20	Chi-square works on percentages	it needs counts; expected counts should mostly be at least 5	academic/stats-contingency-tables
21	An odds ratio is the same as a relative risk	they are close only when the outcome is rare	academic/stats-contingency-tables
22	A logistic coefficient is a change in probability	it is a change in log odds; $e^{b}$ is an odds ratio	academic/stats-logistic-regression
23	Observed power explains a non-significant result	power computed from the observed effect is a function of the p-value and adds nothing	academic/stats-power-sample-size
24	Deleting incomplete cases is always safe	in general it is guaranteed unbiased only under MCAR (with some exceptions, such as missingness that depends only on regression predictors) and it loses power; document and consider multiple imputation	academic/stats-data-cleaning-log
25	Correcting for multiple tests is only needed for the tests you report	every test you ran (or would have run) counts; pre-specify in an analysis plan	academic/stats-analysis-plan

Fix-it drills

Each drill takes one wrong statement, names its error type, explains why it is wrong, and rewrites it correctly.

Drill A (fully worked). Statement in a lab report: "The Welch t-test gave p = .27, so the two Brightwell Bakery (fictional) flour suppliers produce loaves of the same weight."

Error type: interpretation (the number is right, the conclusion is not).
Why it is wrong: a non-significant result means the data are compatible with no difference, but they may also be compatible with a sizeable difference. The size of plausible differences is in the CI, not in the p-value.
Check the CI: suppose the 95% CI for the difference is [-6, 20] grams. Differences up to 20 grams remain plausible.
Rewrite: "The mean weights did not differ significantly (p = .27); the 95% CI for the difference, [-6, 20] grams, is too wide to rule out a practically relevant difference."
Fix rule for flashcards: "Non-significant: report the CI and say what it cannot rule out."

Drill B (half worked). Statement: "Students who used the Lakemoor University (fictional) tutoring service had grades 0.4 points higher, r = .41, p < .001, so tutoring improves grades."

Error type: [ ].
Why it is wrong: the data are [ ], so students chose whether to use tutoring; [ ] such as motivation could explain both.
Rewrite: "Use of the tutoring service was [ ] with higher grades (r = .41, p < .001)."
Fix rule: [ ].

Drill C (unsolved). Statement in a thesis draft: "With n = 2,000 the Shapiro-Wilk test rejected normality (p = .002), so we could not use regression and switched to a Spearman correlation." Name the error type(s), explain what is wrong and rewrite the decision.

Mixed set (interleaved). Mark each statement right or wrong; for wrong ones, give the misconception number.

"The SE of a mean falls as the sample size grows."
"Our two 95% CIs overlap slightly, so the difference cannot be significant."
"An odds ratio of 2.0 means the outcome is twice as likely."
"Because $r = 0.05$ , study hours and grades are unrelated."
"The paired t-test is a one-sample t-test on the within-pair differences."
"The ANOVA was significant, F(2, 12) = 12.80, so all three sections differ."

Weekly review routine

Step	Time	What to do
1	5 min	Count new errors by type; update the error profile chart values
2	10 min	For each error due for re-test, solve a new problem of the same kind without notes; record right or wrong
3	5 min	Close errors with two correct re-tests at least a week apart; reschedule the rest (+3 days if wrong)
4	5 min	Read the catalogue entries that match this week's concept and interpretation errors
5	5 min	Turn every new fix rule into a flashcard; pick next week's focus template from the most frequent type

Where to find new problems for re-tests: the Exercise C sections of the method templates, the scenarios in academic/stats-practice-which-test, and past exam papers. Use a different data set from the original error, so that you test the rule rather than your memory of one answer.

Self-test questions

Name the six error types in the order you should check them.
You used scipy.stats.ttest_ind(a, b) and reported a pooled p-value when the course requires Welch. Which type is this, and what is the fix rule?
Why is a fix rule written as a short sentence, and where should it go next?
When is an error "closed" in this log?
State the correct idea for misconception 7 in one sentence.

My notes and next steps

Most frequent error type this month: [ ]. Focus template for next week: [academic/...]. Errors still open: [ ]. Confidence that I will not repeat my top error (1-5): [ ]. Questions for the tutor: [ ].

Sources and further reading

Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., and Altman, D. G. (2016). Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. European Journal of Epidemiology, 31(4), 337–350.
Wasserstein, R. L., and Lazar, N. A. (2016). The ASA statement on p-values: context, process, and purpose. The American Statistician, 70(2), 129–133.
Roediger, H. L., and Karpicke, J. D. (2006). Test-enhanced learning: taking memory tests improves long-term retention. Psychological Science, 17(3), 249–255.
Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., and Rohrer, D. (2006). Distributed practice in verbal recall tasks: a review and quantitative synthesis. Psychological Bulletin, 132(3), 354–380.
Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., and Willingham, D. T. (2013). Improving students' learning with effective learning techniques. Psychological Science in the Public Interest, 14(1), 4–58.
Further reading (by name only): de Smith, M. J. (2024). Statistical Analysis Handbook, 2024 edition. StatsRef, https://www.statsref.com/; OpenIntro Statistics; NIST/SEMATECH e-Handbook of Statistical Methods.

Answer key

Drill B. 1: interpretation. 2: observational; a confounder (for example motivation or prior grades). 3: "...was associated with higher grades..." 4: "No randomisation, no causal verbs: say associated with."

Drill C. Error types: concept (normality of the outcome is not the regression assumption; the assumption concerns residuals, misconception 14) and procedure (with n = 2,000, a normality test flags trivial departures, misconception 15; and switching from regression to a correlation changes the question). Rewrite: "We fitted the planned regression and checked the residuals with a Q-Q plot and a residuals-versus-fitted plot; with n = 2,000 the inference on coefficients is robust to mild non-normality. As a sensitivity analysis we report robust (HC3) standard errors."

Mixed set. 1 right. 2 wrong (misconception 6). 3 wrong (21: it doubles the odds, not the probability, unless the outcome is rare). 4 wrong (11: only the linear association is near zero; also check a scatter plot). 5 right. 6 wrong (18: at least one mean differs; use post hoc tests).

Self-test.

Reading the question, concept, procedure, calculation, software, interpretation (calculation and software are checked at the same step: by hand or by the tool).
Software. Fix rule: "In scipy write equal_var=False for two independent groups."
A short rule can be recalled in one step and tested as a flashcard; it goes into academic/stats-flashcards-spaced-review with a re-test date.
When you solve a new problem of the same kind correctly on two re-tests at least a week apart.
The SD estimates the spread of the population and does not shrink with n; the SE of the mean, $\sigma/\sqrt{n}$ (estimated by $s/\sqrt{n}$ ), does.

Revision history

Version	Date	Author	Change
0.1	YYYY-MM-DD	[Name]	First version from template
[0.2]	YYYY-MM-DD	[Name]	[Started term 2 log]