# Regression Model Report: [Model name and study question]

> A working document for one regression model, from the question it answers
> to the sign-off: purpose and estimand, data and sample, variable
> definitions, model specification with its justification, an assumption and
> diagnostics checklist that records the evidence, the results table, model
> comparison, validation, sensitivity analyses, limitations, reproducibility
> details and review. It is for thesis writers who must defend a model to a
> supervisor or examiner, students submitting a regression project, and
> analysts handing a model to a reviewer or a client. It speeds up learning
> because every section asks a question you must answer in writing, and
> because the filled example shows what a complete answer looks like. The
> worked examples use fictional data; replace them with your own.
> Based on standard regression practice, the distinction between explanatory
> and predictive modelling (Shmueli 2010), TRIPOD reporting items for
> prediction models (named only) and APA JARS-Quant (named only).

<!--
Guidance: How to use this template
- Fill sections 1 to 4 before you look at any results. Writing the purpose,
  the estimand and the planned model first is what separates confirmatory
  from exploratory analysis.
- Keep the filled example rows marked *Example:* until you have written your
  own; then delete them.
- In the diagnostics checklist, write the evidence (statistic, plot, value),
  never only "OK".
- Fill the validation section only if the purpose includes prediction. For a
  purely explanatory model, say so and explain why.
- Use the self-check questions at the end to test whether the report would
  survive a critical reader.
- Related templates: academic/stats-linear-regression,
  academic/stats-logistic-regression, academic/stats-count-regression,
  academic/stats-analysis-plan, academic/stats-assumption-checklist,
  academic/stats-results-reporting, academic/stats-effect-sizes,
  academic/stats-analysis-report, academic/pre-analysis-plan,
  science/reproducibility-checklist, academic/replication-package-readme.
-->

## Document control

<!-- Guidance: Identify the model and its status. A reviewer must be able to see which data version and which code version every number in the report came from. -->

| Field | Value |
|---|---|
| Course or project | [*Example:* STAT 210 regression project / thesis chapter 4] |
| Learner or analyst | [Your name] |
| Model ID | [*Example:* LKM-EXAM-M2] |
| Data set | [*Example:* Lakemoor University exam study, n = 20, fictional, file `exam_v1.csv`] |
| Software and versions | [*Example:* R 4.5.1 with car 3.1.5; Python 3.14 with numpy 2.5.4, pandas 3.1.0, statsmodels 0.15.0] |
| Analysis plan reference | [*Example:* Analysis plan v1.0, dated YYYY-MM-DD] |
| Status | [Draft / Under review / Approved] |
| Date | YYYY-MM-DD |
| Version | [0.1] |

## 1. Purpose and estimand

<!-- Guidance: State in one sentence what the model is for: to describe, to explain (estimate an adjusted association or causal effect) or to predict. Then state the estimand: the exact quantity you want, in words and units. The rest of the report follows from this choice. -->

The purpose decides what "a good model" means. An **explanatory** model is
judged by whether its key coefficient is estimated without bias and with an
honest interval, so confounders must be adjusted for and the specification is
fixed in advance. A **predictive** model is judged by its accuracy on new
data, so validation matters more than the meaning of any one coefficient.
Mixing the two aims (choosing predictors by p-value, then interpreting the
coefficients as effects) is the most common weakness in student reports.

| Item | Entry |
|---|---|
| Purpose (describe / explain / predict) | [*Example:* Explain, with a secondary prediction check] |
| Research question | [*Example:* How is weekly study time associated with exam score in introductory statistics at Lakemoor University?] |
| Primary estimand | [*Example:* The difference in mean exam score (points) per additional weekly study hour, holding nightly sleep and course section constant] |
| Target population | [*Example:* Students taking the introductory statistics course in the current academic year] |
| Causal or associational claim | [*Example:* Associational only: study time was not randomised] |
| Secondary questions | [*Example:* Section difference (online vs in-person); does the hours slope differ by section? (exploratory)] |

## 2. Data and sample

<!-- Guidance: Describe where the data came from, how units were selected and how many were lost at each step. A flow from "eligible" to "analysed" lets a reader judge selection bias. -->

| Item | Entry |
|---|---|
| Source | [*Example:* Course records joined to an end-of-term survey (fictional)] |
| Sampling | [*Example:* Convenience sample of students who completed the survey] |
| Unit of analysis | [*Example:* One student] |
| Period | [*Example:* Autumn term, YYYY] |
| Eligible / consented / analysed | [*Example:* 26 / 22 / 20] |
| Exclusions and reasons | [*Example:* 2 with missing sleep data (complete-case analysis)] |
| Missing data handling | [*Example:* Complete cases; 2 of 22 (9%) excluded; sensitivity with multiple imputation listed as future work] |
| Ethics or data permission | [*Example:* Course data use approved by the department (fictional)] |

The flowchart shows how the analysed sample was reached; adapt the counts.

```mermaid
flowchart TD
  A["Enrolled students: 26"] --> B["Completed the survey: 22"]
  B --> C["Missing sleep data: 2 excluded"]
  C --> D["Analysed sample: 20"]
  D --> E["In-person: 10"]
  D --> F["Online: 10"]
```

## 3. Variable definitions

<!-- Guidance: One row per variable in the model. Give the unit, the coding, the reference level and how the variable was measured. If you transformed or centred a variable, say so here, not only in the code. -->

| Variable | Role | Definition and unit | Coding or transformation | Source |
|---|---|---|---|---|
| *Example:* `score` | Outcome | Final exam score, points 0 to 100 | None | Course records |
| *Example:* `hours` | Exposure of interest | Mean weekly study hours over the term | None; mean 8.05 | Survey |
| *Example:* `sleep` | Confounder | Mean nightly sleep, hours | None | Survey |
| *Example:* `section` | Confounder | Course section taken | Reference: `in_person`; dummy `online` | Course records |
| [Variable] | [Role] | [Definition] | [Coding] | [Source] |

**Descriptives of the analysed sample** (fill from your data): score mean
75.7 (SD 11.46), hours mean 8.05, 10 in-person and 10 online students.

## 4. Model specification and justification

<!-- Guidance: Write the model as an equation and explain every choice: the model family, each predictor, any interaction or transformation. Say which choices were fixed before seeing the data and which were made afterwards. -->

**Primary model (M2, specified before analysis).**

$$
\text{score}_i = \beta_0 + \beta_1\,\text{hours}_i + \beta_2\,\text{sleep}_i + \beta_3\,\text{online}_i + \varepsilon_i, \qquad \varepsilon_i \sim N(0, \sigma^2)
$$

| Decision | Choice | Justification |
|---|---|---|
| Model family | *Example:* OLS linear regression | Continuous outcome with a bounded but not crowded range (55 to 94) |
| Predictor of interest | *Example:* `hours` | The research question |
| Adjustment set | *Example:* `sleep`, `section` | Both plausibly affect study time and score (confounders); chosen from prior knowledge, not from p-values |
| Not adjusted for | *Example:* Coursework mark | Likely a mediator (study time raises coursework, which raises exam score); adjusting would remove part of the effect of interest |
| Functional form | *Example:* Linear in hours | Scatterplot roughly linear; checked in diagnostics |
| Interaction | *Example:* `hours x section` only as exploratory model M3 | Not pre-specified; too few students to test it with good power |
| Standard errors | *Example:* Classical, with HC3 as a sensitivity analysis | n = 20; HC3 guards against unequal variance |
| Comparison models | *Example:* M1 = hours only; M3 = M2 plus `hours x section` | To show how adjustment and the interaction change the estimate |

**Sample size check.** With p = 4 coefficients and n = 20, there are 5
observations per coefficient, below the common guide of 10 to 20. The
primary model is kept small for this reason, and this is listed as a
limitation.

## 5. Assumption and diagnostics checklist

<!-- Guidance: For each check, write the evidence you saw and the action you took. A reviewer should be able to verify each row from a figure or a number in the appendix. -->

| Check | Method | Evidence | Verdict | Action |
|---|---|---|---|---|
| Linearity | Residual vs fitted plot; component-plus-residual plots | *Example:* No curve; adding `hours^2` not needed | Met | None |
| Independence | Design review | *Example:* One row per student; no clustering beyond section, which is in the model | Met | None |
| Normality of errors | Q-Q plot; Shapiro-Wilk as a weak check | *Example:* Points close to the line; W = 0.961, *p* = .559 | Met | None |
| Equal variance | Scale-location plot; score test | *Example:* Flat; χ²(1) = 0.45, *p* = .504 | Met | HC3 SEs as sensitivity anyway |
| Influence | Cook's distance, leverage | *Example:* Student 10: Cook's *D* = 0.50 (> 4/n = 0.20, < 1); all others < 0.20 | Flag | Sensitivity analysis without student 10 |
| Collinearity | VIF | *Example:* 1.11 to 1.29 | Met | None |
| Outcome range | Fitted values within 0 to 100 | *Example:* Fitted 58.0 to 97.2 | Met | None |
| [Check] | [Method] | [Evidence] | [Verdict] | [Action] |

## 6. Results

<!-- Guidance: Report every coefficient of the primary model with its 95% CI and p-value, and the model-level statistics. Put the effect size for the estimand in a sentence in plain language under the table. -->

**Primary model M2** (OLS, n = 20). *F*(3, 16) = 104.04, *p* < .001,
*R*² = .951, adjusted *R*² = .942, residual SE = 2.76 points.

| Term | *b* | SE | 95% CI | *t*(16) | *p* | Partial η² |
|---|---|---|---|---|---|---|
| *Example:* Intercept | 39.94 | 4.60 | [30.20, 49.69] | 8.69 | < .001 | |
| *Example:* Study hours (per hour) | 2.89 | 0.20 | [2.48, 3.31] | 14.64 | < .001 | .931 |
| *Example:* Sleep (per hour) | 2.39 | 0.74 | [0.82, 3.97] | 3.22 | .005 | .393 |
| *Example:* Online (vs in-person) | -7.41 | 1.35 | [-10.26, -4.55] | -5.50 | < .001 | .654 |
| [Term] | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] |

**Estimand in plain language.** *Example:* Holding sleep and section
constant, each additional weekly study hour is associated with an exam score
2.89 points higher (95% CI 2.48 to 3.31). Over a realistic change of 5 hours
a week that is about 14.5 points, more than one standard deviation of the
scores (11.46).

The coefficient plot shows the three slopes with their 95% CIs; an interval
that does not cross the zero line is significant at α = 0.05.

```vega-lite
{
  "$schema": "https://vega.github.io/schema/vega-lite/v5.json",
  "description": "Coefficient plot for the primary model with 95% confidence intervals",
  "width": 360,
  "height": 150,
  "data": {"values": [
    {"term": "Study hours (per hour)", "b": 2.89, "lo": 2.48, "hi": 3.31},
    {"term": "Sleep (per hour)", "b": 2.39, "lo": 0.82, "hi": 3.97},
    {"term": "Online vs in-person", "b": -7.41, "lo": -10.26, "hi": -4.55}
  ]},
  "layer": [
    {
      "mark": {"type": "rule", "strokeWidth": 2},
      "encoding": {
        "y": {"field": "term", "type": "nominal", "title": null, "sort": null},
        "x": {"field": "lo", "type": "quantitative", "title": "Coefficient (exam points), 95% CI"},
        "x2": {"field": "hi"}
      }
    },
    {
      "mark": {"type": "point", "filled": true, "size": 80},
      "encoding": {
        "y": {"field": "term", "type": "nominal", "sort": null},
        "x": {"field": "b", "type": "quantitative"}
      }
    },
    {
      "data": {"values": [{"zero": 0}]},
      "mark": {"type": "rule", "strokeDash": [4, 4], "color": "#888888"},
      "encoding": {"x": {"field": "zero", "type": "quantitative"}}
    }
  ]
}
```

## 7. Model comparison

<!-- Guidance: Compare models that answer the same question on the same rows. Use nested F or likelihood-ratio tests for nested models and AIC or BIC for any set. Keep the pre-specified model as primary unless there is a strong, stated reason to change. -->

| Model | Terms | Adj. *R*² | AIC (R) | BIC (R) | LOOCV RMSE | Nested test vs previous |
|---|---|---|---|---|---|---|
| *Example:* M1 | hours | .847 | 120.62 | 123.61 | 4.670 | |
| *Example:* M2 (primary) | hours, sleep, section | .942 | 102.86 | 107.84 | 3.144 | *F*(2, 15) = 17.21, *p* < .001 |
| *Example:* M3 (exploratory) | M2 + hours x section | .947 | 101.79 | 107.77 | 3.036 | *F*(1, 15) = 2.49, *p* = .136 |
| [Model] | [Terms] | [ ] | [ ] | [ ] | [ ] | [ ] |

**Reading the comparison.** *Example:* Adding sleep and section (M1 to M2)
clearly improves the fit and reduces the hours slope from 3.14 to 2.89,
showing why adjustment matters for the estimand. The interaction (M3) lowers
AIC by about 1 and BIC by less than 0.1, and its F test is not significant;
by a common rule of thumb, AIC differences below about 2 do not separate
two models. M2 stays primary,
because it was pre-specified and M3 does not change the answer to the
research question.

**Why *F*(2, 15) and not *F*(2, 16)?** When `anova()` (or `anova_lm`)
compares three or more nested models in one call, every *F* uses the
residual mean square of the largest model (M3, 15 df) as its denominator:
(239.33 / 2) / (104.33 / 15) = 17.21. Comparing M1 and M2 on their own,
`anova(m1, m2)`, uses M2's residual df instead and gives *F*(2, 16) = 15.74.
Report the df that match the test you ran.

**Software note.** For the same models statsmodels reports AIC values
exactly 2 lower (118.62, 100.86, 99.79), because R counts the residual
variance as an estimated parameter. Differences between models are the same
in both programs; never mix AIC values from the two.

## 8. Validation

<!-- Guidance: For a prediction aim, estimate out-of-sample error with cross-validation or a holdout set and compare it with a simple baseline. For a purely explanatory aim, state that predictive validation is not the goal, but an internal check like this still shows whether the model overfits. -->

| Item | Entry |
|---|---|
| Is prediction an aim? | [*Example:* Secondary: to check overfitting, not to deploy] |
| Method | [*Example:* Leave-one-out CV (exact formula for OLS) and 5-fold CV with seed 2026] |
| Baseline | [*Example:* M1 (hours only); predicting the sample mean for everyone gives an in-sample RMSE of 11.17] |
| Result | [*Example:* LOOCV RMSE: M1 4.67, M2 3.14, M3 3.04. 5-fold RMSE: M1 4.67, M2 3.03, M3 2.81] |
| Calibration (for prediction models) | [*Example:* Not applicable for this aim] |
| External validation | [*Example:* None; next cohort planned] |

**Interpretation.** *Example:* The out-of-sample error of M2 (about 3.1
points) is close to its in-sample residual SE (2.76), so M2 is not badly
overfitted. M3 predicts slightly better in both CV schemes, which is
consistent with a small interaction that these 20 students cannot confirm.
With n = 20, CV estimates are themselves noisy: rerunning the 5-fold CV
with 40 other seeds gave an M2 RMSE between 2.85 and 3.63, and M3 beat M2
in only about two splits out of three. Treat the ranking of M2 and M3 as
undecided.

## 9. Sensitivity analyses

<!-- Guidance: List each analysis that tests whether the main conclusion depends on a debatable choice. Report the estimand under each one, and say whether the conclusion changes. -->

| Analysis | Why | Hours slope (95% CI) | Other change | Conclusion changes? |
|---|---|---|---|---|
| *Example:* Primary M2 | Reference | 2.89 [2.48, 3.31] | | |
| *Example:* HC3 robust SEs | Unequal variance | 2.89, SE 0.22, *p* < .001 | Sleep *p* = .027 (was .005) | No |
| *Example:* Without student 10 (Cook's *D* = 0.50) | Influence | 2.96 [2.62, 3.30] | Sleep slope 1.59 [0.21, 2.97] (was 2.39) | No for hours; sleep estimate less certain |
| *Example:* With interaction (M3), hours slope by section | Specification | 2.64 (in-person); 3.22 (online) | Interaction *p* = .136 | No |
| [Analysis] | [Why] | [ ] | [ ] | [ ] |

## 10. Limitations

<!-- Guidance: Name each limitation, its likely direction of bias if you can, and what would fix it. Generic phrases such as "small sample" are not enough without the consequence. -->

| Limitation | Consequence | Mitigation or next step |
|---|---|---|
| *Example:* Observational design | Hours slope may include effects of motivation or prior ability (likely upward bias) | Add a prior-achievement measure; describe as association |
| *Example:* n = 20, 5 observations per coefficient | Wide CIs for sleep and section; interaction not testable | Larger cohort; pre-specify interaction tests |
| *Example:* Self-reported study hours and sleep | Measurement error tends to bias slopes towards zero | Use learning-platform logs if available |
| *Example:* One influential student | Sleep estimate depends on one row | Reported sensitivity analysis |
| *Example:* Complete-case analysis (2 excluded) | Bias if missingness relates to the outcome | Multiple imputation as sensitivity |
| [Limitation] | [Consequence] | [Mitigation] |

## 11. Reproducibility

<!-- Guidance: Give everything needed to rerun the analysis: data version, code, software versions, random seeds. Paste the core code here or link to the repository and commit. -->

| Item | Entry |
|---|---|
| Data file and version | [*Example:* `exam_v1.csv`, checksum recorded in the repository] |
| Code location | [*Example:* `analysis/model_report.R` and `analysis/model_report.py`, commit `[hash]`] |
| Software | [*Example:* R 4.5.1, car 3.1.5; Python 3.14, numpy 2.5.4, pandas 3.1.0, statsmodels 0.15.0] |
| Random seed | [*Example:* 2026 (5-fold CV split)] |
| Run time and environment | [*Example:* Under 1 s on a laptop; no internet needed] |

**R code** (model comparison, LOOCV, partial η², sensitivity).

```r
# Model report: Lakemoor exam study. R 4.5.1, car 3.1.5
library(car)
d <- data.frame(
  hours   = c(2, 3, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, 10, 10, 11, 12, 12, 13, 14),
  sleep   = c(6, 7, 5, 8, 6, 7, 6, 8, 7, 5, 7, 6, 8, 7, 6, 8, 7, 6, 8, 7),
  section = factor(c("in_person", "online", "in_person", "online", "in_person",
                     "online", "online", "in_person", "online", "in_person",
                     "online", "in_person", "online", "in_person", "online",
                     "in_person", "online", "in_person", "online", "in_person"),
                   levels = c("in_person", "online")),
  score   = c(61, 55, 64, 66, 72, 65, 66, 81, 69, 69,
              75, 82, 75, 86, 77, 89, 83, 92, 93, 94))

m1 <- lm(score ~ hours, data = d)                      # baseline
m2 <- lm(score ~ hours + sleep + section, data = d)    # primary (pre-specified)
m3 <- lm(score ~ hours * section + sleep, data = d)    # exploratory interaction

anova(m1, m2, m3)                                      # nested F tests
data.frame(model = c("M1", "M2", "M3"),
           AIC = round(sapply(list(m1, m2, m3), AIC), 2),
           BIC = round(sapply(list(m1, m2, m3), BIC), 2),
           adjR2 = round(sapply(list(m1, m2, m3), function(m) summary(m)$adj.r.squared), 3))

# Leave-one-out CV RMSE (exact shortcut for OLS: e_i / (1 - h_ii))
loocv <- function(m) sqrt(mean((resid(m) / (1 - hatvalues(m)))^2))
round(sapply(list(M1 = m1, M2 = m2, M3 = m3), loocv), 3)

# Effect sizes: partial eta-squared from type II sums of squares
a <- Anova(m2); ss <- a$`Sum Sq`
round(ss[1:3] / (ss[1:3] + ss[4]), 3)

# Sensitivity: drop the most influential student (row 10, Cook's D = 0.50)
m2b <- update(m2, data = d[-10, ])
round(cbind(full = coef(m2), drop10 = coef(m2b)), 3)
round(confint(m2b), 3)
```

**R output** (output abridged: headings, significance codes and blank
lines between results removed).

```text
Model 1: score ~ hours
Model 2: score ~ hours + sleep + section
Model 3: score ~ hours * section + sleep
  Res.Df    RSS Df Sum of Sq       F    Pr(>F)    
1     18 360.95                                   
2     16 121.62  2    239.33 17.2050 0.0001309 ***
3     15 104.33  1     17.29  2.4859 0.1357256    

  model    AIC    BIC adjR2
1    M1 120.62 123.61 0.847
2    M2 102.86 107.84 0.942
3    M3 101.79 107.77 0.947

   M1    M2    M3 
4.670 3.144 3.036 

[1] 0.931 0.393 0.654

                full drop10
(Intercept)   39.943 45.379
hours          2.894  2.955
sleep          2.395  1.592
sectiononline -7.408 -7.626
               2.5 % 97.5 %
(Intercept)   36.711 54.048
hours          2.615  3.295
sleep          0.211  2.973
sectiononline -9.931 -5.321
```

**Python code** (nested tests, AIC and BIC, seeded 5-fold CV).

```python
# Model report: Lakemoor exam study. Python 3.14, numpy 2.5.4, pandas 3.1.0, statsmodels 0.15.0
import numpy as np
import pandas as pd
import statsmodels.formula.api as smf
from statsmodels.stats.anova import anova_lm

d = pd.DataFrame({
    "hours": [2, 3, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, 10, 10, 11, 12, 12, 13, 14],
    "sleep": [6, 7, 5, 8, 6, 7, 6, 8, 7, 5, 7, 6, 8, 7, 6, 8, 7, 6, 8, 7],
    "section": ["in_person", "online", "in_person", "online", "in_person",
                "online", "online", "in_person", "online", "in_person",
                "online", "in_person", "online", "in_person", "online",
                "in_person", "online", "in_person", "online", "in_person"],
    "score": [61, 55, 64, 66, 72, 65, 66, 81, 69, 69,
              75, 82, 75, 86, 77, 89, 83, 92, 93, 94]})
formulas = {"M1": "score ~ hours",
            "M2": "score ~ hours + sleep + section",
            "M3": "score ~ hours * section + sleep"}
fits = {k: smf.ols(f, data=d).fit() for k, f in formulas.items()}
print(anova_lm(fits["M1"], fits["M2"], fits["M3"]).round(4))
for k, m in fits.items():
    print(k, "AIC", round(m.aic, 2), "BIC", round(m.bic, 2))

# 5-fold cross-validation with a fixed seed
rng = np.random.default_rng(2026)
folds = rng.permutation(np.arange(len(d)) % 5)
for k, f in formulas.items():
    err = []
    for j in range(5):
        train, test = d[folds != j], d[folds == j]
        err.extend(test["score"] - smf.ols(f, data=train).fit().predict(test))
    print(k, "5-fold CV RMSE:", round(float(np.sqrt(np.mean(np.square(err)))), 3))
```

**Python output.**

```text
   df_resid       ssr  df_diff   ss_diff        F  Pr(>F)
0      18.0  360.9523      0.0       NaN      NaN     NaN
1      16.0  121.6199      2.0  239.3324  17.2050  0.0001
2      15.0  104.3299      1.0   17.2900   2.4859  0.1357
M1 AIC 118.62 BIC 120.61
M2 AIC 100.86 BIC 104.84
M3 AIC 99.79 BIC 104.77
M1 5-fold CV RMSE: 4.671
M2 5-fold CV RMSE: 3.033
M3 5-fold CV RMSE: 2.814
```

The nested F tests agree exactly between R and Python. AIC and BIC differ by
a constant (2 for AIC, ln 20 = 3.00 for BIC) for the reason given in section 7.

## 12. Review and sign-off

<!-- Guidance: A second reader checks that every number in sections 6 to 9 can be traced to the code output, and that the conclusions match the purpose in section 1. Record open issues rather than leaving them in comments. -->

| Role | Name | Decision | Date | Comments |
|---|---|---|---|---|
| Analyst | [Name] | [Submitted] | YYYY-MM-DD | [ ] |
| Reviewer or supervisor | [Name] | [Approved / Changes requested] | YYYY-MM-DD | [*Example:* Add prior-achievement covariate in next cohort] |
| Open issues | [List] | | | |

**Reviewer checklist.**

- [ ] Purpose and estimand stated before results
- [ ] Every excluded row is accounted for
- [ ] Adjustment set justified by reasoning, not by p-values
- [ ] Every diagnostic row has evidence
- [ ] Primary model is the pre-specified one, or the change is explained
- [ ] Effect of interest reported with CI and in plain language
- [ ] Validation matches the purpose
- [ ] Sensitivity analyses reported, including any that weaken the result
- [ ] Limitations give consequences, not only labels
- [ ] Code reruns and reproduces the numbers

## Common mistakes in model reports

<!-- Guidance: Read this list before you send the report. Each item is a question a supervisor or examiner is likely to ask. -->

- **Selecting predictors by stepwise p-values and then interpreting the
  coefficients as effects.** The CIs and p-values after selection are too
  narrow and too small. For explanation, choose the adjustment set from
  subject knowledge; for prediction, validate the whole selection process.
- **Adjusting for a mediator or a collider.** Controlling for coursework
  marks would remove part of the study-hours effect you want to estimate.
- **Reporting only p-values.** The estimand needs a size, a unit and an
  interval.
- **Using AIC to choose a model and then reporting its p-values as
  confirmatory.** Say which model was pre-specified and which was chosen
  after looking at the data.
- **Calling in-sample R² "accuracy".** Prediction claims need out-of-sample
  error, and a baseline to compare with.
- **Hiding a sensitivity analysis that weakened the result.** The sleep
  estimate dropping by a third without one student is information a reader
  needs.
- **Listing limitations without consequences.** "The sample is small"
  should say what it does: wide intervals, an untestable interaction.

## Self-check questions

<!-- Guidance: Answer these about your own report, then compare your reasoning with the answers for the example. -->

1. In the example, why is M2 kept as the primary model even though M3 has a
   lower AIC and a lower cross-validated RMSE?
2. Why is the coursework mark not in the adjustment set, even though it
   would increase R²?
3. The 5-fold CV RMSE of M2 is 3.03 and the LOOCV RMSE is 3.14. Is one of
   them wrong?
4. Removing student 10 changes the sleep coefficient from 2.39 to 1.59.
   Should student 10 be deleted from the primary analysis?
5. Which sections of this report would change most if the purpose were
   prediction for next year's students instead of explanation?

## Sources and further reading

<!-- Guidance: Cite the method sources and reporting guidance you followed, and add your course or field's own reporting standard. -->

Structure aligned with common reporting guidance for regression analyses:
APA JARS-Quant (named only), the TRIPOD statement for prediction models
and its 2024 update TRIPOD+AI (named only), and the explain-versus-predict
distinction.

- Shmueli, G. (2010). To explain or to predict? *Statistical Science*,
  25(3), 289-310.
- Collins, G. S., Moons, K. G. M., Dhiman, P., and colleagues (2024).
  TRIPOD+AI statement: updated guidance for reporting clinical prediction
  models that use regression or machine learning methods. *BMJ*, 385,
  e078378 (supersedes the 2015 TRIPOD checklist).
- Harrell, F. E. (2015). *Regression Modeling Strategies* (2nd ed.).
  Springer.
- Collins, G. S., Reitsma, J. B., Altman, D. G., and Moons, K. G. M. (2015).
  Transparent reporting of a multivariable prediction model for individual
  prognosis or diagnosis (TRIPOD): the TRIPOD statement. *BMJ*, 350, g7594.
- Akaike, H. (1974). A new look at the statistical model identification.
  *IEEE Transactions on Automatic Control*, 19(6), 716-723.
- Schwarz, G. (1978). Estimating the dimension of a model. *Annals of
  Statistics*, 6(2), 461-464.
- Cook, R. D. (1977). Detection of influential observation in linear
  regression. *Technometrics*, 19(1), 15-18.
- Long, J. S., and Ervin, L. H. (2000). Using heteroscedasticity consistent
  standard errors in the linear regression model. *The American
  Statistician*, 54(3), 217-224.
- Wasserstein, R. L., and Lazar, N. A. (2016). The ASA statement on
  p-values: context, process, and purpose. *The American Statistician*,
  70(2), 129-133.
- Further reading by name: de Smith, M. J., *Statistical Analysis Handbook*
  (StatsRef, 2024 edition); NIST/SEMATECH *e-Handbook of Statistical
  Methods*; APA *Publication Manual* (7th ed.).

## Answer key

<!-- Guidance: Use these answers to calibrate your own reasoning, not as text to copy. -->

1. M2 was pre-specified to answer the research question; M3 adds a term
   that was not planned and whose test is not significant (*p* = .136). The
   AIC difference (about 1) is below the usual threshold of 2, and the CV
   difference is within the noise of a 20-row data set. Choosing the primary
   model after seeing results would make its inference over-optimistic. M3
   is reported as exploratory.
2. Coursework is likely on the causal path from study time to exam score (a
   mediator). Adjusting for it would estimate only the part of the
   study-hours association that does not pass through coursework, which is
   not the estimand. A higher R² does not make the estimand more correct.
3. No. They are two estimates of the same out-of-sample error from
   different resampling schemes; both are noisy at n = 20. LOOCV has low bias
   but high variance; 5-fold CV depends on the random split (seed 2026).
4. No, unless the row is a data error. It is a real student with an unusual
   combination (little sleep, average study, lower score). Keep the student,
   report the sensitivity analysis, and soften the claim about sleep.
5. Sections 1 (purpose and estimand: predicted score, not a slope), 4
   (predictors chosen for predictive value, possibly including the
   interaction), 8 (validation becomes central, with calibration and ideally
   an external cohort) and 6 (accuracy metrics instead of coefficient
   interpretation). Section 10 would focus on transportability to new
   cohorts.

## Revision history

<!-- Guidance: Add a row for every change to data, model or conclusions, and link reviewer comments to the version that answered them. -->

| Version | Date | Author | Change |
|---|---|---|---|
| 0.1 | YYYY-MM-DD | [Name] | First draft from the template |
| [0.2] | YYYY-MM-DD | [Name] | [Added sensitivity analysis without student 10 after review] |
