Regression Model Report Template (Statistics, R and Python)

Regression Model Report: [Model name and study question]

A working document for one regression model, from the question it answers to the sign-off: purpose and estimand, data and sample, variable definitions, model specification with its justification, an assumption and diagnostics checklist that records the evidence, the results table, model comparison, validation, sensitivity analyses, limitations, reproducibility details and review. It is for thesis writers who must defend a model to a supervisor or examiner, students submitting a regression project, and analysts handing a model to a reviewer or a client. It speeds up learning because every section asks a question you must answer in writing, and because the filled example shows what a complete answer looks like. The worked examples use fictional data; replace them with your own. Based on standard regression practice, the distinction between explanatory and predictive modelling (Shmueli 2010), TRIPOD reporting items for prediction models (named only) and APA JARS-Quant (named only).

Document control

1. Purpose and estimand

The purpose decides what "a good model" means. An explanatory model is judged by whether its key coefficient is estimated without bias and with an honest interval, so confounders must be adjusted for and the specification is fixed in advance. A predictive model is judged by its accuracy on new data, so validation matters more than the meaning of any one coefficient. Mixing the two aims (choosing predictors by p-value, then interpreting the coefficients as effects) is the most common weakness in student reports.

2. Data and sample

The flowchart shows how the analysed sample was reached; adapt the counts.

3. Variable definitions

Descriptives of the analysed sample (fill from your data): score mean 75.7 (SD 11.46), hours mean 8.05, 10 in-person and 10 online students.

4. Model specification and justification

Primary model (M2, specified before analysis).

Sample size check. With p = 4 coefficients and n = 20, there are 5 observations per coefficient, below the common guide of 10 to 20. The primary model is kept small for this reason, and this is listed as a limitation.

5. Assumption and diagnostics checklist

6. Results

Primary model M2 (OLS, n = 20). F(3, 16) = 104.04, p < .001, R² = .951, adjusted R² = .942, residual SE = 2.76 points.

Estimand in plain language. Example: Holding sleep and section constant, each additional weekly study hour is associated with an exam score 2.89 points higher (95% CI 2.48 to 3.31). Over a realistic change of 5 hours a week that is about 14.5 points, more than one standard deviation of the scores (11.46).

The coefficient plot shows the three slopes with their 95% CIs; an interval that does not cross the zero line is significant at α = 0.05.

7. Model comparison

Reading the comparison. Example: Adding sleep and section (M1 to M2) clearly improves the fit and reduces the hours slope from 3.14 to 2.89, showing why adjustment matters for the estimand. The interaction (M3) lowers AIC by about 1 and BIC by less than 0.1, and its F test is not significant; by a common rule of thumb, AIC differences below about 2 do not separate two models. M2 stays primary, because it was pre-specified and M3 does not change the answer to the research question.

Why F(2, 15) and not F(2, 16)? When anova() (or anova_lm) compares three or more nested models in one call, every F uses the residual mean square of the largest model (M3, 15 df) as its denominator: (239.33 / 2) / (104.33 / 15) = 17.21. Comparing M1 and M2 on their own, anova(m1, m2), uses M2's residual df instead and gives F(2, 16) = 15.74. Report the df that match the test you ran.

Software note. For the same models statsmodels reports AIC values exactly 2 lower (118.62, 100.86, 99.79), because R counts the residual variance as an estimated parameter. Differences between models are the same in both programs; never mix AIC values from the two.

8. Validation

Interpretation. Example: The out-of-sample error of M2 (about 3.1 points) is close to its in-sample residual SE (2.76), so M2 is not badly overfitted. M3 predicts slightly better in both CV schemes, which is consistent with a small interaction that these 20 students cannot confirm. With n = 20, CV estimates are themselves noisy: rerunning the 5-fold CV with 40 other seeds gave an M2 RMSE between 2.85 and 3.63, and M3 beat M2 in only about two splits out of three. Treat the ranking of M2 and M3 as undecided.

9. Sensitivity analyses

10. Limitations

11. Reproducibility

R code (model comparison, LOOCV, partial η², sensitivity).

# Model report: Lakemoor exam study. R 4.5.1, car 3.1.5
library(car)
d <- data.frame(
  hours   = c(2, 3, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, 10, 10, 11, 12, 12, 13, 14),
  sleep   = c(6, 7, 5, 8, 6, 7, 6, 8, 7, 5, 7, 6, 8, 7, 6, 8, 7, 6, 8, 7),
  section = factor(c("in_person", "online", "in_person", "online", "in_person",
                     "online", "online", "in_person", "online", "in_person",
                     "online", "in_person", "online", "in_person", "online",
                     "in_person", "online", "in_person", "online", "in_person"),
                   levels = c("in_person", "online")),
  score   = c(61, 55, 64, 66, 72, 65, 66, 81, 69, 69,
              75, 82, 75, 86, 77, 89, 83, 92, 93, 94))

m1 <- lm(score ~ hours, data = d)                      # baseline
m2 <- lm(score ~ hours + sleep + section, data = d)    # primary (pre-specified)
m3 <- lm(score ~ hours * section + sleep, data = d)    # exploratory interaction

anova(m1, m2, m3)                                      # nested F tests
data.frame(model = c("M1", "M2", "M3"),
           AIC = round(sapply(list(m1, m2, m3), AIC), 2),
           BIC = round(sapply(list(m1, m2, m3), BIC), 2),
           adjR2 = round(sapply(list(m1, m2, m3), function(m) summary(m)$adj.r.squared), 3))

# Leave-one-out CV RMSE (exact shortcut for OLS: e_i / (1 - h_ii))
loocv <- function(m) sqrt(mean((resid(m) / (1 - hatvalues(m)))^2))
round(sapply(list(M1 = m1, M2 = m2, M3 = m3), loocv), 3)

# Effect sizes: partial eta-squared from type II sums of squares
a <- Anova(m2); ss <- a$`Sum Sq`
round(ss[1:3] / (ss[1:3] + ss[4]), 3)

# Sensitivity: drop the most influential student (row 10, Cook's D = 0.50)
m2b <- update(m2, data = d[-10, ])
round(cbind(full = coef(m2), drop10 = coef(m2b)), 3)
round(confint(m2b), 3)

R output (output abridged: headings, significance codes and blank lines between results removed).

Model 1: score ~ hours
Model 2: score ~ hours + sleep + section
Model 3: score ~ hours * section + sleep
  Res.Df    RSS Df Sum of Sq       F    Pr(>F)    
1     18 360.95                                   
2     16 121.62  2    239.33 17.2050 0.0001309 ***
3     15 104.33  1     17.29  2.4859 0.1357256    

  model    AIC    BIC adjR2
1    M1 120.62 123.61 0.847
2    M2 102.86 107.84 0.942
3    M3 101.79 107.77 0.947

   M1    M2    M3 
4.670 3.144 3.036 

[1] 0.931 0.393 0.654

                full drop10
(Intercept)   39.943 45.379
hours          2.894  2.955
sleep          2.395  1.592
sectiononline -7.408 -7.626
               2.5 % 97.5 %
(Intercept)   36.711 54.048
hours          2.615  3.295
sleep          0.211  2.973
sectiononline -9.931 -5.321

Python code (nested tests, AIC and BIC, seeded 5-fold CV).

# Model report: Lakemoor exam study. Python 3.14, numpy 2.5.4, pandas 3.1.0, statsmodels 0.15.0
import numpy as np
import pandas as pd
import statsmodels.formula.api as smf
from statsmodels.stats.anova import anova_lm

d = pd.DataFrame({
    "hours": [2, 3, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, 10, 10, 11, 12, 12, 13, 14],
    "sleep": [6, 7, 5, 8, 6, 7, 6, 8, 7, 5, 7, 6, 8, 7, 6, 8, 7, 6, 8, 7],
    "section": ["in_person", "online", "in_person", "online", "in_person",
                "online", "online", "in_person", "online", "in_person",
                "online", "in_person", "online", "in_person", "online",
                "in_person", "online", "in_person", "online", "in_person"],
    "score": [61, 55, 64, 66, 72, 65, 66, 81, 69, 69,
              75, 82, 75, 86, 77, 89, 83, 92, 93, 94]})
formulas = {"M1": "score ~ hours",
            "M2": "score ~ hours + sleep + section",
            "M3": "score ~ hours * section + sleep"}
fits = {k: smf.ols(f, data=d).fit() for k, f in formulas.items()}
print(anova_lm(fits["M1"], fits["M2"], fits["M3"]).round(4))
for k, m in fits.items():
    print(k, "AIC", round(m.aic, 2), "BIC", round(m.bic, 2))

# 5-fold cross-validation with a fixed seed
rng = np.random.default_rng(2026)
folds = rng.permutation(np.arange(len(d)) % 5)
for k, f in formulas.items():
    err = []
    for j in range(5):
        train, test = d[folds != j], d[folds == j]
        err.extend(test["score"] - smf.ols(f, data=train).fit().predict(test))
    print(k, "5-fold CV RMSE:", round(float(np.sqrt(np.mean(np.square(err)))), 3))

Python output.

   df_resid       ssr  df_diff   ss_diff        F  Pr(>F)
0      18.0  360.9523      0.0       NaN      NaN     NaN
1      16.0  121.6199      2.0  239.3324  17.2050  0.0001
2      15.0  104.3299      1.0   17.2900   2.4859  0.1357
M1 AIC 118.62 BIC 120.61
M2 AIC 100.86 BIC 104.84
M3 AIC 99.79 BIC 104.77
M1 5-fold CV RMSE: 4.671
M2 5-fold CV RMSE: 3.033
M3 5-fold CV RMSE: 2.814

The nested F tests agree exactly between R and Python. AIC and BIC differ by a constant (2 for AIC, ln 20 = 3.00 for BIC) for the reason given in section 7.

12. Review and sign-off

Reviewer checklist.

• Purpose and estimand stated before results
• Every excluded row is accounted for
• Adjustment set justified by reasoning, not by p-values
• Every diagnostic row has evidence
• Primary model is the pre-specified one, or the change is explained
• Effect of interest reported with CI and in plain language
• Validation matches the purpose
• Sensitivity analyses reported, including any that weaken the result
• Limitations give consequences, not only labels
• Code reruns and reproduces the numbers

Common mistakes in model reports

• Selecting predictors by stepwise p-values and then interpreting the coefficients as effects. The CIs and p-values after selection are too narrow and too small. For explanation, choose the adjustment set from subject knowledge; for prediction, validate the whole selection process.
• Adjusting for a mediator or a collider. Controlling for coursework marks would remove part of the study-hours effect you want to estimate.
• Reporting only p-values. The estimand needs a size, a unit and an interval.
• Using AIC to choose a model and then reporting its p-values as confirmatory. Say which model was pre-specified and which was chosen after looking at the data.
• Calling in-sample R² "accuracy". Prediction claims need out-of-sample error, and a baseline to compare with.
• Hiding a sensitivity analysis that weakened the result. The sleep estimate dropping by a third without one student is information a reader needs.
• Listing limitations without consequences. "The sample is small" should say what it does: wide intervals, an untestable interaction.

Self-check questions

1. In the example, why is M2 kept as the primary model even though M3 has a lower AIC and a lower cross-validated RMSE?
2. Why is the coursework mark not in the adjustment set, even though it would increase R²?
3. The 5-fold CV RMSE of M2 is 3.03 and the LOOCV RMSE is 3.14. Is one of them wrong?
4. Removing student 10 changes the sleep coefficient from 2.39 to 1.59. Should student 10 be deleted from the primary analysis?
5. Which sections of this report would change most if the purpose were prediction for next year's students instead of explanation?

Sources and further reading

Structure aligned with common reporting guidance for regression analyses: APA JARS-Quant (named only), the TRIPOD statement for prediction models and its 2024 update TRIPOD+AI (named only), and the explain-versus-predict distinction.

• Shmueli, G. (2010). To explain or to predict? Statistical Science, 25(3), 289-310.
• Collins, G. S., Moons, K. G. M., Dhiman, P., and colleagues (2024). TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ, 385, e078378 (supersedes the 2015 TRIPOD checklist).
• Harrell, F. E. (2015). Regression Modeling Strategies (2nd ed.). Springer.
• Collins, G. S., Reitsma, J. B., Altman, D. G., and Moons, K. G. M. (2015). Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ, 350, g7594.
• Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716-723.
• Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6(2), 461-464.
• Cook, R. D. (1977). Detection of influential observation in linear regression. Technometrics, 19(1), 15-18.
• Long, J. S., and Ervin, L. H. (2000). Using heteroscedasticity consistent standard errors in the linear regression model. The American Statistician, 54(3), 217-224.
• Wasserstein, R. L., and Lazar, N. A. (2016). The ASA statement on p-values: context, process, and purpose. The American Statistician, 70(2), 129-133.
• Further reading by name: de Smith, M. J., Statistical Analysis Handbook (StatsRef, 2024 edition); NIST/SEMATECH e-Handbook of Statistical Methods; APA Publication Manual (7th ed.).

Answer key

1. M2 was pre-specified to answer the research question; M3 adds a term that was not planned and whose test is not significant (p = .136). The AIC difference (about 1) is below the usual threshold of 2, and the CV difference is within the noise of a 20-row data set. Choosing the primary model after seeing results would make its inference over-optimistic. M3 is reported as exploratory.
2. Coursework is likely on the causal path from study time to exam score (a mediator). Adjusting for it would estimate only the part of the study-hours association that does not pass through coursework, which is not the estimand. A higher R² does not make the estimand more correct.
3. No. They are two estimates of the same out-of-sample error from different resampling schemes; both are noisy at n = 20. LOOCV has low bias but high variance; 5-fold CV depends on the random split (seed 2026).
4. No, unless the row is a data error. It is a real student with an unusual combination (little sleep, average study, lower score). Keep the student, report the sensitivity analysis, and soften the claim about sleep.
5. Sections 1 (purpose and estimand: predicted score, not a slope), 4 (predictors chosen for predictive value, possibly including the interaction), 8 (validation becomes central, with calibration and ideally an external cohort) and 6 (accuracy metrics instead of coefficient interpretation). Section 10 would focus on transportability to new cohorts.

Revision history