# Regression for Managers Concept Card: [Your business question or course] > A concept card for multiple linear regression as managers use it: reading coefficients, dummy variables, > interactions, log transforms, R-squared, multicollinearity, omitted variable bias, correlation versus causation, and > prediction intervals. It is for MBA students in the data and decisions core and for managers who read regression > output from analysts and must decide what it does and does not support. Source basis: the least squares method of > Legendre (1805) and Gauss (1809), Galton (1886), White (1980), Belsley, Kuh and Welsch (1980) and Holland (1986), written > up in original words with a small fictional Northfield Outdoor Co. data set. Every number was computed in Python > (statsmodels) and is shown with its Excel equivalent (`LINEST`). > Replace every [bracketed] value and every row marked *Example*. ## Document control | Field | Value | |---|---| | Document ID | [DA-REG-001] | | Course or project | [Data and decisions core / store performance review] | | Learner or analyst | [Your name] | | Decision owner | [@owner] | | Reviewed by | [Study group / analytics lead] | | Data set | [Northfield Outdoor Co. store panel, one month, 20 stores (fictional) / your data and source] | | Software | Excel (LINEST, T.INV.2T, T.DIST.2T); Python 3.14 with pandas and statsmodels | | Last updated | YYYY-MM-DD | | Version | [0.1] | | Classification | [Course work / internal] | ## When to use it **One-sentence rule:** use multiple regression when you want to describe how an outcome moves with several factors at once, predict the outcome for new cases, or (with a credible design) estimate the effect of one factor holding the others fixed. | Good uses | Poor uses | |---|---| | Explaining store sales from size, location and marketing spend | Proving that ads cause sales from observational data alone | | Forecasting a new store's sales with an honest range | Predicting far outside the range of the data (extrapolation) | | Adjusting a comparison for differences in size or mix | Binary outcomes such as churn yes or no (use logistic regression, academic/stats-logistic-regression) | | Estimating price or advertising elasticities (with log transforms) | Time series with trend and autocorrelation, without time-series checks (see academic/stats-time-series-arima) | | Screening which factors deserve an experiment | Replacing a randomised experiment when one is feasible (see academic/stats-experiment-design) | ## The idea in plain language Regression draws the best straight line (or flat surface, with several inputs) through a cloud of data points. "Best" means the line that makes the sum of squared vertical misses as small as possible: the method of least squares, published by Legendre in 1805 and developed by Gauss in 1809. Galton (1886) gave the method its name when he noticed that tall parents' children are on average less extreme than their parents, a "regression" towards the mean. Each coefficient answers a "holding the others fixed" question. In a model of store sales on ad spend and floor area, the ad coefficient says how much sales differ between two stores **of the same size** whose ad budgets differ by one unit. That is the great strength of multiple regression and the source of its biggest trap: it only holds fixed the variables you included. Anything important you left out still moves with your variables and contaminates the coefficients (omitted variable bias). Managers need four habits: 1. **Interpret in units.** A coefficient has the units of Y per unit of X. Say it in money and in plain words. 2. **Read the uncertainty.** A coefficient comes with a standard error and a confidence interval. A wide interval that includes zero says the data cannot tell you the sign, not that the effect is zero. 3. **Separate prediction from cause.** A model can predict well and still give a misleading coefficient for any one variable. 4. **Use intervals for forecasts.** A forecast for one new case needs a prediction interval, which is much wider than the confidence interval for the average. ## Formulas and notation **Model:** $$y_i = \beta_0 + \beta_1 x_{1i} + \beta_2 x_{2i} + \dots + \beta_k x_{ki} + \varepsilon_i$$ **Least squares** chooses $b_0, \dots, b_k$ to minimise $\sum_i (y_i - \hat{y}_i)^2$. In matrix form $b = (X^\top X)^{-1} X^\top y$. **Fit:** $$R^2 = 1 - \frac{SS_{\text{res}}}{SS_{\text{tot}}}, \qquad \bar{R}^2 = 1 - (1 - R^2)\frac{n - 1}{n - k - 1}, \qquad s_e = \sqrt{\frac{SS_{\text{res}}}{n - k - 1}}$$ **Inference on a coefficient:** $t = b_j / SE(b_j)$ with $n - k - 1$ degrees of freedom; 95% CI $b_j \pm t_{0.975,\,n-k-1}\, SE(b_j)$. **Prediction at a new point $x_0$:** confidence interval for the mean $\hat{y}_0 \pm t \cdot SE(\hat{y}_0)$; prediction interval for one new case $$\hat{y}_0 \pm t_{0.975,\,n-k-1} \sqrt{s_e^2 + SE(\hat{y}_0)^2}$$ **Omitted variable bias** (long model with $x_1$ and $x_2$, short model drops $x_2$): $$b_1^{\text{short}} = b_1^{\text{long}} + b_2^{\text{long}} \cdot \delta_1, \qquad \delta_1 = \text{slope of } x_2 \text{ on } x_1 \text{ (and the other included regressors)}$$ **Variance inflation factor:** $VIF_j = 1 / (1 - R_j^2)$, where $R_j^2$ comes from regressing $x_j$ on the other regressors. **Interpretation of transforms:** | Model | Coefficient $b$ means | |---|---| | $y = b_0 + b x$ | one more unit of $x$ goes with $b$ more units of $y$ | | $\ln y = b_0 + b x$ | one more unit of $x$ goes with about $100b$ percent more $y$ (exactly $100(e^b - 1)$ percent) | | $y = b_0 + b \ln x$ | a 1 percent rise in $x$ goes with about $b/100$ more units of $y$ | | $\ln y = b_0 + b \ln x$ | elasticity: a 1 percent rise in $x$ goes with about $b$ percent more $y$ | | Symbol | Meaning | |---|---| | $y_i$ | Outcome for observation $i$ (here monthly store sales, USD k) | | $x_{ji}$ | Value of regressor $j$ for observation $i$ | | $\beta_j$, $b_j$ | True coefficient and its least squares estimate | | $\varepsilon_i$ | Error term: everything not in the model | | $n$, $k$ | Number of observations, number of regressors (excluding the intercept) | | $SS_{\text{res}}$, $SS_{\text{tot}}$ | Residual and total sums of squares | | $s_e$ | Standard error of the regression (typical size of a miss) | | $SE(\hat{y}_0)$ | Standard error of the fitted mean at $x_0$ | ## The data: Northfield Outdoor Co. stores Northfield's retail director asks: "Should we raise local advertising in our stores, and what should we expect from a new urban store of 20,000 square feet with USD 8k a month of local ads?" The analytics team has one month of data for 20 stores (fictional). | Store | Sales (USD k) | Ads (USD k) | Area (k sq ft) | Urban | Store | Sales | Ads | Area | Urban | |---|---|---|---|---|---|---|---|---|---| | S01 | 190 | 6.4 | 16.2 | 1 | S11 | 232 | 9.2 | 22.7 | 1 | | S02 | 226 | 9.2 | 25.0 | 0 | S12 | 182 | 5.6 | 21.3 | 0 | | S03 | 240 | 9.3 | 22.8 | 1 | S13 | 227 | 9.6 | 20.6 | 1 | | S04 | 198 | 6.4 | 23.6 | 0 | S14 | 211 | 6.2 | 22.4 | 0 | | S05 | 200 | 8.0 | 17.5 | 1 | S15 | 181 | 5.7 | 13.3 | 1 | | S06 | 271 | 8.5 | 29.9 | 0 | S16 | 173 | 5.7 | 18.2 | 0 | | S07 | 301 | 10.6 | 30.0 | 1 | S17 | 156 | 5.7 | 9.7 | 1 | | S08 | 228 | 8.8 | 28.9 | 0 | S18 | 238 | 9.9 | 26.5 | 0 | | S09 | 161 | 6.4 | 12.2 | 1 | S19 | 142 | 5.4 | 8.4 | 1 | | S10 | 200 | 4.7 | 23.1 | 0 | S20 | 105 | 4.1 | 8.5 | 0 | Summary: mean sales 203.10 (SD 45.18), mean ads 7.27 (SD 1.96), mean area 20.04 (SD 6.85), 10 urban stores. Correlations: sales with ads 0.853, sales with area 0.898, **ads with area 0.698**, urban with area -0.404. The strong link between ads and area matters: larger stores get larger ad budgets. ```vega-lite { "$schema": "https://vega.github.io/schema/vega-lite/v5.json", "description": "Northfield Outdoor Co. stores: monthly sales against local ad spend, point size by floor area, colour by location (fictional)", "width": 400, "height": 240, "data": {"values": [ {"store": "S01", "sales": 190, "ads": 6.4, "area": 16.2, "loc": "Urban"}, {"store": "S02", "sales": 226, "ads": 9.2, "area": 25.0, "loc": "Suburban"}, {"store": "S03", "sales": 240, "ads": 9.3, "area": 22.8, "loc": "Urban"}, {"store": "S04", "sales": 198, "ads": 6.4, "area": 23.6, "loc": "Suburban"}, {"store": "S05", "sales": 200, "ads": 8.0, "area": 17.5, "loc": "Urban"}, {"store": "S06", "sales": 271, "ads": 8.5, "area": 29.9, "loc": "Suburban"}, {"store": "S07", "sales": 301, "ads": 10.6, "area": 30.0, "loc": "Urban"}, {"store": "S08", "sales": 228, "ads": 8.8, "area": 28.9, "loc": "Suburban"}, {"store": "S09", "sales": 161, "ads": 6.4, "area": 12.2, "loc": "Urban"}, {"store": "S10", "sales": 200, "ads": 4.7, "area": 23.1, "loc": "Suburban"}, {"store": "S11", "sales": 232, "ads": 9.2, "area": 22.7, "loc": "Urban"}, {"store": "S12", "sales": 182, "ads": 5.6, "area": 21.3, "loc": "Suburban"}, {"store": "S13", "sales": 227, "ads": 9.6, "area": 20.6, "loc": "Urban"}, {"store": "S14", "sales": 211, "ads": 6.2, "area": 22.4, "loc": "Suburban"}, {"store": "S15", "sales": 181, "ads": 5.7, "area": 13.3, "loc": "Urban"}, {"store": "S16", "sales": 173, "ads": 5.7, "area": 18.2, "loc": "Suburban"}, {"store": "S17", "sales": 156, "ads": 5.7, "area": 9.7, "loc": "Urban"}, {"store": "S18", "sales": 238, "ads": 9.9, "area": 26.5, "loc": "Suburban"}, {"store": "S19", "sales": 142, "ads": 5.4, "area": 8.4, "loc": "Urban"}, {"store": "S20", "sales": 105, "ads": 4.1, "area": 8.5, "loc": "Suburban"} ]}, "mark": {"type": "point", "filled": true, "opacity": 0.75}, "encoding": { "x": {"field": "ads", "type": "quantitative", "title": "Local ad spend (USD k per month)", "scale": {"zero": false}}, "y": {"field": "sales", "type": "quantitative", "title": "Sales (USD k per month)", "scale": {"zero": false}}, "size": {"field": "area", "type": "quantitative", "title": "Area (k sq ft)"}, "color": {"field": "loc", "type": "nominal", "title": "Location"}, "tooltip": [{"field": "store"}, {"field": "sales"}, {"field": "ads"}, {"field": "area"}] } } ``` The chart already hints at the answer: the big points (large stores) sit at the top right. Is it the ads or the size? ## Assumptions and limits | Assumption | What it means | How to check | If it fails | |---|---|---|---| | Linearity | The average of $y$ is a straight-line function of each $x$, given the others | Residuals against fitted values and against each $x$ | Add a log or squared term, or split into segments | | Exogeneity (no omitted confounder) | The error is unrelated to the regressors | Thinking, not testing: list what drives both $x$ and $y$ | Add the confounder, use an experiment, or use a causal design | | Independence | Observations do not influence each other | Design: stores in the same city, repeated months | Cluster-robust standard errors, panel or mixed models (academic/stats-mixed-models) | | Constant error spread (homoskedasticity) | The size of the misses does not grow with $x$ | Residual plot fanning out | Heteroskedasticity-robust standard errors (White 1980), or log $y$ | | Roughly normal errors | Needed for exact small-sample t and prediction intervals | Q-Q plot of residuals | Larger $n$ helps; bootstrap (academic/stats-resampling-methods) | | No extreme influence | One store should not drive the result | Leverage and Cook's distance | Check the data; report with and without the point | | No perfect collinearity | No regressor is an exact combination of others | Software drops a variable or fails | Remove the redundant variable (the dummy trap) | In the Northfield model the residuals look well behaved (Jarque-Bera p = 0.988, Durbin-Watson 1.54), and the heteroskedasticity-robust (HC1) standard errors are smaller than the classical ones (ads 1.23 against 2.19), so the conclusions below do not hinge on the constant-spread assumption. With $n$ = 20 and three regressors, the data are thin: every interval is wide, and that is part of the lesson. ## How managers use it in practice - **Benchmarking.** Predicted sales from size and location give each store a fair target; the residual shows which stores over or under-perform for reasons the model does not capture. - **Budget questions.** "What happens if we spend more on X?" is a causal question. Regression on observational data gives a first estimate and, more importantly, shows whether the answer is pinned down or wide open. When it is wide open, the next step is an experiment. - **Pricing and elasticity.** A log-log model of quantity on price gives an elasticity directly (see mba/mba-demand-elasticity-card and mba/mba-pricing-card). - **Forecasting single cases.** Site selection and sales targets need prediction intervals, not point forecasts. - **Reviewing analysts' work.** The questions a manager should ask: what is left out, what is the range of the data, how wide are the intervals, and was the model chosen before or after looking at the results. ## Worked example A (fully solved): what drives Northfield store sales? ### Step 1: the naive model Regress sales on ads alone: $\widehat{\text{sales}} = 60.36 + 19.63 \times \text{ads}$, $R^2 = 0.727$, SE of the ads slope 2.84. Excel: `=SLOPE(B2:B21,C2:C21)` returns 19.63, `=INTERCEPT(B2:B21,C2:C21)` returns 60.36, `=RSQ(B2:B21,C2:C21)` returns 0.727. Read naively, "every USD 1k of ads brings USD 19.6k of sales". At a gross margin of 45 percent that would be a superb return. Do not act on it yet. ### Step 2: the full model ```python import pandas as pd import statsmodels.formula.api as smf rows = [(190,6.4,16.2,1),(226,9.2,25.0,0),(240,9.3,22.8,1),(198,6.4,23.6,0),(200,8.0,17.5,1), (271,8.5,29.9,0),(301,10.6,30.0,1),(228,8.8,28.9,0),(161,6.4,12.2,1),(200,4.7,23.1,0), (232,9.2,22.7,1),(182,5.6,21.3,0),(227,9.6,20.6,1),(211,6.2,22.4,0),(181,5.7,13.3,1), (173,5.7,18.2,0),(156,5.7,9.7,1),(238,9.9,26.5,0),(142,5.4,8.4,1),(105,4.1,8.5,0)] df = pd.DataFrame(rows, columns=["sales", "ads", "area", "urban"]) m2 = smf.ols("sales ~ ads + area + urban", data=df).fit() print(m2.summary()) ``` Output from the run (statsmodels 0.15). Omitted: the Date and Time lines, which also carry Prob (F-statistic) 9.23e-12 and Log-Likelihood -70.834, the closing notes and trailing spaces; every other line is as printed. ```text OLS Regression Results ============================================================================== Dep. Variable: sales R-squared: 0.964 Model: OLS Adj. R-squared: 0.957 Method: Least Squares F-statistic: 142.9 No. Observations: 20 AIC: 149.7 Df Residuals: 16 BIC: 153.7 Df Model: 3 Covariance Type: nonrobust ============================================================================== coef std err t P>|t| [0.025 0.975] ------------------------------------------------------------------------------ Intercept 40.5260 8.540 4.745 0.000 22.422 58.630 ads 2.1707 2.188 0.992 0.336 -2.467 6.809 area 6.4940 0.672 9.657 0.000 5.068 7.920 urban 33.3050 6.555 5.081 0.000 19.409 47.201 ============================================================================== Omnibus: 0.487 Durbin-Watson: 1.539 Prob(Omnibus): 0.784 Jarque-Bera (JB): 0.023 Skew: -0.079 Prob(JB): 0.988 Kurtosis: 3.058 Cond. No. 95.0 ============================================================================== ``` The F statistic tests whether the three regressors together explain anything: $F = (SS_{\text{reg}}/k)/(SS_{\text{res}}/(n-k-1)) = (37{,}395.98/3)/(1{,}395.82/16) = 142.89$, with a p-value of $9.2 \times 10^{-12}$ (`=F.DIST.RT(142.89,3,16)`). **Interpretation, coefficient by coefficient:** - **ads = 2.17:** holding area and location fixed, a store with USD 1k more monthly ads has, on average, USD 2.17k more sales. The 95% CI runs from -2.47 to 6.81 and p = 0.336: the data cannot even tell whether the association is positive. At a 45 percent gross margin, USD 2.17k of sales returns about USD 0.98k of margin per USD 1k of ads, which does not pay back on the point estimate. - **area = 6.49:** holding ads and location fixed, each extra 1,000 square feet goes with USD 6.49k more monthly sales (95% CI 5.07 to 7.92). - **urban = 33.31:** a **dummy variable**. Urban stores sell USD 33.3k more per month than suburban stores of the same area and ad spend (95% CI 19.4 to 47.2). The suburban group is the reference category; including a dummy for both groups as well as an intercept would make the regressors perfectly collinear (the dummy variable trap). - **Intercept = 40.53:** predicted sales for a suburban store with zero area and zero ads. No such store exists; the intercept is a fitting constant, not a business fact. - **$R^2$ = 0.964, adjusted 0.957:** size, location and ads together account for 96 percent of the variation in sales across these 20 stores. A high $R^2$ says the model fits; it says nothing about whether the ads coefficient is causal. $s_e$ = 9.34: a typical store misses its prediction by about USD 9k. ### Step 3: why the ads coefficient collapsed (omitted variable bias) Drop area from the full model and the ads coefficient becomes 20.37. The omitted variable bias formula reproduces this exactly. Regress area on ads and urban: $\delta$ = 2.8018 (each USD 1k of ads goes with 2,802 more square feet). Then $$b_{\text{ads}}^{\text{short}} = 2.1707 + 6.4940 \times 2.8018 = 20.37$$ Almost all of the naive "ad effect" is the size effect, carried by the fact that big stores get big budgets. Area is a **confounder**: it drives both ad spend (budget rules) and sales. ### Step 4: multicollinearity | Regressor | VIF | |---|---| | ads | 4.01 | | area | 4.63 | | urban | 2.46 | A VIF of 4 means the variance of the ads coefficient is about four times what it would be if ads were uncorrelated with the other regressors, so its standard error is about twice as large. These values are moderate (common rules of thumb flag 5 or 10), but with only 20 stores they help explain the wide interval on ads. Collinearity does not bias the coefficients; it makes them imprecise. The remedy is more data, or variation in ads that is not tied to size, which is what an experiment delivers. ### Step 5: does the ad effect differ between urban and suburban stores? (interaction) Add the product term: `sales ~ ads * urban + area`. Results: ads 2.41, urban 42.09, ads:urban -1.10 (p = 0.653), area 6.60. The ads slope is 2.41 for suburban stores and 2.41 - 1.10 = 1.31 for urban stores. The interaction is far from significant and adjusted $R^2$ falls from 0.9573 to 0.9551, so keep the simpler model. Note that with the interaction in the model, the urban coefficient (42.09) is the urban premium at zero ads, an extrapolation: main effects change meaning once an interaction is added. ### Step 6: log transforms `np.log(sales) ~ np.log(ads) + area + urban` gives: log(ads) 0.048 (SE 0.089), area 0.0354, urban 0.1874, $R^2$ = 0.959. - **Elasticity of sales with respect to ads:** 0.048. A 10 percent rise in ad spend goes with about 0.5 percent more sales, holding size and location fixed; the interval includes zero. - **Area (semi-elasticity):** each extra 1,000 square feet goes with about 3.5 percent more sales (exactly $e^{0.0354} - 1$ = 3.60 percent). - **Urban:** urban stores sell about $e^{0.1874} - 1$ = 20.6 percent more than comparable suburban stores. For a dummy in a log model, use the exact formula; $100b$ = 18.7 percent understates it. ### Step 7: prediction for the new urban store For ads = 8, area = 20, urban = 1: $\hat{y} = 40.526 + 2.1707(8) + 6.4940(20) + 33.305 = 221.08$. | Interval | Lower | Upper | Use it for | |---|---|---|---| | 95% CI for the mean of all such stores | 214.35 | 227.81 | Average performance of a format | | **95% prediction interval for this one store** | **200.16** | **241.99** | Setting this store's target and budget | The prediction interval is about three times as wide as the confidence interval because a single store also carries its own random miss ($s_e$ = 9.34). ### Step 8: the same model in Excel with LINEST Put sales in `B2:B21` and ads, area, urban in `C2:E21`. Select a 5-row by 4-column range and enter `=LINEST(B2:B21,C2:E21,TRUE,TRUE)` (in older Excel, confirm with Ctrl+Shift+Enter). The layout follows Microsoft's documentation for LINEST: row 1 holds the coefficients **in reverse order** of the input columns (urban, area, ads), with the intercept last; row 2 their standard errors in the same order; row 3 $R^2$ and the standard error of the y estimate ($s_e$); row 4 the F statistic and the residual degrees of freedom ($n - k - 1$ = 16); row 5 the regression and residual sums of squares. Cells of rows 3 to 5 beyond column 2 return #N/A. | Row | Column 1 (urban) | Column 2 (area) | Column 3 (ads) | Column 4 (intercept) | |---|---|---|---|---| | 1: coefficients | 33.3050 | 6.4940 | 2.1707 | 40.5260 | | 2: standard errors | 6.5551 | 0.6725 | 2.1878 | 8.5401 | | 3: $R^2$, $s_e$ | 0.9640 | 9.3402 | #N/A | #N/A | | 4: F, residual df | 142.89 | 16 | #N/A | #N/A | | 5: $SS_{\text{reg}}$, $SS_{\text{res}}$ | 37395.98 | 1395.82 | #N/A | #N/A | Check row 4 by hand: $F = (SS_{\text{reg}}/3)/(SS_{\text{res}}/16) = 12{,}465.33/87.24 = 142.89$, the same as statsmodels. A spreadsheet engine that divides $SS_{\text{reg}}$ by 4 (counting the intercept) would show 107.17, which is wrong. Useful companions: t critical value `=T.INV.2T(0.05,16)` = 2.1199; p-value for ads `=T.DIST.2T(2.1707/2.1878,16)` = 0.336; a single coefficient without the array: `=INDEX(LINEST(B2:B21,C2:E21,TRUE,TRUE),1,3)`. The Analysis ToolPak Regression tool gives the same numbers in a labelled table. ### Step 9: the answer to the retail director "Bigger and urban stores sell more; ad spend adds little once size and location are accounted for, and these 20 stores cannot tell us whether the ad effect is positive. The naive USD 19.6k per USD 1k figure is mostly a size effect. Do not raise ad budgets on this evidence. Run a geo-randomised test: vary ad spend across matched stores for eight weeks and measure the difference. For the new urban store of 20,000 square feet with USD 8k ads, plan on about USD 221k a month, with a realistic range of USD 200k to 242k." ## Correlation versus causation A regression coefficient is a conditional association. It becomes a causal effect only if, after conditioning on the included variables, the regressor is as good as randomly assigned. Holland (1986) framed this as the fundamental problem of causal inference: we never see the same store with and without the extra ad spend. | Threat | Northfield illustration | Remedy | |---|---|---| | Confounding (omitted variable) | Area drives both ad budgets and sales | Include it, or randomise ad spend | | Reverse causality | Budget rules give stores with high last-year sales more ads | Use lagged or planned budgets; experiment | | Selection | Only surviving stores are in the data | Include closed stores; state the population | | Measurement error | Ads recorded at region level and split by a formula | Better data; error attenuates coefficients towards zero | | Overfitting and data dredging | Trying 30 variables and reporting the best | Fix the model before looking; hold out data | Designs that support causal claims: randomised experiments and A/B tests (academic/stats-experiment-design), natural experiments, difference-in-differences, instrumental variables and regression discontinuity. Name them in your recommendation even when you cannot run them yet. ```mermaid flowchart TD Q["Business question"] --> K{"Prediction or cause?"} K -->|Prediction| P1["Choose inputs available at prediction time"] P1 --> P2["Fit, check residuals, hold out data"] P2 --> P3["Report prediction intervals"] K -->|Cause| C1["Draw the causal story: confounders, reverse causality"] C1 --> C2{"Can you randomise?"} C2 -->|Yes| C3["Run an experiment or A/B test"] C2 -->|No| C4["Regression with confounders, robustness checks"] C4 --> C5["State the remaining threats in the recommendation"] C3 --> D["Decision with effect size and interval"] C5 --> D P3 --> D ``` ## Worked example B (half solved): what if location is left out? The team first fitted `sales ~ ads + area` without the urban dummy. Output: intercept 51.307, ads 10.145 (SE 2.390), area 3.894 (SE 0.684), $R^2$ = 0.906, adjusted $R^2$ = 0.8949. 1. Interpret the ads coefficient in one sentence: holding [ ] fixed, USD 1k more ads goes with USD [ ]k more sales. 2. t statistic for ads: $10.145 / 2.390 = [\ ]$. Is it significant at 5 percent with 17 df (critical value 2.110)? [ ] 3. The full model gives 2.1707 for ads and 33.305 for urban. Regressing urban on ads and area gives a slope on ads of 0.2394. Use the OVB formula: $2.1707 + 33.305 \times [\ ] = [\ ]$. Does it match 10.145? [ ] 4. Why does leaving out location inflate the ads coefficient here? Use the sign of each term. [ ] 5. Predict sales for a store with ads = 7 and area = 18 from this model: $51.307 + 10.145(7) + 3.894(18) = [\ ]$. 6. Which model would you show the retail director, and why? [ ] ## Exercise C (unsolved): a size-and-location benchmark 1. Fit `sales ~ area + urban` (no ads). Report the coefficients, their standard errors, $R^2$ and adjusted $R^2$. 2. Compare adjusted $R^2$ with the full model. What does dropping ads cost in fit? 3. Compute each store's residual. Which three stores over-perform most for their size and location? Suggest two reasons a store manager could investigate. 4. Give a 95% prediction interval for a suburban store of 25,000 square feet. Excel hint: compute $\hat{y}$ from the LINEST row 1, then use `=T.INV.2T(0.05,17)` and the standard error of prediction from software. 5. Fit `np.log(sales) ~ np.log(area) + urban` and interpret the area coefficient as an elasticity. ## Connections to other cards - **Statistics library:** derivations, diagnostics and reporting in academic/stats-linear-regression, academic/stats-regression-model-report and academic/stats-assumption-checklist. - **Decision analytics course:** where regression sits among forecasting, testing and optimisation: mba/mba-decision-analytics-guide. - **Decision trees:** regression outputs (forecasts, hit rates) feed tree probabilities: mba/mba-decision-trees-card. - **Marketing:** response models and marketing-mix models: mba/mba-marketing-mix-card, mba/mba-customer-lifetime-value-card. - **Economics:** log-log demand and elasticities: mba/mba-demand-elasticity-card. - **Finance:** a market-model regression of a stock's returns on index returns gives beta: mba/mba-capm-wacc-card. - **AI for managers:** predictive models extend regression; the same prediction-versus-cause discipline applies: mba/mba-ai-decision-making-card. ## Common mistakes and misconceptions 1. **Reading a coefficient as causal** without a design that supports it (Step 1 of Worked example A). 2. **Forgetting "holding the others fixed".** A multiple regression coefficient is not the same as the simple correlation slope. 3. **"Not significant" read as "no effect".** The ads interval runs from -2.5 to 6.8: a meaningful effect is still possible. Absence of evidence is not evidence of absence. 4. **Chasing R-squared.** Adding variables never lowers $R^2$; compare adjusted $R^2$ and out-of-sample error instead. 5. **Dropping a variable because of high VIF** when it is a confounder. Collinearity costs precision; omitting a confounder costs bias. 6. **The dummy variable trap:** one dummy per category plus an intercept. Use $c - 1$ dummies for $c$ categories. 7. **Misreading main effects after adding an interaction.** They become effects at zero of the other variable. 8. **Using 100b percent for large log coefficients or dummies** instead of $100(e^b - 1)$ percent. 9. **Using a confidence interval as a forecast range** for one case. Use the prediction interval. 10. **Extrapolating** beyond the data: the Northfield stores range from 8,400 to 30,000 square feet; a 60,000 square foot flagship is outside the model. 11. **Interpreting the intercept** as a real business quantity when zero lies outside the data. ## Self-test questions 1. State the interpretation sentence for the area coefficient in the full Northfield model. 2. Why did the ads coefficient fall from 19.63 to 2.17 when area and location were added? 3. Write the omitted variable bias formula and say when the bias is zero. 4. What does a VIF of 4 mean for the standard error of a coefficient? 5. A log-sales model has a dummy coefficient of 0.40. What is the exact percentage difference? 6. Why is the prediction interval wider than the confidence interval for the mean? 7. In LINEST output for inputs in columns C, D, E, which column of row 1 holds the coefficient of column C? 8. An analyst reports $R^2$ = 0.96 and concludes that the ads effect is reliable. What do you reply? 9. Give two alternative explanations, other than ads causing sales, for a positive ads coefficient. 10. When is a randomised experiment better than a regression on existing data, and what does it cost? ## My notes and next steps | Date | Note | Next step | |---|---|---| | YYYY-MM-DD | [What I understood today] | [Card or exercise to do next] | | YYYY-MM-DD | *Example:* I can now explain OVB with the Northfield numbers | Do Exercise C and present it in two minutes | | YYYY-MM-DD | [What still confuses me] | [Question for office hours] | ## Sources and further reading - Legendre, A.-M. (1805). *Nouvelles méthodes pour la détermination des orbites des comètes*. Paris: Firmin Didot. - Gauss, C. F. (1809). *Theoria motus corporum coelestium in sectionibus conicis solem ambientium*. Hamburg: Perthes and Besser. - Galton, F. (1886). Regression towards mediocrity in hereditary stature. *Journal of the Anthropological Institute of Great Britain and Ireland*, 15, 246-263. - White, H. (1980). A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. *Econometrica*, 48(4), 817-838. - Belsley, D. A., Kuh, E., and Welsch, R. E. (1980). *Regression Diagnostics: Identifying Influential Data and Sources of Collinearity*. Wiley. - Holland, P. W. (1986). Statistics and causal inference. *Journal of the American Statistical Association*, 81(396), 945-960. - Textbooks (further reading, by name only): Wooldridge, *Introductory Econometrics: A Modern Approach*; Stock and Watson, *Introduction to Econometrics*; Angrist and Pischke, *Mostly Harmless Econometrics*. - Open resources: OpenStax, *Introductory Statistics 2e* (CC BY-NC-SA 4.0), chapter on linear regression and correlation; statsmodels documentation for `ols` and `get_prediction`; Microsoft support page for the LINEST function. ## Answer key **Worked example B.** (1) Holding area fixed, USD 1k more ads goes with USD 10.15k more sales. (2) $t$ = 4.24; yes, 4.24 > 2.110. (3) $2.1707 + 33.305 \times 0.2394 = 10.145$; it matches. (4) Urban stores earn a positive premium (33.3) and, at a given area, spend more on ads (0.2394 per USD 1k), so part of the urban premium is credited to ads: positive times positive gives upward bias. (5) $51.307 + 71.015 + 70.092 = 192.41$; the unrounded coefficients give 192.42. (6) The full model: it includes the confounder (location), has the higher adjusted $R^2$ (0.957 against 0.895), and its ads coefficient is less biased, though still not proven causal. **Exercise C (parts 1, 2 and 4).** (1) Intercept 42.459 (SE 8.311), area 7.069 (SE 0.342), urban 37.971 (SE 4.565), $R^2$ = 0.9618, adjusted 0.9573. (2) Adjusted $R^2$ is the same to four decimals as the full model (0.9573): ads adds almost nothing once area and location are known. (4) Point 219.18; 95% prediction interval 198.45 to 239.90. Parts 3 and 5 are for your own work. **Self-test.** 1. Holding ads and location fixed, each extra 1,000 square feet goes with USD 6.49k more monthly sales on average. 2. Area is correlated with ads (0.698) and strongly raises sales; leaving it out loads the size effect onto ads. In the model with ads and urban only, the ads coefficient is 20.37, and the OVB term $6.494 \times 2.8018$ = 18.20 accounts exactly for the gap to 2.17. 3. $b_1^{\text{short}} = b_1^{\text{long}} + b_2^{\text{long}}\delta_1$. Zero when the omitted variable has no effect on $y$ ($b_2$ = 0) or is uncorrelated with the included regressor ($\delta_1$ = 0). 4. Its variance is four times, and so its standard error about twice, what it would be with uncorrelated regressors. 5. $100(e^{0.40} - 1)$ = 49.2 percent. 6. It adds the variance of one new observation's error ($s_e^2$) to the uncertainty about the mean. 7. The third column (LINEST reverses the order; the intercept is in the last column). 8. $R^2$ measures fit, not the reliability of one coefficient; the ads interval (-2.5 to 6.8) includes zero and the coefficient may still be confounded. 9. Reverse causality (successful stores get bigger budgets); confounding (store size, location, manager quality). 10. When you need a causal effect to make an investment decision and you can vary the input; the costs are time, possible lost sales in the control group and the operational effort of randomising. ## Revision history | Version | Date | Author | Change summary | |---|---|---|---| | 0.1 | YYYY-MM-DD | [Your name] | First draft from the template | | 0.2 | YYYY-MM-DD | [Your name] | *Example:* Added 12 months of data, cluster-robust standard errors by store | ## Standard and licence note Based on the least squares method (Legendre 1805, Gauss 1809), Galton (1886), White (1980), Belsley, Kuh and Welsch (1980) and Holland (1986). The text, data and exercises are original and fictional; no textbook, published case or course material was copied or paraphrased. This template is not affiliated with or endorsed by any business school, publisher, author or software vendor. Check the current documentation of your software (Excel, statsmodels) before you rely on a specific output layout. Template licence: MIT.