MBA Regression for Managers Concept Card Template

Regression for Managers Concept Card: [Your business question or course]

A concept card for multiple linear regression as managers use it: reading coefficients, dummy variables, interactions, log transforms, R-squared, multicollinearity, omitted variable bias, correlation versus causation, and prediction intervals. It is for MBA students in the data and decisions core and for managers who read regression output from analysts and must decide what it does and does not support. Source basis: the least squares method of Legendre (1805) and Gauss (1809), Galton (1886), White (1980), Belsley, Kuh and Welsch (1980) and Holland (1986), written up in original words with a small fictional Northfield Outdoor Co. data set. Every number was computed in Python (statsmodels) and is shown with its Excel equivalent (LINEST). Replace every [bracketed] value and every row marked Example.

Document control

When to use it

One-sentence rule: use multiple regression when you want to describe how an outcome moves with several factors at once, predict the outcome for new cases, or (with a credible design) estimate the effect of one factor holding the others fixed.

The idea in plain language

Regression draws the best straight line (or flat surface, with several inputs) through a cloud of data points. "Best" means the line that makes the sum of squared vertical misses as small as possible: the method of least squares, published by Legendre in 1805 and developed by Gauss in 1809. Galton (1886) gave the method its name when he noticed that tall parents' children are on average less extreme than their parents, a "regression" towards the mean.

Each coefficient answers a "holding the others fixed" question. In a model of store sales on ad spend and floor area, the ad coefficient says how much sales differ between two stores of the same size whose ad budgets differ by one unit. That is the great strength of multiple regression and the source of its biggest trap: it only holds fixed the variables you included. Anything important you left out still moves with your variables and contaminates the coefficients (omitted variable bias).

Managers need four habits:

1. Interpret in units. A coefficient has the units of Y per unit of X. Say it in money and in plain words.
2. Read the uncertainty. A coefficient comes with a standard error and a confidence interval. A wide interval that includes zero says the data cannot tell you the sign, not that the effect is zero.
3. Separate prediction from cause. A model can predict well and still give a misleading coefficient for any one variable.
4. Use intervals for forecasts. A forecast for one new case needs a prediction interval, which is much wider than the confidence interval for the average.

Formulas and notation

Model:

$y_i = \beta_0 + \beta_1 x_{1i} + \beta_2 x_{2i} + \dots + \beta_k x_{ki} + \varepsilon_i$

Least squares chooses $b_0, \dots, b_k$ to minimise $\sum_i (y_i - \hat{y}_i)^2$. In matrix form $b = (X^\top X)^{-1} X^\top y$.

Fit:

$R^2 = 1 - \frac{SS_{\text{res}}}{SS_{\text{tot}}}, \qquad \bar{R}^2 = 1 - (1 - R^2)\frac{n - 1}{n - k - 1}, \qquad s_e = \sqrt{\frac{SS_{\text{res}}}{n - k - 1}}$

Inference on a coefficient: $t = b_j / SE(b_j)$ with $n - k - 1$ degrees of freedom; 95% CI $b_j \pm t_{0.975,\,n-k-1}\, SE(b_j)$.

Prediction at a new point $x_0$: confidence interval for the mean $\hat{y}_0 \pm t \cdot SE(\hat{y}_0)$; prediction interval for one new case

$\hat{y}_0 \pm t_{0.975,\,n-k-1} \sqrt{s_e^2 + SE(\hat{y}_0)^2}$

Omitted variable bias (long model with $x_1$ and $x_2$, short model drops $x_2$):

$b_1^{\text{short}} = b_1^{\text{long}} + b_2^{\text{long}} \cdot \delta_1, \qquad \delta_1 = \text{slope of } x_2 \text{ on } x_1 \text{ (and the other included regressors)}$

Variance inflation factor: $VIF_j = 1 / (1 - R_j^2)$, where $R_j^2$ comes from regressing $x_j$ on the other regressors.

Interpretation of transforms:

The data: Northfield Outdoor Co. stores

Northfield's retail director asks: "Should we raise local advertising in our stores, and what should we expect from a new urban store of 20,000 square feet with USD 8k a month of local ads?" The analytics team has one month of data for 20 stores (fictional).

Summary: mean sales 203.10 (SD 45.18), mean ads 7.27 (SD 1.96), mean area 20.04 (SD 6.85), 10 urban stores. Correlations: sales with ads 0.853, sales with area 0.898, ads with area 0.698, urban with area -0.404. The strong link between ads and area matters: larger stores get larger ad budgets.

The chart already hints at the answer: the big points (large stores) sit at the top right. Is it the ads or the size?

Assumptions and limits

In the Northfield model the residuals look well behaved (Jarque-Bera p = 0.988, Durbin-Watson 1.54), and the heteroskedasticity-robust (HC1) standard errors are smaller than the classical ones (ads 1.23 against 2.19), so the conclusions below do not hinge on the constant-spread assumption. With $n$ = 20 and three regressors, the data are thin: every interval is wide, and that is part of the lesson.

How managers use it in practice

• Benchmarking. Predicted sales from size and location give each store a fair target; the residual shows which stores over or under-perform for reasons the model does not capture.
• Budget questions. "What happens if we spend more on X?" is a causal question. Regression on observational data gives a first estimate and, more importantly, shows whether the answer is pinned down or wide open. When it is wide open, the next step is an experiment.
• Pricing and elasticity. A log-log model of quantity on price gives an elasticity directly (see mba/mba-demand-elasticity-card and mba/mba-pricing-card).
• Forecasting single cases. Site selection and sales targets need prediction intervals, not point forecasts.
• Reviewing analysts' work. The questions a manager should ask: what is left out, what is the range of the data, how wide are the intervals, and was the model chosen before or after looking at the results.

Worked example A (fully solved): what drives Northfield store sales?

Step 1: the naive model

Regress sales on ads alone: $\widehat{\text{sales}} = 60.36 + 19.63 \times \text{ads}$, $R^2 = 0.727$, SE of the ads slope 2.84. Excel: =SLOPE(B2:B21,C2:C21) returns 19.63, =INTERCEPT(B2:B21,C2:C21) returns 60.36, =RSQ(B2:B21,C2:C21) returns 0.727.

Read naively, "every USD 1k of ads brings USD 19.6k of sales". At a gross margin of 45 percent that would be a superb return. Do not act on it yet.

Step 2: the full model

import pandas as pd
import statsmodels.formula.api as smf

rows = [(190,6.4,16.2,1),(226,9.2,25.0,0),(240,9.3,22.8,1),(198,6.4,23.6,0),(200,8.0,17.5,1),
        (271,8.5,29.9,0),(301,10.6,30.0,1),(228,8.8,28.9,0),(161,6.4,12.2,1),(200,4.7,23.1,0),
        (232,9.2,22.7,1),(182,5.6,21.3,0),(227,9.6,20.6,1),(211,6.2,22.4,0),(181,5.7,13.3,1),
        (173,5.7,18.2,0),(156,5.7,9.7,1),(238,9.9,26.5,0),(142,5.4,8.4,1),(105,4.1,8.5,0)]
df = pd.DataFrame(rows, columns=["sales", "ads", "area", "urban"])
m2 = smf.ols("sales ~ ads + area + urban", data=df).fit()
print(m2.summary())

Output from the run (statsmodels 0.15). Omitted: the Date and Time lines, which also carry Prob (F-statistic) 9.23e-12 and Log-Likelihood -70.834, the closing notes and trailing spaces; every other line is as printed.

                            OLS Regression Results
==============================================================================
Dep. Variable:                  sales   R-squared:                       0.964
Model:                            OLS   Adj. R-squared:                  0.957
Method:                 Least Squares   F-statistic:                     142.9
No. Observations:                  20   AIC:                             149.7
Df Residuals:                      16   BIC:                             153.7
Df Model:                           3
Covariance Type:            nonrobust
==============================================================================
                 coef    std err          t      P>|t|      [0.025      0.975]
------------------------------------------------------------------------------
Intercept     40.5260      8.540      4.745      0.000      22.422      58.630
ads            2.1707      2.188      0.992      0.336      -2.467       6.809
area           6.4940      0.672      9.657      0.000       5.068       7.920
urban         33.3050      6.555      5.081      0.000      19.409      47.201
==============================================================================
Omnibus:                        0.487   Durbin-Watson:                   1.539
Prob(Omnibus):                  0.784   Jarque-Bera (JB):                0.023
Skew:                          -0.079   Prob(JB):                        0.988
Kurtosis:                       3.058   Cond. No.                         95.0
==============================================================================

The F statistic tests whether the three regressors together explain anything: $F = (SS_{\text{reg}}/k)/(SS_{\text{res}}/(n-k-1)) = (37{,}395.98/3)/(1{,}395.82/16) = 142.89$, with a p-value of $9.2 \times 10^{-12}$ (=F.DIST.RT(142.89,3,16)).

Interpretation, coefficient by coefficient:

• ads = 2.17: holding area and location fixed, a store with USD 1k more monthly ads has, on average, USD 2.17k more sales. The 95% CI runs from -2.47 to 6.81 and p = 0.336: the data cannot even tell whether the association is positive. At a 45 percent gross margin, USD 2.17k of sales returns about USD 0.98k of margin per USD 1k of ads, which does not pay back on the point estimate.
• area = 6.49: holding ads and location fixed, each extra 1,000 square feet goes with USD 6.49k more monthly sales (95% CI 5.07 to 7.92).
• urban = 33.31: a dummy variable. Urban stores sell USD 33.3k more per month than suburban stores of the same area and ad spend (95% CI 19.4 to 47.2). The suburban group is the reference category; including a dummy for both groups as well as an intercept would make the regressors perfectly collinear (the dummy variable trap).
• Intercept = 40.53: predicted sales for a suburban store with zero area and zero ads. No such store exists; the intercept is a fitting constant, not a business fact.
• $R^2$ = 0.964, adjusted 0.957: size, location and ads together account for 96 percent of the variation in sales across these 20 stores. A high $R^2$ says the model fits; it says nothing about whether the ads coefficient is causal. $s_e$ = 9.34: a typical store misses its prediction by about USD 9k.

Step 3: why the ads coefficient collapsed (omitted variable bias)

Drop area from the full model and the ads coefficient becomes 20.37. The omitted variable bias formula reproduces this exactly. Regress area on ads and urban: $\delta$ = 2.8018 (each USD 1k of ads goes with 2,802 more square feet). Then

$b_{\text{ads}}^{\text{short}} = 2.1707 + 6.4940 \times 2.8018 = 20.37$

Almost all of the naive "ad effect" is the size effect, carried by the fact that big stores get big budgets. Area is a confounder: it drives both ad spend (budget rules) and sales.

Step 4: multicollinearity

A VIF of 4 means the variance of the ads coefficient is about four times what it would be if ads were uncorrelated with the other regressors, so its standard error is about twice as large. These values are moderate (common rules of thumb flag 5 or 10), but with only 20 stores they help explain the wide interval on ads. Collinearity does not bias the coefficients; it makes them imprecise. The remedy is more data, or variation in ads that is not tied to size, which is what an experiment delivers.

Step 5: does the ad effect differ between urban and suburban stores? (interaction)

Add the product term: sales ~ ads * urban + area. Results: ads 2.41, urban 42.09, ads:urban -1.10 (p = 0.653), area 6.60. The ads slope is 2.41 for suburban stores and 2.41 - 1.10 = 1.31 for urban stores. The interaction is far from significant and adjusted $R^2$ falls from 0.9573 to 0.9551, so keep the simpler model. Note that with the interaction in the model, the urban coefficient (42.09) is the urban premium at zero ads, an extrapolation: main effects change meaning once an interaction is added.

Step 6: log transforms

np.log(sales) ~ np.log(ads) + area + urban gives: log(ads) 0.048 (SE 0.089), area 0.0354, urban 0.1874, $R^2$ = 0.959.

• Elasticity of sales with respect to ads: 0.048. A 10 percent rise in ad spend goes with about 0.5 percent more sales, holding size and location fixed; the interval includes zero.
• Area (semi-elasticity): each extra 1,000 square feet goes with about 3.5 percent more sales (exactly $e^{0.0354} - 1$ = 3.60 percent).
• Urban: urban stores sell about $e^{0.1874} - 1$ = 20.6 percent more than comparable suburban stores. For a dummy in a log model, use the exact formula; $100b$ = 18.7 percent understates it.

Step 7: prediction for the new urban store

For ads = 8, area = 20, urban = 1: $\hat{y} = 40.526 + 2.1707(8) + 6.4940(20) + 33.305 = 221.08$.

The prediction interval is about three times as wide as the confidence interval because a single store also carries its own random miss ($s_e$ = 9.34).

Step 8: the same model in Excel with LINEST

Put sales in B2:B21 and ads, area, urban in C2:E21. Select a 5-row by 4-column range and enter =LINEST(B2:B21,C2:E21,TRUE,TRUE) (in older Excel, confirm with Ctrl+Shift+Enter). The layout follows Microsoft's documentation for LINEST: row 1 holds the coefficients in reverse order of the input columns (urban, area, ads), with the intercept last; row 2 their standard errors in the same order; row 3 $R^2$ and the standard error of the y estimate ($s_e$); row 4 the F statistic and the residual degrees of freedom ($n - k - 1$ = 16); row 5 the regression and residual sums of squares. Cells of rows 3 to 5 beyond column 2 return #N/A.

Check row 4 by hand: $F = (SS_{\text{reg}}/3)/(SS_{\text{res}}/16) = 12{,}465.33/87.24 = 142.89$, the same as statsmodels. A spreadsheet engine that divides $SS_{\text{reg}}$ by 4 (counting the intercept) would show 107.17, which is wrong. Useful companions: t critical value =T.INV.2T(0.05,16) = 2.1199; p-value for ads =T.DIST.2T(2.1707/2.1878,16) = 0.336; a single coefficient without the array: =INDEX(LINEST(B2:B21,C2:E21,TRUE,TRUE),1,3). The Analysis ToolPak Regression tool gives the same numbers in a labelled table.

Step 9: the answer to the retail director

"Bigger and urban stores sell more; ad spend adds little once size and location are accounted for, and these 20 stores cannot tell us whether the ad effect is positive. The naive USD 19.6k per USD 1k figure is mostly a size effect. Do not raise ad budgets on this evidence. Run a geo-randomised test: vary ad spend across matched stores for eight weeks and measure the difference. For the new urban store of 20,000 square feet with USD 8k ads, plan on about USD 221k a month, with a realistic range of USD 200k to 242k."

Correlation versus causation

A regression coefficient is a conditional association. It becomes a causal effect only if, after conditioning on the included variables, the regressor is as good as randomly assigned. Holland (1986) framed this as the fundamental problem of causal inference: we never see the same store with and without the extra ad spend.

Designs that support causal claims: randomised experiments and A/B tests (academic/stats-experiment-design), natural experiments, difference-in-differences, instrumental variables and regression discontinuity. Name them in your recommendation even when you cannot run them yet.

Worked example B (half solved): what if location is left out?

The team first fitted sales ~ ads + area without the urban dummy. Output: intercept 51.307, ads 10.145 (SE 2.390), area 3.894 (SE 0.684), $R^2$ = 0.906, adjusted $R^2$ = 0.8949.

1. Interpret the ads coefficient in one sentence: holding [ ] fixed, USD 1k more ads goes with USD [ ]k more sales.
2. t statistic for ads: $10.145 / 2.390 = [\ ]$. Is it significant at 5 percent with 17 df (critical value 2.110)? [ ]
3. The full model gives 2.1707 for ads and 33.305 for urban. Regressing urban on ads and area gives a slope on ads of 0.2394. Use the OVB formula: $2.1707 + 33.305 \times [\ ] = [\ ]$. Does it match 10.145? [ ]
4. Why does leaving out location inflate the ads coefficient here? Use the sign of each term. [ ]
5. Predict sales for a store with ads = 7 and area = 18 from this model: $51.307 + 10.145(7) + 3.894(18) = [\ ]$.
6. Which model would you show the retail director, and why? [ ]

Exercise C (unsolved): a size-and-location benchmark

1. Fit sales ~ area + urban (no ads). Report the coefficients, their standard errors, $R^2$ and adjusted $R^2$.
2. Compare adjusted $R^2$ with the full model. What does dropping ads cost in fit?
3. Compute each store's residual. Which three stores over-perform most for their size and location? Suggest two reasons a store manager could investigate.
4. Give a 95% prediction interval for a suburban store of 25,000 square feet. Excel hint: compute $\hat{y}$ from the LINEST row 1, then use =T.INV.2T(0.05,17) and the standard error of prediction from software.
5. Fit np.log(sales) ~ np.log(area) + urban and interpret the area coefficient as an elasticity.

Connections to other cards

• Statistics library: derivations, diagnostics and reporting in academic/stats-linear-regression, academic/stats-regression-model-report and academic/stats-assumption-checklist.
• Decision analytics course: where regression sits among forecasting, testing and optimisation: mba/mba-decision-analytics-guide.
• Decision trees: regression outputs (forecasts, hit rates) feed tree probabilities: mba/mba-decision-trees-card.
• Marketing: response models and marketing-mix models: mba/mba-marketing-mix-card, mba/mba-customer-lifetime-value-card.
• Economics: log-log demand and elasticities: mba/mba-demand-elasticity-card.
• Finance: a market-model regression of a stock's returns on index returns gives beta: mba/mba-capm-wacc-card.
• AI for managers: predictive models extend regression; the same prediction-versus-cause discipline applies: mba/mba-ai-decision-making-card.

Common mistakes and misconceptions

1. Reading a coefficient as causal without a design that supports it (Step 1 of Worked example A).
2. Forgetting "holding the others fixed". A multiple regression coefficient is not the same as the simple correlation slope.
3. "Not significant" read as "no effect". The ads interval runs from -2.5 to 6.8: a meaningful effect is still possible. Absence of evidence is not evidence of absence.
4. Chasing R-squared. Adding variables never lowers $R^2$; compare adjusted $R^2$ and out-of-sample error instead.
5. Dropping a variable because of high VIF when it is a confounder. Collinearity costs precision; omitting a confounder costs bias.
6. The dummy variable trap: one dummy per category plus an intercept. Use $c - 1$ dummies for $c$ categories.
7. Misreading main effects after adding an interaction. They become effects at zero of the other variable.
8. Using 100b percent for large log coefficients or dummies instead of $100(e^b - 1)$ percent.
9. Using a confidence interval as a forecast range for one case. Use the prediction interval.
10. Extrapolating beyond the data: the Northfield stores range from 8,400 to 30,000 square feet; a 60,000 square foot flagship is outside the model.
11. Interpreting the intercept as a real business quantity when zero lies outside the data.

Self-test questions

1. State the interpretation sentence for the area coefficient in the full Northfield model.
2. Why did the ads coefficient fall from 19.63 to 2.17 when area and location were added?
3. Write the omitted variable bias formula and say when the bias is zero.
4. What does a VIF of 4 mean for the standard error of a coefficient?
5. A log-sales model has a dummy coefficient of 0.40. What is the exact percentage difference?
6. Why is the prediction interval wider than the confidence interval for the mean?
7. In LINEST output for inputs in columns C, D, E, which column of row 1 holds the coefficient of column C?
8. An analyst reports $R^2$ = 0.96 and concludes that the ads effect is reliable. What do you reply?
9. Give two alternative explanations, other than ads causing sales, for a positive ads coefficient.
10. When is a randomised experiment better than a regression on existing data, and what does it cost?

My notes and next steps

Sources and further reading

• Legendre, A.-M. (1805). Nouvelles méthodes pour la détermination des orbites des comètes. Paris: Firmin Didot.
• Gauss, C. F. (1809). Theoria motus corporum coelestium in sectionibus conicis solem ambientium. Hamburg: Perthes and Besser.
• Galton, F. (1886). Regression towards mediocrity in hereditary stature. Journal of the Anthropological Institute of Great Britain and Ireland, 15, 246-263.
• White, H. (1980). A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica, 48(4), 817-838.
• Belsley, D. A., Kuh, E., and Welsch, R. E. (1980). Regression Diagnostics: Identifying Influential Data and Sources of Collinearity. Wiley.
• Holland, P. W. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81(396), 945-960.
• Textbooks (further reading, by name only): Wooldridge, Introductory Econometrics: A Modern Approach; Stock and Watson, Introduction to Econometrics; Angrist and Pischke, Mostly Harmless Econometrics.
• Open resources: OpenStax, Introductory Statistics 2e (CC BY-NC-SA 4.0), chapter on linear regression and correlation; statsmodels documentation for ols and get_prediction; Microsoft support page for the LINEST function.

Answer key

Worked example B. (1) Holding area fixed, USD 1k more ads goes with USD 10.15k more sales. (2) $t$ = 4.24; yes, 4.24 > 2.110. (3) $2.1707 + 33.305 \times 0.2394 = 10.145$; it matches. (4) Urban stores earn a positive premium (33.3) and, at a given area, spend more on ads (0.2394 per USD 1k), so part of the urban premium is credited to ads: positive times positive gives upward bias. (5) $51.307 + 71.015 + 70.092 = 192.41$; the unrounded coefficients give 192.42. (6) The full model: it includes the confounder (location), has the higher adjusted $R^2$ (0.957 against 0.895), and its ads coefficient is less biased, though still not proven causal.

Exercise C (parts 1, 2 and 4). (1) Intercept 42.459 (SE 8.311), area 7.069 (SE 0.342), urban 37.971 (SE 4.565), $R^2$ = 0.9618, adjusted 0.9573. (2) Adjusted $R^2$ is the same to four decimals as the full model (0.9573): ads adds almost nothing once area and location are known. (4) Point 219.18; 95% prediction interval 198.45 to 239.90. Parts 3 and 5 are for your own work.

Self-test.

1. Holding ads and location fixed, each extra 1,000 square feet goes with USD 6.49k more monthly sales on average.
2. Area is correlated with ads (0.698) and strongly raises sales; leaving it out loads the size effect onto ads. In the model with ads and urban only, the ads coefficient is 20.37, and the OVB term $6.494 \times 2.8018$ = 18.20 accounts exactly for the gap to 2.17.
3. $b_1^{\text{short}} = b_1^{\text{long}} + b_2^{\text{long}}\delta_1$. Zero when the omitted variable has no effect on $y$ ($b_2$ = 0) or is uncorrelated with the included regressor ($\delta_1$ = 0).
4. Its variance is four times, and so its standard error about twice, what it would be with uncorrelated regressors.
5. $100(e^{0.40} - 1)$ = 49.2 percent.
6. It adds the variance of one new observation's error ($s_e^2$) to the uncertainty about the mean.
7. The third column (LINEST reverses the order; the intercept is in the last column).
8. $R^2$ measures fit, not the reliability of one coefficient; the ads interval (-2.5 to 6.8) includes zero and the coefficient may still be confounded.
9. Reverse causality (successful stores get bigger budgets); confounding (store size, location, manager quality).
10. When you need a causal effect to make an investment decision and you can vary the input; the costs are time, possible lost sales in the control group and the operational effort of randomising.

Revision history

Standard and licence note

Based on the least squares method (Legendre 1805, Gauss 1809), Galton (1886), White (1980), Belsley, Kuh and Welsch (1980) and Holland (1986). The text, data and exercises are original and fictional; no textbook, published case or course material was copied or paraphrased. This template is not affiliated with or endorsed by any business school, publisher, author or software vendor. Check the current documentation of your software (Excel, statsmodels) before you rely on a specific output layout. Template licence: MIT.