# Bayesian Inference Concept Card: [Your study question or course] > A concept card for Bayesian inference: prior, likelihood and > posterior, the beta-binomial and normal-normal conjugate models, > credible intervals and how they differ from confidence intervals, the > posterior predictive distribution, prior sensitivity, an overview of > Bayes factors with cautions, and grid approximation in base R and > numpy. It is for students meeting Bayesian ideas for the first time, > master's and PhD students deciding whether a Bayesian analysis suits > their thesis, and analysts who want to state "the probability that the > rate exceeds 50%" honestly. It speeds up learning by working one model > fully by formula and by code, fading the next example, and ending with > self-test questions whose answers sit in a separate key. No special > software is needed: everything runs in base R and numpy/scipy. The > worked examples use fictional data; replace them with your own. > Source basis: Bayes (1763), Jeffreys (1961), Kass and > Raftery (1995), Gelman et al. *Bayesian Data Analysis* (named as > further reading), with all text and examples written fresh. ## Document control | Field | Value | |---|---| | Course or project | [STAT 330 Bayesian Statistics / thesis methods chapter] | | Learner or analyst | [Your name] | | Data set | [Lakemoor University app survey, 14 of 20; Brightwell Bakery loaf weights, n = 12 (fictional)] | | Software and versions | R 4.5.1 (stats); Python 3.14, numpy 2.5.4, scipy 1.18.1 | | Date | YYYY-MM-DD | | Version | [0.1] | ## The idea in one paragraph Bayesian inference treats an unknown parameter as uncertain and describes that uncertainty with a probability distribution. You start with a prior distribution (what is plausible before the data), multiply it by the likelihood (how well each parameter value predicts the observed data), and rescale to get the posterior distribution (what is plausible after the data). Every answer then comes from the posterior: a best estimate, a credible interval that contains the parameter with a stated probability, the probability that the parameter exceeds a threshold, or predictions for new data. The price is that the prior must be chosen and defended, and results should be checked for sensitivity to that choice. ## Intuition Think of a weather forecaster updating a forecast. Before looking at today's radar, she has a belief based on the season (prior). The radar (data) is more or less compatible with different weather outcomes (likelihood). Her updated forecast (posterior) blends the two, leaning towards whichever source is more informative. A vague seasonal belief is quickly overruled by clear radar; a very confident belief needs a lot of radar evidence to shift. Tomorrow, today's posterior becomes the new prior: Bayesian updating is sequential learning. The chart shows a weak Beta(2, 2) prior for the share θ of Lakemoor University (fictional) students who use the library app, and the posterior Beta(16, 8) after 14 of 20 surveyed students said yes. The data moved the centre from 0.5 to about 0.67 and made the distribution much narrower. ```vega-lite { "$schema": "https://vega.github.io/schema/vega-lite/v5.json", "description": "Prior Beta(2,2) and posterior Beta(16,8) densities, computed in R with dbeta", "width": 380, "height": 240, "data": {"values": [ {"theta": 0.00, "density": 0.000, "curve": "Prior Beta(2,2)"}, {"theta": 0.05, "density": 0.285, "curve": "Prior Beta(2,2)"}, {"theta": 0.10, "density": 0.540, "curve": "Prior Beta(2,2)"}, {"theta": 0.15, "density": 0.765, "curve": "Prior Beta(2,2)"}, {"theta": 0.20, "density": 0.960, "curve": "Prior Beta(2,2)"}, {"theta": 0.25, "density": 1.125, "curve": "Prior Beta(2,2)"}, {"theta": 0.30, "density": 1.260, "curve": "Prior Beta(2,2)"}, {"theta": 0.35, "density": 1.365, "curve": "Prior Beta(2,2)"}, {"theta": 0.40, "density": 1.440, "curve": "Prior Beta(2,2)"}, {"theta": 0.45, "density": 1.485, "curve": "Prior Beta(2,2)"}, {"theta": 0.50, "density": 1.500, "curve": "Prior Beta(2,2)"}, {"theta": 0.55, "density": 1.485, "curve": "Prior Beta(2,2)"}, {"theta": 0.60, "density": 1.440, "curve": "Prior Beta(2,2)"}, {"theta": 0.65, "density": 1.365, "curve": "Prior Beta(2,2)"}, {"theta": 0.70, "density": 1.260, "curve": "Prior Beta(2,2)"}, {"theta": 0.75, "density": 1.125, "curve": "Prior Beta(2,2)"}, {"theta": 0.80, "density": 0.960, "curve": "Prior Beta(2,2)"}, {"theta": 0.85, "density": 0.765, "curve": "Prior Beta(2,2)"}, {"theta": 0.90, "density": 0.540, "curve": "Prior Beta(2,2)"}, {"theta": 0.95, "density": 0.285, "curve": "Prior Beta(2,2)"}, {"theta": 1.00, "density": 0.000, "curve": "Prior Beta(2,2)"}, {"theta": 0.00, "density": 0.000, "curve": "Posterior Beta(16,8)"}, {"theta": 0.05, "density": 0.000, "curve": "Posterior Beta(16,8)"}, {"theta": 0.10, "density": 0.000, "curve": "Posterior Beta(16,8)"}, {"theta": 0.15, "density": 0.000, "curve": "Posterior Beta(16,8)"}, {"theta": 0.20, "density": 0.000, "curve": "Posterior Beta(16,8)"}, {"theta": 0.25, "density": 0.000, "curve": "Posterior Beta(16,8)"}, {"theta": 0.30, "density": 0.005, "curve": "Posterior Beta(16,8)"}, {"theta": 0.35, "density": 0.028, "curve": "Posterior Beta(16,8)"}, {"theta": 0.40, "density": 0.118, "curve": "Posterior Beta(16,8)"}, {"theta": 0.45, "density": 0.375, "curve": "Posterior Beta(16,8)"}, {"theta": 0.50, "density": 0.935, "curve": "Posterior Beta(16,8)"}, {"theta": 0.55, "density": 1.868, "curve": "Posterior Beta(16,8)"}, {"theta": 0.60, "density": 3.022, "curve": "Posterior Beta(16,8)"}, {"theta": 0.65, "density": 3.942, "curve": "Posterior Beta(16,8)"}, {"theta": 0.70, "density": 4.073, "curve": "Posterior Beta(16,8)"}, {"theta": 0.75, "density": 3.199, "curve": "Posterior Beta(16,8)"}, {"theta": 0.80, "density": 1.767, "curve": "Posterior Beta(16,8)"}, {"theta": 0.85, "density": 0.585, "curve": "Posterior Beta(16,8)"}, {"theta": 0.90, "density": 0.081, "curve": "Posterior Beta(16,8)"}, {"theta": 0.95, "density": 0.001, "curve": "Posterior Beta(16,8)"}, {"theta": 1.00, "density": 0.000, "curve": "Posterior Beta(16,8)"} ]}, "mark": {"type": "line", "interpolate": "monotone"}, "encoding": { "x": {"field": "theta", "type": "quantitative", "title": "Share of students using the app (theta)"}, "y": {"field": "density", "type": "quantitative", "title": "Density"}, "color": {"field": "curve", "type": "nominal", "title": null} } } ``` ```mermaid flowchart TD A["Define the parameter and the question"] --> B["Choose a likelihood (data model)"] B --> C["Choose and justify a prior"] C --> D["Prior predictive check: are simulated data plausible?"] D --> E{"Conjugate pair?"} E -- Yes --> F["Update by formula"] E -- "No, one or two parameters" --> G["Grid approximation"] E -- "No, many parameters" --> H["MCMC: Stan, PyMC, brms"] F --> I["Summarise posterior: mean, credible interval, probabilities"] G --> I H --> I I --> J["Posterior predictive check against the data"] J --> K["Prior sensitivity analysis"] K --> L["Report prior, model, posterior summaries"] ``` ## Formal definition **Bayes' theorem for parameters.** The posterior is proportional to the prior times the likelihood; the denominator, the marginal likelihood, only rescales so the posterior integrates to 1. $$p(\theta \mid y) = \frac{p(y \mid \theta)\,p(\theta)}{p(y)}, \qquad p(y) = \int p(y \mid \theta)\,p(\theta)\,d\theta, \qquad p(\theta \mid y) \propto p(y \mid \theta)\,p(\theta)$$ **Beta-binomial (conjugate).** With x successes in n trials and a Beta(a, b) prior on θ, the posterior is again a beta distribution: add successes to a and failures to b. The prior acts like a + b earlier observations. $$\theta \sim \text{Beta}(a, b), \quad x \mid \theta \sim \text{Binomial}(n, \theta) \;\Rightarrow\; \theta \mid x \sim \text{Beta}(a + x,\; b + n - x)$$ $$E[\theta \mid x] = \frac{a + x}{a + b + n} = w\,\frac{a}{a+b} + (1 - w)\,\frac{x}{n}, \qquad w = \frac{a + b}{a + b + n}$$ The posterior mean is a weighted average of the prior mean and the sample proportion; the data weight grows with n. **Normal-normal (conjugate, σ known).** With a normal prior on the mean μ and n observations with known SD σ, precisions (one over variances) add, and the posterior mean is a precision-weighted average. $$\mu \sim N(\mu_0, \tau_0^2), \quad y_i \mid \mu \sim N(\mu, \sigma^2) \;\Rightarrow\; \mu \mid y \sim N(\mu_n, \tau_n^2)$$ $$\frac{1}{\tau_n^2} = \frac{1}{\tau_0^2} + \frac{n}{\sigma^2}, \qquad \mu_n = \tau_n^2\left(\frac{\mu_0}{\tau_0^2} + \frac{n\,\bar{y}}{\sigma^2}\right)$$ **Credible interval.** A 95% equal-tailed credible interval runs from the 2.5th to the 97.5th percentile of the posterior, so $P(L \le \theta \le U \mid y) = 0.95$. A highest-density interval (HDI) is the narrowest interval with 95% posterior probability; it differs from the equal-tailed one when the posterior is skewed. **Posterior predictive.** Predictions for new data average the data model over the posterior, so they carry both sampling noise and parameter uncertainty. $$p(\tilde{y} \mid y) = \int p(\tilde{y} \mid \theta)\,p(\theta \mid y)\,d\theta$$ For the beta-binomial, the number of successes in m new trials follows a beta-binomial distribution with parameters m, a + x and b + n − x. **Bayes factor.** The ratio of marginal likelihoods of two models: how much more probable the data are under H1 than under H0. Posterior odds = Bayes factor × prior odds. $$BF_{10} = \frac{p(y \mid H_1)}{p(y \mid H_0)}, \qquad \frac{P(H_1 \mid y)}{P(H_0 \mid y)} = BF_{10} \times \frac{P(H_1)}{P(H_0)}$$ | Symbol | Meaning | |---|---| | $\theta$ | Parameter (here a proportion) | | $p(\theta)$ | Prior distribution | | $p(y \mid \theta)$ | Likelihood: data model read as a function of θ | | $p(\theta \mid y)$ | Posterior distribution | | $p(y)$ | Marginal likelihood (evidence) | | $a$, $b$ | Beta prior parameters (prior successes and failures) | | $x$, $n$ | Observed successes and number of trials | | $\mu_0$, $\tau_0$ | Prior mean and prior SD for μ | | $\sigma$ | Known data SD | | $\mu_n$, $\tau_n$ | Posterior mean and posterior SD | | $\tilde{y}$ | New, not yet observed data | | $BF_{10}$ | Bayes factor for H1 over H0 | ### Credible versus confidence intervals A 95% credible interval supports the direct statement "given the model, the prior and the data, θ lies in this interval with probability 0.95". A 95% confidence interval supports only a statement about the long-run behaviour of the method (see stats-confidence-intervals). With a flat or weak prior and moderate n, the two intervals are often numerically close, but they answer different questions. When the prior is informative, they can differ substantially, and the credible interval is only as good as the prior. ### Bayes factors: overview and cautions Bayes factors compare hypotheses directly and can express evidence for H0, which a p-value cannot. Rough verbal scales exist. Jeffreys (1961) graded Bayes factors in half-powers of 10: from 1 to about 3.2 the evidence is worth only a bare mention, from 3.2 to 10 it is substantial, and above 100 it is decisive. Kass and Raftery (1995) reprinted that scale and also proposed one on the 2 log BF scale, with cut-points near BF = 3, 20 and 150 for positive, strong and very strong evidence. Cautions: Bayes factors depend strongly on the prior under H1, even when the posterior for θ barely changes; a very vague prior under H1 pushes the Bayes factor towards H0 (the Jeffreys-Lindley paradox); and verbal labels are as arbitrary as α = 0.05. Report the prior used and, where possible, a sensitivity analysis. For estimation questions, posterior intervals are usually more informative than a Bayes factor. ## Worked example A: fully solved **Question.** Lakemoor University (fictional) asks 20 randomly chosen students whether they use the library app; 14 say yes. What share θ of all students use it, and how probable is it that a majority do? **Step 1: likelihood.** Independent yes/no answers: x = 14 successes in n = 20 binomial trials. **Step 2: prior.** A weak Beta(2, 2) prior: centred on 0.5, gently ruling out values very close to 0 or 1, worth about 4 earlier observations. **Step 3: update by hand.** Posterior Beta(2 + 14, 2 + 6) = Beta(16, 8). Posterior mean = 16 / 24 = 0.667. The data weight is 1 − w = 20 / 24 = 0.83, so the estimate sits 83% of the way from the prior mean 0.5 to the sample proportion 0.70. **Step 4: compute summaries.** Outputs on this card are trimmed to the key lines (for example R's `attr(,"conf.level")` lines are dropped). ```r x <- 14; n <- 20; a0 <- 2; b0 <- 2 a1 <- a0 + x; b1 <- b0 + n - x round(c(a1 = a1, b1 = b1, mean = a1 / (a1 + b1), mode = (a1 - 1) / (a1 + b1 - 2), sd = sqrt(a1 * b1 / ((a1 + b1)^2 * (a1 + b1 + 1))), lo = qbeta(0.025, a1, b1), hi = qbeta(0.975, a1, b1), p_gt_half = 1 - pbeta(0.5, a1, b1)), 4) round(prop.test(14, 20, correct = FALSE)$conf.int, 4) # frequentist Wilson CI # Posterior predictive: how many of the next 10 students use the app? m <- 10; k <- 0:m pp <- choose(m, k) * beta(a1 + k, b1 + m - k) / beta(a1, b1) round(c(mean = sum(k * pp), p_ge7 = sum(pp[k >= 7])), 4) ``` ```text a1 b1 mean mode sd lo hi p_gt_half 16.0000 8.0000 0.6667 0.6818 0.0943 0.4708 0.8362 0.9534 [1] 0.4810 0.8545 mean p_ge7 6.6667 0.5601 ``` ```python from scipy import stats x, n, a0, b0 = 14, 20, 2, 2 post = stats.beta(a0 + x, b0 + n - x) print(round(post.mean(), 4), post.ppf([0.025, 0.975]).round(4), round(post.sf(0.5), 4)) pp = stats.betabinom(10, a0 + x, b0 + n - x) # posterior predictive print(round(pp.mean(), 4), round(pp.sf(6), 4)) # sf(6) = P(K >= 7) ``` ```text 0.6667 [0.4708 0.8362] 0.9534 6.6667 0.5601 ``` **Step 5: interpret.** Given the weak prior and the survey, the share of app users is most plausibly around 0.67, with a 95% credible interval from 0.47 to 0.84. The posterior probability that a majority use the app is 0.95. In the next 10 students, we expect about 6.7 users, and there is a 56% chance that 7 or more of them will be users. **Step 6: compare with the frequentist view.** The Wilson 95% CI is [0.48, 0.85], close to the credible interval because the prior is weak. The two-sided exact binomial test of H0: θ = 0.5 gives p = 0.115: not significant. The Bayesian summary "P(θ > 0.5 | data) = 0.95" and the p-value answer different questions, so they do not contradict each other. *Report:* With a Beta(2, 2) prior, the posterior for the share of app users was Beta(16, 8): posterior mean 0.67, 95% credible interval [0.47, 0.84], P(θ > 0.5) = 0.95. ### Prior sensitivity table Always show how much the conclusions depend on the prior. Same data (14 of 20), five priors (the code continues from Example A, so `x` and `n` are already defined; the table repeats R's rounded output): ```r pri <- data.frame(prior = c("Beta(1,1) flat", "Beta(0.5,0.5) Jeffreys", "Beta(2,2) weak", "Beta(10,10) strong at 0.5", "Beta(2,8) sceptical"), a = c(1, 0.5, 2, 10, 2), b = c(1, 0.5, 2, 10, 8)) pri$post_mean <- (pri$a + x) / (pri$a + pri$b + n) pri$lo <- qbeta(0.025, pri$a + x, pri$b + n - x) pri$hi <- qbeta(0.975, pri$a + x, pri$b + n - x) pri$p_gt_half <- 1 - pbeta(0.5, pri$a + x, pri$b + n - x) cbind(pri[, 1:3], round(pri[, 4:7], 3)) ``` | Prior | Posterior mean | 95% credible interval | P(θ > 0.5 \| data) | |---|---|---|---| | Beta(1, 1) flat | 0.682 | [0.478, 0.854] | 0.961 | | Beta(0.5, 0.5) Jeffreys | 0.690 | [0.483, 0.864] | 0.965 | | Beta(2, 2) weak | 0.667 | [0.471, 0.836] | 0.953 | | Beta(10, 10) strong at 0.5 | 0.600 | [0.446, 0.744] | 0.900 | | Beta(2, 8) sceptical | 0.533 | [0.357, 0.706] | 0.644 | The three vague priors agree closely: the data dominate. A strong prior at 0.5 (worth 20 observations, as many as the survey) pulls the mean to 0.60; a sceptical prior expecting about 20% usage pulls it to 0.53 and drops P(θ > 0.5) to 0.64. With only 20 observations, a strong prior matters, so it must be justified from real earlier evidence. ### Bayes factor for the same data Compare H0: θ = 0.5 with H1: θ ~ Beta(1, 1). Under a uniform prior the marginal probability of any count x is 1 / (n + 1). ```r m0 <- dbinom(14, 20, 0.5); m1 <- 1 / 21 round(c(m0 = m0, m1 = m1, BF10 = m1 / m0), 4) m1b <- choose(20, 14) * beta(10 + 14, 10 + 6) / beta(10, 10) # H1: Beta(10,10) round(c(m1b = m1b, BF10b = m1b / m0), 4) ``` ```text m0 m1 BF10 0.0370 0.0476 1.2882 m1b BF10b 0.0593 1.6054 ``` The Bayes factor is 1.29 with a uniform prior under H1 and 1.61 with a Beta(10, 10) prior: the data favour H1 only slightly, which is weak evidence on either scale above, and the value visibly depends on the H1 prior. The posterior probability of a majority (0.95) and the Bayes factor (about 1.3) answer different questions: the first asks where θ is, the second whether θ is exactly 0.5. ## Worked example B: half solved **Question.** Brightwell Bakery (fictional) weighs 12 sourdough loaves (mean 497.55 g). The scale's and process's SD is known from long experience to be σ = 2.5 g. Before today, the head baker believed the batch mean was near 500 g, give or take about 4 g (two prior SDs): prior μ ~ N(500, 2²). What is the posterior for the batch mean? ```r sigma <- 2.5; mu0 <- 500; tau0 <- 2 w <- c(497.2, 501.5, 494.8, 498.9, 496.1, 499.4, 495.7, 500.2, 497.8, 493.9, 498.5, 496.6) n <- length(w); ybar <- mean(w) prec <- 1 / tau0^2 + n / sigma^2 mun <- (mu0 / tau0^2 + n * ybar / sigma^2) / prec; taun <- sqrt(1 / prec) round(c(prior_prec = 1 / tau0^2, data_prec = n / sigma^2, post_prec = prec, post_mean = mun, post_sd = taun, lo = mun - 1.96 * taun, hi = mun + 1.96 * taun, p_below_500 = pnorm(500, mun, taun), weight_data = (n / sigma^2) / prec), 4) ``` ```text prior_prec data_prec post_prec post_mean post_sd lo 0.2500 1.9200 2.1700 497.8323 0.6788 496.5017 hi p_below_500 weight_data 499.1628 0.9993 0.8848 ``` ```python import numpy as np from scipy import stats sigma, mu0, tau0 = 2.5, 500, 2 w = np.array([497.2, 501.5, 494.8, 498.9, 496.1, 499.4, 495.7, 500.2, 497.8, 493.9, 498.5, 496.6]) prec = 1 / tau0**2 + w.size / sigma**2 mun = (mu0 / tau0**2 + w.size * w.mean() / sigma**2) / prec; taun = prec**-0.5 print(round(mun, 4), round(taun, 4), stats.norm.ppf([0.025, 0.975], mun, taun).round(4), round(stats.norm.cdf(500, mun, taun), 4)) ``` ```text 497.8323 0.6788 [496.5017 499.1628] 0.9993 ``` 1. Prior precision = 1 / 2² = [ ]; data precision = 12 / 2.5² = [ ]. 2. Posterior precision = [ ]; posterior SD = [ ] g. 3. Weight on the data = data precision / posterior precision = [ ]; so the posterior mean sits [ ]% of the way from 500 to 497.55. 4. Posterior mean = [ ] g; 95% credible interval [ ] to [ ] g. 5. P(μ < 500 \| data) = [ ]. Write one sentence for the bakery manager using this number. 6. Compare with the frequentist t-interval [496.11, 498.99] (stats- hypothesis-testing, Example A): why is the credible interval [narrower / wider], and why is its centre [closer to / further from] 500? [ ]. ## Exercise C: unsolved Harbour City Library (fictional) checks 15 returned books and finds 3 returned late. Use a flat Beta(1, 1) prior for the late-return rate θ. 1. Write down the posterior distribution. 2. Compute the posterior mean. 3. Find the 95% equal-tailed credible interval (use `qbeta` or `stats.beta.ppf`). 4. Compute P(θ < 0.3 \| data). 5. A colleague says, "So there is a 95% chance θ is in that interval." Is that correct for this interval? Would it be correct for a confidence interval? 6. How would a Beta(5, 20) prior from last year's audit change the posterior mean? Compute it. ## Simulation you can run **Grid approximation.** For one parameter, chop its range into many points, compute prior × likelihood at each, and normalise. No conjugacy or special software needed. It reproduces the exact Beta(16, 8) answer and then handles a non-conjugate triangular prior just as easily. ```r grid <- seq(0, 1, length.out = 1001) prior <- dbeta(grid, 2, 2) lik <- dbinom(14, 20, grid) post <- prior * lik; post <- post / sum(post) cdf <- cumsum(post) round(c(mean = sum(grid * post), lo = grid[which(cdf >= 0.025)[1]], hi = grid[which(cdf >= 0.975)[1]], p_gt_half = sum(post[grid > 0.5])), 4) prior2 <- 1 - abs(grid - 0.5) * 2 # triangular prior peaked at 0.5 post2 <- prior2 * lik; post2 <- post2 / sum(post2); cdf2 <- cumsum(post2) round(c(mean = sum(grid * post2), lo = grid[which(cdf2 >= 0.025)[1]], hi = grid[which(cdf2 >= 0.975)[1]]), 4) set.seed(2026) # Monte Carlo from the posterior th <- rbeta(10000, 16, 8) round(c(mean = mean(th), p_gt_half = mean(th > 0.5), quantile(th, c(0.025, 0.975))), 4) ``` ```text mean lo hi p_gt_half 0.6667 0.4710 0.8360 0.9530 mean lo hi 0.6546 0.4650 0.8280 mean p_gt_half 2.5% 97.5% 0.6660 0.9499 0.4665 0.8351 ``` ```python import numpy as np from scipy import stats grid = np.linspace(0, 1, 1001) prior = stats.beta.pdf(grid, 2, 2) lik = stats.binom.pmf(14, 20, grid) post = prior * lik; post /= post.sum(); cdf = np.cumsum(post) print(round((grid * post).sum(), 4), round(grid[np.argmax(cdf >= 0.025)], 3), round(grid[np.argmax(cdf >= 0.975)], 3), round(post[grid > 0.5].sum(), 4)) rng = np.random.default_rng(2026) th = rng.beta(16, 8, 10000) print(round(th.mean(), 4), round((th > 0.5).mean(), 4), np.quantile(th, [0.025, 0.975]).round(4)) ``` ```text 0.6667 0.471 0.836 0.953 0.667 0.9532 [0.4712 0.8351] ``` The grid answer (mean 0.6667, interval [0.471, 0.836], P(θ > 0.5) = 0.953) matches the exact beta results to the grid's resolution of 0.001. The triangular prior gives a slightly lower mean (0.655) because it puts more weight near 0.5. Sampling 10,000 draws from the posterior gives P(θ > 0.5) of 0.950 in R and 0.953 in Python: Monte Carlo answers wobble around the exact 0.9534, and R and Python generators differ even with the same seed. Posterior sampling is the idea behind MCMC. **Next steps.** Grids become impractical beyond two or three parameters, because the number of grid points explodes. For regression and hierarchical models, use Markov chain Monte Carlo through Stan (via `rstan`, `cmdstanr` or `brms` in R, `cmdstanpy` in Python) or PyMC in Python, and check convergence diagnostics such as R-hat and effective sample size. ## Misconceptions | Wrong statement | Correct statement | |---|---| | Bayesian results are subjective, so they are unscientific. | Every analysis has assumptions; a Bayesian one states the prior openly, and a sensitivity table shows how much it matters. | | A flat prior means "no information". | Flatness depends on the scale: flat on θ is not flat on log-odds. Priors always carry some information. | | The posterior probability P(θ > 0.5 \| data) = 0.95 is the same as 1 − p. | p-values are computed assuming H0; posterior probabilities condition on the data and need a prior. They answer different questions. | | A 95% credible interval and a 95% confidence interval mean the same thing. | Only the credible interval gives a probability for the parameter, and only under the stated prior and model. | | A Bayes factor of 1.3 means H1 is 1.3 times as probable as H0. | It multiplies the prior odds. With prior odds of 1, the posterior odds are 1.3; with other prior odds, they differ. | | Bayes factors do not depend on the prior. | They depend heavily on the prior under H1, more than the posterior for θ does. | | A strong prior can be chosen to get the answer you want. | It can, which is why priors must be set before seeing the data, justified and reported, with sensitivity checks (see stats-analysis-plan). | | Bayesian methods need special software. | One- and two-parameter models run in base R or numpy with conjugate formulas or a grid; MCMC software is for larger models. | ## Connections to other concepts - **Probability basics** (stats-probability-basics): Bayes' theorem for events is the same rule applied to parameters. - **Maximum likelihood** (stats-estimation-mle): the likelihood is shared; with a flat prior, the posterior mode equals the MLE. - **Confidence intervals** (stats-confidence-intervals): credible intervals answer the question many people think a confidence interval answers. - **Hypothesis testing** (stats-hypothesis-testing): Bayes factors and posterior probabilities are alternatives to p-values, with their own cautions. - **Distributions** (stats-distributions-sheet): beta, binomial, normal and beta-binomial are the building blocks here. - **Analysis plans** (stats-analysis-plan, science/preregistration): priors, like α and hypotheses, belong in the plan before the data. ## Self-test 1. State Bayes' theorem for a parameter θ and name each part. 2. A Beta(3, 7) prior meets 9 successes in 10 trials. What is the posterior, and what is its mean? 3. Explain in one sentence why the posterior mean in the beta-binomial model is a compromise between the prior mean and x / n. 4. In the normal-normal model, what happens to the posterior when n grows very large? When the prior SD τ0 grows very large? 5. Give one sentence that is true of a 95% credible interval but not of a 95% confidence interval. 6. Name two cautions when reporting a Bayes factor. ## My notes and next steps | Item | Entry | |---|---| | Confidence today (1-5) | [3] | | Hardest idea for me | [*Example:* why Bayes factors are so sensitive to the H1 prior] | | Open questions | [Question for the tutor or supervisor] | | Prior I would defend for my own data | [Distribution and its justification] | | Mistakes logged in stats-error-log | [Entry IDs] | | Next review date (spacing) | YYYY-MM-DD | | Next template to study | [stats-resampling-methods] | ## Sources and further reading Structure aligned with a first Bayesian course (prior, likelihood, posterior, prediction, model checking) and the GAISE College Report (ASA 2016) emphasis on simulation; based on the works below. All wording and examples are original. - Bayes, T. (1763). An essay towards solving a problem in the doctrine of chances. *Philosophical Transactions of the Royal Society of London*, 53, 370–418. - Jeffreys, H. (1961). *Theory of Probability* (3rd ed.). Clarendon Press. - Kass, R. E., and Raftery, A. E. (1995). Bayes factors. *Journal of the American Statistical Association*, 90(430), 773–795. - Lindley, D. V. (1957). A statistical paradox. *Biometrika*, 44(1–2), 187–192. - Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., and Rubin, D. B. (2013). *Bayesian Data Analysis* (3rd ed.). CRC Press. - McElreath, R. (2020). *Statistical Rethinking: A Bayesian Course with Examples in R and Stan* (2nd ed.). CRC Press. - Kruschke, J. K. (2015). *Doing Bayesian Data Analysis: A Tutorial with R, JAGS, and Stan* (2nd ed.). Academic Press. - GAISE College Report ASA Revision Committee (2016). *Guidelines for Assessment and Instruction in Statistics Education: College Report 2016*. American Statistical Association. - Software next steps: Stan (mc-stan.org), brms, PyMC. - Further reading (by name only): de Smith, M. J. (2024). *Statistical Analysis Handbook* (2024 edition, StatsRef); *OpenIntro Statistics* (CC BY-SA). ## Answer key **Example B.** 1. Prior precision 0.25; data precision 1.92. 2. Posterior precision 2.17; posterior SD = 1 / √2.17 = 0.679 g. 3. Weight on data 0.885, so the posterior mean sits about 88% of the way from 500 to 497.55. 4. Posterior mean 497.83 g; 95% credible interval [496.50, 499.16] g. 5. 0.9993. *Example:* "Given what we believed before and today's weighings, it is almost certain (over 99.9%) that this batch averages below the 500 g label." 6. Narrower, because σ is treated as known and the prior adds information; the centre is closer to 500 because the prior, centred on 500, pulls the estimate towards it (shrinkage). **Exercise C.** 1. Beta(1 + 3, 1 + 12) = Beta(4, 13). 2. 4 / 17 = 0.235. 3. [0.073, 0.457] (R `qbeta(c(0.025, 0.975), 4, 13)`). 4. `pbeta(0.3, 4, 13)` = 0.754. 5. Yes for the credible interval (given the flat prior and the model); no for a confidence interval, whose 95% refers to the procedure. 6. Beta(5 + 3, 20 + 12) = Beta(8, 32); posterior mean 8 / 40 = 0.200, pulled towards last year's rate of 0.2. **Self-test.** 1. $p(\theta \mid y) = p(y \mid \theta)\,p(\theta) / p(y)$: posterior, likelihood, prior, marginal likelihood. 2. Beta(12, 8); mean 12 / 20 = 0.60. 3. It equals a weighted average of a / (a + b) and x / n with weights proportional to a + b and n. 4. Large n: the data precision dominates and the posterior concentrates around $\bar{y}$ with SD near σ / √n. Large τ0: the prior precision tends to 0, so the posterior approaches N($\bar{y}$, σ²/n). 5. "Given the prior and model, the parameter lies in this interval with probability 0.95." 6. It depends strongly on the prior under H1 (very vague priors favour H0), and it gives posterior odds only after multiplying by prior odds; verbal labels are arbitrary. ## Revision history | Version | Date | Author | Change summary | |---|---|---|---| | 0.1 | YYYY-MM-DD | [Your name] | First draft from template | | 0.2 | YYYY-MM-DD | [Your name] | *Example:* Added my thesis prior and its sensitivity table |