Probability Basics Concept Card Template (Statistics, R and Python)

Probability Basics: [Your course or study question]

A concept card for the probability you need before any statistics course makes sense: sample spaces and events, the three axioms, the addition and multiplication rules, conditional probability, independence, the law of total probability, Bayes' theorem, counting with permutations and combinations, random variables, and the rules for expectation and variance. It is for students preparing for a course or exam, thesis writers who need to reason about uncertainty, and analysts who want a clean refresher. It speeds up learning by putting the plain-language idea first, a picture second and the formula third, then making you practise with worked examples that fade from fully solved to unsolved, a seeded simulation you can run in R and Python, and self-test questions with a separate answer key. The worked examples use fictional data; replace them with your own. Source basis: standard probability theory (Kolmogorov's axioms, Bayes' theorem), the NIST/SEMATECH e-Handbook of Statistical Methods, and the learning-science literature on retrieval practice and dual coding.

Document control

The idea in one paragraph

Probability is a number between 0 and 1 that measures how often something tends to happen when a situation is repeated, or how strongly the available information supports it. Every probability is defined relative to a sample space, the list of everything that could happen. An event is a set of those outcomes. The rules of probability are only a handful: probabilities cannot be negative, the whole sample space has probability 1, and the probabilities of events that cannot happen together add up. Everything else, including conditional probability and Bayes' theorem, follows from these rules plus one definition: the probability of A given B is the share of B's probability that also belongs to A. Most errors in practice come from mixing up which group you are taking a share of, for example confusing "the share of sick people who test positive" with "the share of positive testers who are sick". The cure is to count in a concrete population of, say, 1,000 people.

Intuition

Think of probability as counting people in a room. Suppose the Lakemoor University health service (fictional) screens 1,000 students for a condition. Two percent of students have it, the test detects 90% of true cases (sensitivity), and it correctly clears 95% of healthy students (specificity). Instead of juggling percentages, split the 1,000 students into groups. This is called a natural-frequency tree, and research by Gigerenzer and Hoffrage (1995) found that people solve Bayes-type problems far more accurately in this format than with percentages.

Read the tree from the top. Of the 67 students who test positive, only 18 actually have the condition, because the 980 healthy students are so numerous that even a 5% false-positive rate produces 49 false alarms. The probability of the condition given a positive test is about 18/67 = 0.27, not 0.90. The 0.90 is the probability of a positive test given the condition: a different share of a different group. This single picture holds conditional probability, the multiplication rule (each branch multiplies), the law of total probability (the 67 positives come from two branches) and Bayes' theorem (18 out of 67).

A second analogy helps with independence: two events are independent when learning that one happened does not change your counting for the other. Whether a fair coin landed heads on the last toss does not change the share of heads on the next toss. Contrast this with drawing two students from a class of 10 without putting the first one back: the second draw depends on the first, because the room has changed.

Formal definitions and rules

Structure aligned with standard introductory probability as presented in the NIST/SEMATECH e-Handbook and the GAISE College Report recommendations (focus on conceptual understanding; use simulation to illustrate abstract concepts); all wording and examples here are original.

Sample spaces, events and axioms

The sample space $S$ is the set of all possible outcomes of a random process. For one roll of a die, $S = \{1, 2, 3, 4, 5, 6\}$. An event is any subset of $S$, such as "an even number" $= \{2, 4, 6\}$. The complement $A^c$ is everything in $S$ not in $A$. Two events are mutually exclusive (disjoint) when they share no outcomes.

Kolmogorov's three axioms (1933) define a probability $P$:

Consequences you will use constantly: $P(A^c) = 1 - P(A)$, $P(\varnothing) = 0$, and $0 \le P(A) \le 1$. When all outcomes are equally likely, $P(A)$ is the number of outcomes in $A$ divided by the number in $S$.

Addition rule

In words: the probability that A or B happens is the sum of their probabilities, minus the overlap that was counted twice.

If A and B are mutually exclusive, the overlap is zero and the probabilities simply add.

Conditional probability and the multiplication rule

In words: the probability of A given B is the share of B's probability that also lies in A. You restrict the sample space to B and ask how much of it is A.

Rearranging gives the multiplication rule: the probability that both happen is the probability of the first times the probability of the second given the first.

Independence

In words: A and B are independent when knowing that B happened does not change the probability of A.

Independence is a property of the probabilities, not of the physical situation, and you usually assume it from the design (separate coin tosses, randomly sampled different people) rather than prove it. Mutually exclusive events with positive probabilities are never independent: if one happens, the other cannot, so knowing one changes the other's probability to zero.

Law of total probability

In words: if the population splits into non-overlapping groups $B_1, \dots, B_k$ that cover everything, the overall probability of A is a weighted average of A's probability within each group, weighted by group size.

In the tree, the 67 positives are $0.90 \times 0.02 + 0.05 \times 0.98 = 0.067$ of all students.

Bayes' theorem

In words: to turn $P(\text{evidence} \mid \text{cause})$ into $P(\text{cause} \mid \text{evidence})$, take the branch that has both, and divide by all branches that produce the evidence.

The term $P(B_j)$ is the prior or base rate. When the base rate is small, even an accurate test produces mostly false positives. Bayes' theorem is named after Thomas Bayes, whose essay was published posthumously in 1763; it is the foundation of Bayesian inference (see academic/stats-bayesian-inference).

Counting

When outcomes are equally likely, probability becomes counting. Three tools cover most problems:

• Multiplication principle: if a task has $k$ steps with $n_1, n_2, \dots, n_k$ choices, there are $n_1 n_2 \cdots n_k$ ways in total.
• Permutations (order matters): the number of ordered arrangements of $r$ items chosen from $n$ is $P(n, r) = \dfrac{n!}{(n-r)!}$.
• Combinations (order does not matter): the number of unordered selections of $r$ items from $n$ is $\dbinom{n}{r} = \dfrac{n!}{r!\,(n-r)!}$.

Ask one question to choose: "Would swapping two chosen items give a different outcome?" If yes (a president and a secretary), use permutations; if no (a committee of three), use combinations. For example, choosing 3 of 10 students gives $\binom{10}{3} = 120$ committees but $P(10, 3) = 720$ ordered officer slates.

Random variables, expectation and variance

A random variable $X$ is a rule that attaches a number to each outcome, such as "the number shown by the die" or "the number of positive tests in a sample of 20". A discrete random variable has a probability mass function $p(x) = P(X = x)$; a continuous one has a density $f(x)$ whose area gives probabilities (see academic/stats-distributions-sheet).

The expectation (mean) is the probability-weighted average, the long-run average over many repetitions. The variance is the expected squared distance from the mean.

Rules for constants $a, b$ and random variables $X, Y$:

Note the plus sign for $X - Y$: subtracting an independent random quantity still adds uncertainty. Adding a constant shifts the mean but not the spread. For a fair die, $E(X) = 3.5$ and $\operatorname{Var}(X) = 35/12 \approx 2.9167$.

Notation table

Worked example A (fully solved): a screening test

Question. The Lakemoor University health service (fictional) offers a screening test for a condition that affects 2% of students. The test is positive for 90% of students with the condition and negative for 95% of students without it. A student tests positive. What is the probability that the student has the condition?

Step 1: define events. Let $D$ = "has the condition" and $+$ = "tests positive".

Step 2: list what is given.

Step 3: total probability of a positive test.

Step 4: Bayes' theorem.

Step 5: check with natural frequencies. In 1,000 students: 20 have the condition, 18 of them test positive; 980 do not, 49 of them test positive. That gives 18 of 67 positives, 0.2687, matching the formula.

Step 6: interpret. A positive result raises the probability from 2% to about 27%, a big increase, but most positives are still false alarms. That is why a positive screen is normally followed by a more specific confirmatory test. If the same test were used where 20% of students had the condition, the answer would change completely (see self-test question 3): the test did not change, only the base rate did.

Check by simulation. The code in the simulation section reproduces this: the formula gives P(D | +) = 0.2687, and 100,000 simulated students give 0.2720 (R) and 0.2728 (Python). The small gap is simulation noise, because only about 6,700 simulated students test positive.

Worked example B (half solved): two ovens at Brightwell Bakery

Question. Brightwell Bakery (fictional) bakes loaves in two ovens. Oven 1 bakes 60% of the loaves and 3% of its loaves are underbaked. Oven 2 bakes 40% and 5% of its loaves are underbaked. A loaf picked at random is underbaked. What is the probability that it came from oven 2?

Step 1: define events. $O_1$ = "from oven 1", $O_2$ = "from oven 2", $U$ = "underbaked".

Step 2: list what is given. $P(O_1) = 0.60$, $P(O_2) = [\ ]$, $P(U \mid O_1) = 0.03$, $P(U \mid O_2) = [\ ]$.

Step 3: total probability.

Step 4: Bayes' theorem. $P(O_2 \mid U) = \dfrac{[\ \ ]}{[\ \ ]} \approx [\ \ ]$.

Step 5: natural-frequency check. Out of 1,000 loaves, [ ] come from oven 1 and [ ] of those are underbaked; [ ] come from oven 2 and [ ] of those are underbaked.

Step 6: interpret. Oven 2 bakes fewer than half of the loaves but accounts for [ ] of underbaked ones, because [ ].

Exercise C (unsolved): a student committee

The Lakemoor University statistics society (fictional) has 12 members, including two friends, Ana and Ben. A committee of 4 is chosen completely at random.

1. How many different committees are possible?
2. What is the probability that both Ana and Ben are on the committee?
3. If the committee instead elects a chair, a secretary and a treasurer (three distinct roles) from all 12 members, how many different slates are possible?
4. Are the events "Ana is on the committee" and "Ben is on the committee" independent? Give a one-line reason using numbers.

Simulation you can run

The scripts check four results from this card by brute force: Bayes' theorem for the screening test, the counting formulas, the gambler's fallacy (what happens after four heads in a row), and the mean and variance of a fair die. Simulation is the empirical twin of the formulas: if the two disagree by more than noise, one of them is wrong.

R (base R only)

# Screening test at the Lakemoor University health service (fictional)
prev <- 0.02; sens <- 0.90; spec <- 0.95
p_pos <- sens * prev + (1 - spec) * (1 - prev)
ppv <- sens * prev / p_pos
cat(sprintf("P(+) = %.4f, P(D | +) = %.4f\n", p_pos, ppv))

set.seed(2026)
n <- 100000
disease <- rbinom(n, 1, prev)
test_pos <- ifelse(disease == 1, rbinom(n, 1, sens), rbinom(n, 1, 1 - spec))
cat(sprintf("Simulated P(+) = %.4f, P(D | +) = %.4f\n",
            mean(test_pos), mean(disease[test_pos == 1])))

# Counting
cat("choose(10, 3) =", choose(10, 3), "; perms 10P3 =", factorial(10) / factorial(7), "\n")

# Gambler's fallacy: coin flips, next flip after 4 heads in a row
set.seed(2026)
flips <- rbinom(200000, 1, 0.5)
idx <- which(sapply(5:length(flips), function(i) all(flips[(i-4):(i-1)] == 1))) + 4
cat(sprintf("Runs of 4 heads: %d, P(next is heads) = %.4f\n", length(idx), mean(flips[idx])))

# Fair die: expectation and variance
x <- 1:6
EX <- sum(x / 6); VX <- sum((x - EX)^2 / 6)
cat(sprintf("E(X) = %.4f, Var(X) = %.4f\n", EX, VX))
set.seed(2026)
rolls <- sample(1:6, 100000, replace = TRUE)
cat(sprintf("Simulated mean = %.4f, variance = %.4f\n", mean(rolls), var(rolls)))

P(+) = 0.0670, P(D | +) = 0.2687
Simulated P(+) = 0.0667, P(D | +) = 0.2720
choose(10, 3) = 120 ; perms 10P3 = 720
Runs of 4 heads: 12602, P(next is heads) = 0.5060
E(X) = 3.5000, Var(X) = 2.9167
Simulated mean = 3.4978, variance = 2.9233

Python (numpy)

import numpy as np
from math import comb, perm

prev, sens, spec = 0.02, 0.90, 0.95
p_pos = sens * prev + (1 - spec) * (1 - prev)
ppv = sens * prev / p_pos
print(f"P(+) = {p_pos:.4f}, P(D | +) = {ppv:.4f}")

rng = np.random.default_rng(2026)
n = 100_000
disease = rng.random(n) < prev
test_pos = np.where(disease, rng.random(n) < sens, rng.random(n) < 1 - spec)
print(f"Simulated P(+) = {test_pos.mean():.4f}, P(D | +) = {disease[test_pos].mean():.4f}")

print("comb(10, 3) =", comb(10, 3), "; perm(10, 3) =", perm(10, 3))

rng = np.random.default_rng(2026)
flips = rng.integers(0, 2, 200_000)
w = np.lib.stride_tricks.sliding_window_view(flips[:-1], 4)
after = flips[4:][w.sum(axis=1) == 4]
print(f"Runs of 4 heads: {after.size}, P(next is heads) = {after.mean():.4f}")

x = np.arange(1, 7)
EX = x.mean(); VX = ((x - EX) ** 2).mean()
print(f"E(X) = {EX:.4f}, Var(X) = {VX:.4f}")
rng = np.random.default_rng(2026)
rolls = rng.integers(1, 7, 100_000)
print(f"Simulated mean = {rolls.mean():.4f}, variance = {rolls.var(ddof=1):.4f}")

P(+) = 0.0670, P(D | +) = 0.2687
Simulated P(+) = 0.0669, P(D | +) = 0.2728
comb(10, 3) = 120 ; perm(10, 3) = 720
Runs of 4 heads: 12520, P(next is heads) = 0.5023
E(X) = 3.5000, Var(X) = 2.9167
Simulated mean = 3.4941, variance = 2.9181

What the output teaches.

• The exact answers (0.0670, 0.2687, 120, 720, 3.5, 2.9167) are identical in both languages because they come from formulas. The simulated values differ slightly between R and Python because the two languages use different random number generators; each is reproducible on its own with the seed 2026.
• After four heads in a row, the next toss was heads 50.6% (R) and 50.2% (Python) of the time: the coin has no memory. Believing a tail is "due" is the gambler's fallacy.
• The simulated die variance (2.9233 and 2.9181) sits close to the exact 35/12. Notice that the sample variance uses the $n - 1$ divisor (var() in R, ddof=1 in numpy), while the exact population variance divides by 6 because it averages over the probability distribution, not a sample.

Misconceptions

Connections to other concepts

Self-test questions

1. Events A and B are independent with P(A) = 0.3 and P(B) = 0.5. Find P(A or B).
2. Explain in two sentences why two mutually exclusive events, each with positive probability, cannot be independent.
3. Use the screening test from Example A (sensitivity 0.90, specificity 0.95) in a clinic where 20% of patients have the condition. What is P(D | +)? What changed, and what did not?
4. X has mean 10 and variance 4. Find E(2X + 3) and Var(2X + 3), and the standard deviation of 2X + 3.
5. In how many ways can 5 different library books be arranged on a shelf? In how many ways can 2 of them be chosen to lend to a friend?
6. Two fair dice are rolled. Let T be the total. Find E(T) and Var(T), using the die results from the simulation section.

My notes and next steps

Open questions for my tutor or study group: [List them here].

Mistakes to copy into my error log (academic/stats-error-log): [Mistake, why it happened, the fix].

Sources and further reading

Based on standard probability theory written in original wording, with the learning design following Dunlosky et al. (2013) and Weinstein and Sumeracki (2018).

• Kolmogorov, A. N. (1933). Grundbegriffe der Wahrscheinlichkeitsrechnung. Springer. (The axioms.)
• Bayes, T. (1763). An essay towards solving a problem in the doctrine of chances. Philosophical Transactions of the Royal Society of London, 53, 370–418.
• Gigerenzer, G., and Hoffrage, U. (1995). How to improve Bayesian reasoning without instruction: frequency formats. Psychological Review, 102(4), 684–704.
• Tversky, A., and Kahneman, D. (1974). Judgment under uncertainty: heuristics and biases. Science, 185(4157), 1124–1131.
• NIST/SEMATECH e-Handbook of Statistical Methods, section 1.3.6 Probability Distributions, https://www.itl.nist.gov/div898/handbook/ (accessed 2026-10-25). Public domain.
• GAISE College Report ASA Revision Committee (2016). Guidelines for Assessment and Instruction in Statistics Education College Report 2016. American Statistical Association.
• Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., and Willingham, D. T. (2013). Improving students' learning with effective learning techniques. Psychological Science in the Public Interest, 14(1), 4–58.
• Weinstein, Y., and Sumeracki, M. (2018). Understanding How We Learn: A Visual Guide (illustrated by O. Caviglioli). Routledge.
• Further reading (named only): Diez, Çetinkaya-Rundel and Barr, OpenIntro Statistics (CC BY-SA), chapter on probability; de Smith, M. J., Statistical Analysis Handbook (2024 edition, statsref.com).

Answer key

Example B (gaps). $P(O_2) = 0.40$ and $P(U \mid O_2) = 0.05$. $P(U) = 0.018 + 0.020 = 0.038$. $P(O_2 \mid U) = 0.020 / 0.038 \approx 0.5263$. Natural frequencies: 600 loaves from oven 1, 18 underbaked; 400 from oven 2, 20 underbaked; so 20 of 38 underbaked loaves (about 53%) come from oven 2. Oven 2 bakes 40% of the loaves but its defect rate is higher (5% vs 3%), which more than offsets its smaller share.

Exercise C.

1. Order does not matter, so $\binom{12}{4} = 495$ committees.
2. Fix Ana and Ben, then choose the other 2 from the remaining 10: $\binom{10}{2} = 45$. So $P = 45/495 \approx 0.0909$.
3. Order matters (distinct roles): $P(12, 3) = 12 \times 11 \times 10 = 1{,}320$ slates.
4. Not independent. $P(\text{Ana}) = P(\text{Ben}) = 4/12 = 1/3$, so independence would require $P(\text{both}) = 1/9 \approx 0.111$, but the actual value is 0.0909. Once Ana takes a seat, only 3 seats remain for the 11 others, so Ben's chance drops to 3/11.

Self-test.

1. $P(A \cup B) = 0.3 + 0.5 - 0.3 \times 0.5 = 0.65$.
2. If A happens, B cannot, so $P(B \mid A) = 0$, which differs from $P(B) > 0$. Knowing A changes the probability of B, which is the definition of dependence.
3. $P(+) = 0.90(0.20) + 0.05(0.80) = 0.18 + 0.04 = 0.22$, so $P(D \mid +) = 0.18/0.22 \approx 0.818$. The test's sensitivity and specificity did not change; only the base rate did, and it moved the answer from 0.27 to 0.82.
4. $E(2X + 3) = 2(10) + 3 = 23$; $\operatorname{Var}(2X + 3) = 4 \times 4 = 16$; SD = 4. The constant 3 shifts the mean but does not change the spread.
5. Arrangements: $5! = 120$. Choosing 2 to lend (order does not matter): $\binom{5}{2} = 10$.
6. Each die has mean 3.5 and variance 35/12. The dice are independent, so $E(T) = 7$ and $\operatorname{Var}(T) = 2 \times 35/12 = 35/6 \approx 5.8333$.

Revision history