# AI Decision-Making Concept Card: [Your course, case or decision] > A concept card for the manager who has to decide whether to trust an AI system's output and how to put people and > models together: human-in-the-loop designs, calibration, automation bias and algorithm aversion, the evaluation > metrics a manager must read (accuracy, precision, recall, specificity, F1, cost-weighted thresholds) with a computed > confusion matrix, A/B testing of AI features, and when not to use AI. It is for MBA students in an AI for managers > or decision analytics course and for managers who sign off on AI-supported decisions. Worked example A is fully > solved, Worked example B is half solved and Exercise C is left for you, with a separate answer key. Source basis: > Parasuraman and Riley (1997), Skitka, Mosier and Burdick (1999), Dietvorst, Simmons and Massey (2015), Brier (1950), > Fawcett (2006), Kohavi, Tang and Xu (2020) and Dell'Acqua et al. (2026), written in original words with fictional > examples (Northfield Outdoor Co., Tallow Creek Medical Center, Corvane Robotics). Replace every [bracketed] value and > every row marked *Example*. This is a learning aid, not legal, medical or investment advice. ## Document control | Field | Value | |---|---| | Course, case or decision | [AI for managers, session 6 / approval of fraud model v2 for returns] | | AI system and version | [Vendor or internal model, version, date] | | Learner or analyst | [Your name] | | Evaluation data | [Holdout period, size, how labels were obtained] | | Spreadsheet used | [Excel / Google Sheets / LibreOffice Calc] | | Reviewed by | [Study partner, tutor or data science lead] | | Version | [0.1] | | Last updated | YYYY-MM-DD | ## Contents - [When to use it](#when-to-use-it) - [The idea in plain language](#the-idea-in-plain-language) - [Formulas and notation](#formulas-and-notation) - [Assumptions and limits](#assumptions-and-limits) - [How managers use it](#how-managers-use-it) - [Worked example A (fully solved): Northfield returns-fraud model](#worked-example-a-fully-solved-northfield-returns-fraud-model) - [Calibration in practice](#calibration-in-practice) - [A/B testing an AI feature](#ab-testing-an-ai-feature) - [Worked example B (half solved): Tallow Creek readmission model](#worked-example-b-half-solved-tallow-creek-readmission-model) - [Exercise C (unsolved): Corvane failure alerts](#exercise-c-unsolved-corvane-failure-alerts) - [When not to use AI](#when-not-to-use-ai) - [Decision diagram](#decision-diagram) - [Connections to other cards](#connections-to-other-cards) - [Common mistakes and misconceptions](#common-mistakes-and-misconceptions) - [Self-test questions](#self-test-questions) - [My notes and next steps](#my-notes-and-next-steps) - [Sources and further reading](#sources-and-further-reading) - [Answer key](#answer-key) - [Revision history](#revision-history) - [Standard and licence note](#standard-and-licence-note) ## When to use it **One-sentence rule:** judge an AI system by the cost of its errors in your decision, not by its headline accuracy, and design the human role so that people catch the errors that matter without rubber-stamping or ignoring the system. | Situation | Use this section | |---|---| | A vendor reports "95% accuracy" | Worked example A, formulas | | You must set the threshold at which the model flags a case | Worked example A, cost table | | Staff say they "just click accept" | The idea in plain language (automation bias) | | A model gives a probability and people act on it | Calibration in practice | | A team wants to launch a generative AI feature | A/B testing an AI feature | | A use case affects people's rights, health or livelihood | When not to use AI; mba/mba-ai-risk-governance-card | ## The idea in plain language **Human in the loop, on the loop, out of the loop.** In a *human-in-the-loop* design, a person reviews or approves each output before it takes effect (an agent approves a suggested reply; an investigator reviews each flagged return). In a *human-on-the-loop* design the system acts and a person monitors and can intervene (an automated pricing rule with alerts). In a *human-out-of-the-loop* design the system acts alone within set limits (spam filtering). The right design depends on the cost of an error, how quickly it can be undone, the volume of decisions, and whether a person can actually judge the output better than the system. A human in the loop who cannot tell a good output from a bad one adds cost and false comfort, not safety. **Automation bias and algorithm aversion.** People misuse automation in two opposite ways. *Automation bias* is over-reliance: accepting the system's advice without checking, and missing problems the system did not flag (Parasuraman and Riley 1997 called this misuse; Skitka, Mosier and Burdick 1999 showed it experimentally). It grows with workload, time pressure and a long run of correct outputs. *Algorithm aversion* is the opposite: Dietvorst, Simmons and Massey (2015) found that people lose confidence in an algorithm faster than in a human after seeing it make a mistake, even when the algorithm is more accurate overall. Later work (Logg, Minson and Moore 2019) found that lay people often prefer algorithmic advice for some judgments, so neither bias is universal. Managers have to design for both: make checking easy and expected, show uncertainty, audit samples of accepted outputs, and let people adjust the system's output within limits. Dell'Acqua et al. (2026) add a sharper warning for generative AI: on tasks just outside the model's competence, people using AI were less likely to reach the correct answer than people without it, and fluent output can make such errors harder to spot. **Calibration.** A model is *calibrated* when its probabilities mean what they say: of all cases scored 0.30, about 30% turn out positive. Many models rank cases well but are poorly calibrated, so a score is not a probability until someone has checked. Calibration matters whenever people or systems act on the number itself: pricing risk, setting reserves, deciding whether the expected cost of an action is worth it. The Brier (1950) score measures the accuracy of probability forecasts; lower is better. **The confusion matrix.** For a yes/no decision, every case lands in one of four cells: true positive (flagged and really positive), false positive (flagged but negative), false negative (missed), true negative (correctly left alone). Every metric is a ratio of these cells. When positives are rare, accuracy is dominated by the negatives and says almost nothing; *precision* (how many flags are right) and *recall* (how many positives are caught) say what a manager needs to know. The threshold at which a score becomes a flag trades one against the other, and the right threshold depends on the cost of each error, which is a business judgment, not a data science one. **Evaluating generative AI.** For generated content there is no single right answer per case, so evaluation uses rubrics graded by people or by a second model, task success rates, error categories (factual errors, policy errors, unsafe content) and user outcomes in experiments. The same managerial logic applies: define the errors that matter, measure them on realistic cases, and test the effect on outcomes with a controlled experiment (see ai-data/llm-evaluation-plan). ## Formulas and notation With $TP$, $FP$, $FN$, $TN$ the four cells and $N$ the total: $$\text{Accuracy} = \frac{TP + TN}{N}, \qquad \text{Precision} = \frac{TP}{TP + FP}, \qquad \text{Recall} = \frac{TP}{TP + FN}$$ $$\text{Specificity} = \frac{TN}{TN + FP}, \qquad F_1 = \frac{2 \cdot \text{Precision} \cdot \text{Recall}}{\text{Precision} + \text{Recall}}$$ **Expected cost at a threshold** $\tau$, with review cost $c_r$ per flag, extra cost $c_{fp}$ per false flag and loss $c_{fn}$ per missed positive: $$C(\tau) = c_r \,[TP(\tau) + FP(\tau)] + c_{fp}\, FP(\tau) + c_{fn}\, FN(\tau)$$ **Brier score** for $n$ probability forecasts $f_i$ of outcomes $o_i \in \{0, 1\}$: $$BS = \frac{1}{n}\sum_{i=1}^{n} (f_i - o_i)^2$$ **Expected calibration error** over bins $b$ with $n_b$ cases, mean predicted probability $\bar{p}_b$ and observed rate $\bar{o}_b$: $ECE = \sum_b \frac{n_b}{N} \lvert \bar{p}_b - \bar{o}_b \rvert$. **Two-proportion z test** for an A/B test with conversion rates $\hat{p}_1$, $\hat{p}_2$ and pooled rate $\hat{p}$: $$z = \frac{\hat{p}_2 - \hat{p}_1}{\sqrt{\hat{p}(1-\hat{p})\left(\frac{1}{n_1} + \frac{1}{n_2}\right)}}$$ **Sample size per arm** to detect $p_1$ against $p_2$ at significance $\alpha$ (two-sided) and power $1 - \beta$, with $\bar{p} = (p_1 + p_2)/2$: $$n = \frac{\left(z_{1-\alpha/2}\sqrt{2\bar{p}(1-\bar{p})} + z_{1-\beta}\sqrt{p_1(1-p_1) + p_2(1-p_2)}\right)^2}{(p_2 - p_1)^2}$$ | Symbol | Meaning | |---|---| | $TP, FP, FN, TN$ | True positives, false positives, false negatives, true negatives | | $\tau$ | Threshold above which a score becomes a flag | | $c_r, c_{fp}, c_{fn}$ | Review cost per flag, extra cost per false flag, loss per missed positive | | $f_i, o_i$ | Forecast probability and actual outcome (1 or 0) | | $\hat{p}_1, \hat{p}_2$ | Observed conversion rates in control and treatment | | $z_{1-\alpha/2}, z_{1-\beta}$ | Standard normal quantiles (1.96 and 0.8416 for 5% and 80% power) | ## Assumptions and limits - **Labels are correct.** Fraud labels come from investigations, readmission labels from records; if labels are biased or incomplete, every metric inherits the bias. - **The test data look like production.** Metrics on last year's data may not hold after a policy change, a new product range or a change in customer behaviour (drift). - **Costs are known.** The cost-weighted threshold is only as good as the cost estimates; test a range. - **Metrics hide subgroups.** A model with good overall recall can perform poorly for one region, language or customer group; break metrics down. - **People change behaviour.** Once a model is deployed, staff, customers and fraudsters adapt, so the measured performance changes. - **An A/B test measures the average effect** over the test population and period; novelty effects and spill-overs between groups can bias it. ## How managers use it A manager does not build the model but owns the decision. In practice that means asking six questions before approval: (1) What decision does the output feed, and who acts on it? (2) What does each type of error cost, and who bears it (the firm, a customer, an employee)? (3) What are precision and recall at the proposed threshold, overall and by subgroup, on recent data? (4) Are the scores calibrated if anyone reads them as probabilities? (5) How will people review outputs, how will we know whether they are rubber-stamping, and can they override? (6) How will we test the effect on business outcomes (an experiment, not a before-and-after comparison), and what are the stop rules? The answers go into the evaluation record (ai-data/model-validation-report) and the oversight design (ai-data/human-oversight-design). ## Worked example A (fully solved): Northfield returns-fraud model **Situation (fictional).** Northfield Outdoor Co. processes 20,000 returns a quarter, of which 600 (3%) turn out to be fraudulent (worn items, swapped products). A model scores each return from 0 to 1. Every flagged return goes to an investigator, which costs USD 12; a false flag also costs about USD 8 in customer goodwill (delay, an awkward conversation); a missed fraud loses USD 120 on average. Without the model Northfield checks nothing and loses $600 \times 120 =$ USD 72,000 a quarter. **Step 1. Confusion matrix at threshold 0.5** (holdout quarter, labels from completed investigations): | *Example* | Actually fraud | Actually legitimate | Total | |---|---|---|---| | Flagged | TP = 488 | FP = 303 | 791 | | Not flagged | FN = 112 | TN = 19,097 | 19,209 | | Total | 600 | 19,400 | 20,000 | **Step 2. Metrics.** Put TP, FP, FN, TN in B2:B5. | *Example* metric | Result | Excel formula | Reading | |---|---|---|---| | Accuracy | 0.9793 | `=(B2+B5)/SUM(B2:B5)` | Only 0.9 points above "never flag" (0.9700) | | Precision | 0.6169 | `=B2/(B2+B3)` | About 62% of flags are real fraud | | Recall | 0.8133 | `=B2/(B2+B4)` | The model catches 81% of fraud | | Specificity | 0.9844 | `=B5/(B5+B3)` | 1.6% of honest customers are flagged | | F1 | 0.7017 | `=2*B7*B8/(B7+B8)` (precision in B7, recall in B8) | Balance of precision and recall | | Quarterly cost | USD 25,356 | `=12*(B2+B3) + 8*B3 + 120*B4` | Against USD 72,000 without the model | **Step 3. Choose the threshold on cost, not on accuracy.** | *Example* threshold | TP | FP | FN | Precision | Recall | Accuracy | Cost (USD) | |---|---|---|---|---|---|---|---| | 0.1 | 600 | 10,310 | 0 | 0.0550 | 1.0000 | 0.4845 | 213,400 | | 0.2 | 596 | 5,086 | 4 | 0.1049 | 0.9933 | 0.7455 | 109,352 | | 0.3 | 582 | 2,282 | 18 | 0.2032 | 0.9700 | 0.8850 | 54,784 | | 0.4 | 548 | 905 | 52 | 0.3772 | 0.9133 | 0.9522 | 30,916 | | 0.5 | 488 | 303 | 112 | 0.6169 | 0.8133 | 0.9793 | 25,356 | | 0.6 | 398 | 79 | 202 | 0.8344 | 0.6633 | 0.9860 | 30,596 | | 0.7 | 283 | 14 | 317 | 0.9529 | 0.4717 | 0.9835 | 41,716 | | 0.8 | 158 | 1 | 442 | 0.9937 | 0.2633 | 0.9779 | 54,956 | | 0.9 | 49 | 0 | 551 | 1.0000 | 0.0817 | 0.9725 | 66,708 | The cost is lowest at 0.5 (USD 25,356); accuracy is highest at 0.6. The two criteria point to different thresholds, and only the cost criterion reflects Northfield's economics. Note also that at 0.9 the model is 97.25% accurate and almost useless: it catches 49 of 600 frauds. ```vega-lite { "$schema": "https://vega.github.io/schema/vega-lite/v5.json", "description": "Precision and recall by threshold, Northfield returns-fraud model (fictional data)", "width": 420, "height": 220, "data": {"values": [ {"t": 0.1, "precision": 0.055, "recall": 1.0}, {"t": 0.2, "precision": 0.105, "recall": 0.993}, {"t": 0.3, "precision": 0.203, "recall": 0.97}, {"t": 0.4, "precision": 0.377, "recall": 0.913}, {"t": 0.5, "precision": 0.617, "recall": 0.813}, {"t": 0.6, "precision": 0.834, "recall": 0.663}, {"t": 0.7, "precision": 0.953, "recall": 0.472}, {"t": 0.8, "precision": 0.994, "recall": 0.263}, {"t": 0.9, "precision": 1.0, "recall": 0.082} ]}, "transform": [{"fold": ["precision", "recall"], "as": ["metric", "value"]}], "mark": {"type": "line", "point": true}, "encoding": { "x": {"field": "t", "type": "quantitative", "title": "Threshold"}, "y": {"field": "value", "type": "quantitative", "title": "Value", "scale": {"domain": [0, 1]}}, "color": {"field": "metric", "type": "nominal", "title": "Metric"} } } ``` **Step 4. Human review design.** Investigators see 791 flags a quarter. To limit automation bias, Northfield shows the main reasons for each score, asks investigators to record their own judgment before seeing the score on a random 10% of cases, and audits a sample of unflagged returns each month to estimate the false negatives in production. **Conclusion.** Set the threshold at 0.5, which cuts the expected quarterly cost from USD 72,000 to USD 25,356 while flagging 1.6% of honest customers. Re-run the cost table when costs or fraud patterns change. ## Calibration in practice On a second holdout month of 20,000 returns, Northfield grouped scores into five bins and compared the mean score with the observed fraud rate. | *Example* mean score in bin | Returns | Frauds | Observed rate | Gap (score minus observed) | |---|---|---|---|---| | 0.02 | 15,000 | 90 | 0.0060 | +0.0140 | | 0.08 | 3,000 | 90 | 0.0300 | +0.0500 | | 0.25 | 1,200 | 120 | 0.1000 | +0.1500 | | 0.55 | 550 | 165 | 0.3000 | +0.2500 | | 0.85 | 250 | 135 | 0.5400 | +0.3100 | | Total | 20,000 | 600 | 0.0300 | | The model is overconfident: summing score times returns it expects 1,355 frauds, but 600 occurred. Expected calibration error is $ECE = 0.0377$ (`=SUMPRODUCT(B2:B6, ABS(A2:A6 - D2:D6))/SUM(B2:B6)`). It still ranks returns well, so flags based on a threshold work, but anyone who reads a score of 0.85 as "85% likely to be fraud" is wrong: the observed rate in that bin is 54%. Ask the team to recalibrate (for example with Platt scaling or isotonic regression) before scores are shown to investigators as probabilities. **Brier score illustration.** Ten forecasts 0.9, 0.8, 0.7, 0.6, 0.3, 0.2, 0.2, 0.1, 0.1, 0.05 for outcomes 1, 1, 0, 1, 0, 0, 1, 0, 0, 0 give $BS = 0.14925$ (`=SUMXMY2(A2:A11, B2:B11)/10`). Forecasting the base rate 0.40 for every case gives 0.2400, so the forecasts add information. ## A/B testing an AI feature **Situation (fictional).** Northfield tests AI-generated product descriptions. Visitors are randomised: 40,000 see the existing copy (control), 40,000 the AI-drafted, human-edited copy (treatment). Conversions: 1,240 control (3.10%), 1,336 treatment (3.34%). | *Example* step | Result | Excel formula | |---|---|---| | Difference | 0.24 points (7.74% relative) | `=x2/n2 - x1/n1` | | Pooled rate $\hat{p}$ | 0.0322 | `=(x1+x2)/(n1+n2)` | | Standard error | 0.001248 | `=SQRT(p*(1-p)*(1/n1+1/n2))` | | z | 1.9227 | `=(p2-p1)/se` | | Two-sided p-value | 0.0545 | `=2*(1-NORM.S.DIST(ABS(z), TRUE))` | | 95% CI for the difference | -0.005 to 0.485 points | `=d - 1.96*SQRT(p1*(1-p1)/n1 + p2*(1-p2)/n2)` and the same with `+` (unpooled standard error, 0.001248) | The result is not significant at 5%, and the interval includes zero. The test was underpowered: to detect a lift from 3.10% to 3.30% with 80% power at 5% significance needs about 121,562 visitors per arm, three times what was used. The correct conclusion is "promising but not shown", not "AI copy works" or "AI copy fails". Run the test longer, decide the sample size in advance (academic/stats-power-sample-size), and track guardrail metrics (returns rate, complaints about inaccurate descriptions) as well as conversion. See Kohavi, Tang and Xu (2020) on trustworthy online experiments and product/ab-test-plan for the plan format. ## Worked example B (half solved): Tallow Creek readmission model **Situation (fictional).** Tallow Creek Medical Center tests a model that predicts 30-day readmission at discharge. Of 5,000 discharged patients, 600 (12%) were readmitted. At the proposed threshold the model flagged 1,000 patients, 360 of whom were readmitted. Flagged patients get a follow-up call from a nurse (USD 50 each), which prevents an estimated 20% of readmissions among those correctly flagged; a readmission costs the hospital USD 9,000. | *Example* | Readmitted | Not readmitted | Total | |---|---|---|---| | Flagged | 360 | [ ] | 1,000 | | Not flagged | [ ] | [ ] | [ ] | | Total | 600 | 4,400 | 5,000 | 1. Complete the matrix. 2. Compute precision, recall, specificity and accuracy. Compare accuracy with a rule that flags no one. 3. Compute readmissions prevented, savings, programme cost and net benefit. 4. Nurses say the model "is usually right, so we call whoever it lists". What bias does this describe, and what would you change? 5. What subgroup analysis would you ask for before scaling? ## Exercise C (unsolved): Corvane failure alerts **Situation (fictional).** Corvane Robotics monitors 1,800 robots for 12 months: 21,600 robot-months, with 1,728 failures (8%). A preventive visit costs USD 400; a failure costs USD 3,000. The vendor offers two thresholds: | *Example* threshold | Alerts that were failures (TP) | Alerts that were not (FP) | |---|---|---| | H (high) | 864 | 1,080 | | L (low) | 1,296 | 4,968 | 1. Build the confusion matrix for each threshold. 2. Compute precision, recall and specificity for each. 3. Compute the yearly cost of each (visit every alert, pay for every missed failure) and compare with no model (wait) and with visiting every robot every month. 4. Which threshold should Corvane use? Why does the one with the higher recall cost more? 5. A technician says, "The alerts are wrong half the time, I ignore them." Which managerial response fits: retraining, a different threshold, or explaining the cost logic? Justify. ## When not to use AI | Question | If yes | |---|---| | Is there no reliable outcome data to learn from or test against? | Use expert judgment and rules; collect data first | | Would errors cause serious, irreversible harm to people, and can no one check outputs in time? | Do not automate the decision; consider decision support with mandatory review, or no AI | | Does the decision require reasons the affected person can contest, which the system cannot give? | Use an explainable method or keep the decision human | | Is the use prohibited or heavily restricted by law (see mba/mba-ai-risk-governance-card)? | Stop, or redesign under legal advice | | Is the volume so low that a person can decide well at reasonable cost? | AI may not repay its fixed cost | | Does the environment change faster than the model can be retrained? | Prefer simple, robust rules | | Would using AI erode a skill the organisation must keep (for example, junior staff never learning the task)? | Design for learning, or limit use | | Would customers or employees reasonably object if they knew? | Consult them, or do not proceed | ## Decision diagram The diagram links error cost, reversibility, volume and the reviewer's ability to the level of human involvement. ```mermaid flowchart TD A["AI output feeds a decision"] --> B{"Could an error cause serious harm to people?"} B -- Yes --> C{"Can a qualified person check each output in time?"} C -- No --> C1["Do not automate: no AI, or advice only"] C -- Yes --> D["Human in the loop: approve each output"] B -- No --> E{"Are errors cheap and quickly reversible?"} E -- Yes --> F{"Is the volume high?"} F -- Yes --> G["Human out of the loop within limits, with audits"] F -- No --> H["Human on the loop: monitor and override"] E -- No --> H D --> I["Measure override rates and audit accepted outputs"] G --> I H --> I ``` ## Connections to other cards | Card or template | What connects | |---|---| | mba/mba-ai-strategy-card | The value of a prediction; the business case uses the cost table here | | mba/mba-ai-risk-governance-card | Oversight duties and risk register for the same systems | | mba/mba-decision-trees-card | Expected cost of actions under uncertainty | | mba/mba-regression-for-managers-card | Logistic regression as a scoring model | | academic/stats-proportion-tests | The two-proportion z test | | academic/stats-power-sample-size | Sample size before an A/B test | | academic/stats-experiment-design | Randomisation, blocking, guardrail metrics | | ai-data/human-oversight-design | Documenting the human role | ## Common mistakes and misconceptions 1. **Judging by accuracy when positives are rare.** "Never flag" was 97% accurate in Worked example A. 2. **Reading a score as a probability** without a calibration check. 3. **Choosing the threshold that maximises accuracy or F1** instead of the one that minimises business cost. 4. **Assuming a human in the loop is safe by default.** Under time pressure people rubber-stamp; measure override rates and audit accepted outputs. 5. **Abandoning a good model after one visible error** (algorithm aversion); compare its error rate with the human baseline. 6. **Before-and-after comparisons instead of experiments.** Seasonality, promotions and novelty make them misleading. 7. **Stopping an A/B test when it first looks significant.** Repeated peeking inflates the false positive rate. 8. **Treating "not significant" as "no effect".** An underpowered test cannot tell; report the confidence interval. 9. **Ignoring subgroups.** Overall metrics hide groups for whom the model performs badly. 10. **Evaluating generative AI on a few impressive demos** instead of a representative test set with error categories. ## Self-test questions 1. Define precision and recall in one sentence each, in words a store manager would understand. 2. Positives are 2% of cases. What accuracy does "never flag" achieve, and what does that imply? 3. In Worked example A, why is the lowest-cost threshold lower than the highest-accuracy threshold? 4. A model's scores of about 0.6 correspond to an observed rate of 0.3. Is it over- or underconfident? Does that stop it from being useful for ranking? 5. Compute the Brier score of two forecasts, 0.8 and 0.2, for outcomes 1 and 1. 6. Distinguish human in the loop, on the loop and out of the loop, with one example each. 7. What is automation bias, and name two design measures that reduce it. 8. What is algorithm aversion, and what evidence would you show a team that has stopped using a model? 9. An A/B test gives p = 0.0545. State the correct conclusion and the next step. 10. Give three situations in which you would not use AI for a decision. ## My notes and next steps | Item | My answer | |---|---| | A system I use, and the decision it feeds | [ ] | | Cost of a false positive and a false negative there | [ ] | | Current human role (in, on or out of the loop) | [ ] | | One change I would make | [ ] | ## Sources and further reading - Parasuraman, R., and Riley, V. (1997). Humans and automation: Use, misuse, disuse, abuse. *Human Factors*, 39(2), 230-253. - Skitka, L. J., Mosier, K. L., and Burdick, M. (1999). Does automation bias decision-making? *International Journal of Human-Computer Studies*, 51(5), 991-1006. - Parasuraman, R., and Manzey, D. H. (2010). Complacency and bias in human use of automation: An attentional integration. *Human Factors*, 52(3), 381-410. - Dietvorst, B. J., Simmons, J. P., and Massey, C. (2015). Algorithm aversion: People erroneously avoid algorithms after seeing them err. *Journal of Experimental Psychology: General*, 144(1), 114-126. - Logg, J. M., Minson, J. A., and Moore, D. A. (2019). Algorithm appreciation: People prefer algorithmic to human judgment. *Organizational Behavior and Human Decision Processes*, 151, 90-103. - Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. *Monthly Weather Review*, 78(1), 1-3. - Fawcett, T. (2006). An introduction to ROC analysis. *Pattern Recognition Letters*, 27(8), 861-874. - Dell'Acqua, F., McFowland III, E., Mollick, E. R., et al. (2026). Navigating the jagged technological frontier: Field experimental evidence of the effects of artificial intelligence on knowledge worker productivity and quality. *Organization Science*, 37(2), 403-423. Working paper version 2023, SSRN 4573321. - Kohavi, R., Tang, D., and Xu, Y. (2020). *Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing*. Cambridge University Press. - Textbooks for further reading (named only): Provost and Fawcett, *Data Science for Business*; Kahneman, Sibony and Sunstein, *Noise*; Agrawal, Gans and Goldfarb, *Prediction Machines*. - Open resource: OpenStax, *Introductory Statistics 2e* (CC BY-NC-SA 4.0), chapters on hypothesis testing with two samples. ## Answer key **Worked example B.** (1) FP = 640, FN = 240, TN = 3,760; not flagged total 4,000. (2) Precision $360/1{,}000 = 0.3600$; recall $360/600 = 0.6000$; specificity $3{,}760/4{,}400 = 0.8545$; accuracy $(360 + 3{,}760)/5{,}000 = 0.8240$, below the 0.8800 of flagging no one, yet the model is valuable. (3) Prevented $0.20 \times 360 = 72$; savings $72 \times 9{,}000 =$ USD 648,000; cost $1{,}000 \times 50 =$ USD 50,000; net USD 598,000. (4) Automation bias: nurses defer to the list and stop using their own judgment about patients the model missed (240 readmissions). Let nurses add patients they are worried about, show the main risk factors, and audit a sample of unflagged discharges. (5) Recall and precision by age group, language, insurance status, neighbourhood and diagnosis group, to check that the model does not miss groups who already face worse access to care. **Exercise C.** Threshold H: TP 864, FP 1,080, FN 864, TN 18,792; precision 0.4444, recall 0.5000, specificity 0.9457; cost $(864 + 1{,}080) \times 400 + 864 \times 3{,}000 =$ USD 3,369,600. Threshold L: TP 1,296, FP 4,968, FN 432, TN 14,904; precision 0.2069, recall 0.7500, specificity 0.7500; cost USD 3,801,600. No model (wait): $1{,}728 \times 3{,}000 =$ USD 5,184,000; visit every robot every month: $21{,}600 \times 400 =$ USD 8,640,000. Choose H: it saves USD 1.81m a year against waiting. L catches more failures but sends technicians on 3,888 more false alarms, which cost more than the 432 extra failures it prevents ($3{,}888 \times 400 = 1{,}555{,}200$ against $432 \times (3{,}000 - 400) = 1{,}123{,}200$). On the technician's remark: precision of 0.44 is expected and still profitable, so explain the cost logic and show the savings; a different threshold would cost more. **Self-test.** 1. Precision: of the cases we flag, the share that really are problems. Recall: of all the real problems, the share we catch. 2. 98%. Accuracy says almost nothing about usefulness when positives are rare; use precision, recall and cost. 3. A missed fraud (USD 120) costs far more than a false flag (USD 20 including review), so it pays to flag more cases and accept lower precision; accuracy weighs both errors equally. 4. Overconfident (it predicts 0.6 where 0.3 happens). It can still rank cases well, so thresholds work, but its scores must not be read as probabilities until recalibrated. 5. $((0.8 - 1)^2 + (0.2 - 1)^2)/2 = (0.04 + 0.64)/2 = 0.34$. 6. In the loop: an agent approves each suggested reply. On the loop: an automated pricing rule with alerts and override. Out of the loop: spam filtering within limits, with periodic audits. 7. Over-reliance on automated advice, including missing errors the system did not flag. Measures: ask for an independent judgment before showing the score on a sample of cases, show reasons and uncertainty, audit accepted outputs, manage workload. 8. Losing trust in an algorithm after seeing it err, more than one would for a human with the same error rate. Show the model's error rate side by side with the human baseline on the same cases. 9. The difference is not significant at 5% and the interval includes zero: the test cannot tell whether there is an effect. Compute the required sample size and run the test to that size, with guardrail metrics. 10. No reliable outcome data; serious irreversible harm with no timely human check; the affected person needs contestable reasons the system cannot give; a legal prohibition; volumes so low that people decide well and cheaply. ## Revision history | Version | Date | Author | Change summary | |---|---|---|---| | 0.1 | YYYY-MM-DD | [Your name] | First pass through the card | | 0.2 | YYYY-MM-DD | [Your name] | *Example:* Replaced Northfield example with own model evaluation | ## Standard and licence note Based on the research on human-automation interaction, forecast verification, classifier evaluation and online experiments listed above (Parasuraman and Riley 1997; Skitka, Mosier and Burdick 1999; Dietvorst, Simmons and Massey 2015; Brier 1950; Fawcett 2006; Kohavi, Tang and Xu 2020; Dell'Acqua et al. 2026), checked in October 2026. The text, examples, numbers and questions are original; no textbook or case text is copied or paraphrased, and Northfield Outdoor Co., Tallow Creek Medical Center and Corvane Robotics are fictional. The template is independent and not affiliated with or endorsed by any business school, publisher or software vendor. Template licence: MIT.