# Human-AI Interaction Review: [Product or feature name]

> Design review of an AI-powered product experience, structured on the 18
> Guidelines for Human-AI Interaction (Amershi et al., CHI 2019, also
> published in the Microsoft HAX Toolkit). The guidelines are grouped by
> when they apply: initially, during interaction, when the system is
> wrong, and over time. For each guideline the review records how the
> product applies it, the evidence, any violation, its severity and the
> fix. Replace every `[bracketed]` value and every row marked *Example*.

<!--
Guidance: How to use this template
- Review a real build or a high-fidelity prototype, not a specification.
  Walk through concrete scenarios (section 3), including at least one
  where the AI is wrong.
- Use two to four reviewers from design, research, engineering and product.
  Each reviewer rates independently first, then the group reconciles.
- For every guideline, record evidence: a screenshot, a recording
  timestamp, a scenario step or a user study finding. "Looks fine" is not
  evidence.
- Mark a guideline "Not applicable" only with a reason. The guidelines are
  prompts for discussion, not a pass-fail test, and some trade off against
  each other (for example remembering more versus asking less).
- This review complements a classic usability inspection; see Heuristic
  Evaluation (`design/heuristic-evaluation`). Use both for AI features.
- Common pitfall: reviewing only the happy path. Most AI-specific problems
  appear in the "when wrong" and "over time" groups.
-->

## Document control

<!-- Guidance: Identify the build reviewed, the scenarios used and who took part. A review without a build identifier cannot be repeated. -->

| Field | Value |
|---|---|
| Document ID | [HAIR-001] |
| Product or feature | [Name] |
| Build or prototype reviewed | [*Example:* Web app 4.12.0 staging, model release 2026-09] |
| Version | [0.1] |
| Status | [Draft / In review / Final] |
| Owner | [@design-lead] |
| Approvers / reviewers | [@product-manager], [@ml-lead], [@ux-researcher] |
| Review dates | YYYY-MM-DD to YYYY-MM-DD |
| Last updated | YYYY-MM-DD |
| Classification | [Internal / Confidential] |
| Related documents | Copilot and Agent PRD (`ai-data/copilot-prd`), AI Feature PRD (`ai-data/ai-feature-prd`) |

### Approvals

| Role | Name | Decision | Date |
|---|---|---|---|
| Design lead | [@name] | [Accept findings / Request changes] | YYYY-MM-DD |
| Product manager | [@name] | [Decision] | YYYY-MM-DD |
| Engineering lead | [@name] | [Decision] | YYYY-MM-DD |

## Revision history

<!-- Guidance: Record each review round and re-review after fixes. Keep findings IDs stable across rounds. -->

| Version | Date | Author | Change summary |
|---|---|---|---|
| 0.1 | YYYY-MM-DD | [@name] | First review round |
| 0.2 | YYYY-MM-DD | [@name] | *Example:* Reconciled ratings from three reviewers |
| 1.0 | YYYY-MM-DD | [@name] | *Example:* Re-review after fixes; 4 findings closed |

## Contents

<!-- Guidance: Keep this list aligned with the headings below. -->

1. [Scope and method](#1-scope-and-method)
2. [Rating scales](#2-rating-scales)
3. [Scenarios](#3-scenarios)
4. [Initially (G1 to G2)](#4-initially-g1-to-g2)
5. [During interaction (G3 to G6)](#5-during-interaction-g3-to-g6)
6. [When wrong (G7 to G11)](#6-when-wrong-g7-to-g11)
7. [Over time (G12 to G18)](#7-over-time-g12-to-g18)
8. [Summary scorecard](#8-summary-scorecard)
9. [Findings and action plan](#9-findings-and-action-plan)
10. [Glossary and references](#10-glossary-and-references)

## 1. Scope and method

<!-- Guidance: State what was reviewed and what was not, who reviewed it, and how. Pitfall: reviewing one surface and reporting on the whole product. -->

### 1.1 Scope

| Item | Value |
|---|---|
| AI capabilities in scope | [*Example:* Meeting summary, suggested action items, auto-assign to owner] |
| Surfaces in scope | [*Example:* Web app meeting page, email digest] |
| Out of scope | [*Example:* Mobile app, admin console] |
| Users represented | [*Example:* Team members and team leads; novice and frequent users] |
| Autonomy level of the AI | [*Example:* Suggests summaries; auto-assigns tasks with notification] |

### 1.2 Method

| Step | Description |
|---|---|
| Preparation | Reviewers read the PRD and agree the scenarios in section 3 |
| Independent review | Each reviewer walks every scenario and rates each guideline |
| Reconciliation | Group meeting; disagreements resolved with evidence |
| Optional user check | [*Example:* 5 moderated sessions to confirm the top findings] |
| Reporting | Findings logged in section 9 with owners |

**Reviewers:** [@reviewer-1] (design), [@reviewer-2] (research),
[@reviewer-3] (engineering).

## 2. Rating scales

<!-- Guidance: Use the same scales across reviews so results are comparable between releases. -->

**Application rating** (how well the product applies the guideline):

| Rating | Meaning |
|---|---|
| Applied | Clearly applied across the scenarios reviewed |
| Partly applied | Applied in some places or in a weak form |
| Violated | The product does the opposite, or users are harmed by its absence |
| Not applicable | Does not apply to this product; reason recorded |

**Severity** (for Partly applied or Violated):

| Severity | Meaning |
|---|---|
| S1 Critical | Can cause harm, wrong decisions with real consequences, or loss of user data or control |
| S2 Major | Users are misled, frustrated or lose trust; workarounds are hard |
| S3 Minor | Friction or confusion with an easy workaround |
| S4 Cosmetic | Polish issue; fix when convenient |

## 3. Scenarios

<!-- Guidance: Write scenarios that exercise each guideline group: first use, normal use, an AI error, and use after weeks of history. Pitfall: scenarios written so the AI always succeeds. -->

| ID | Scenario | Group mainly exercised | Setup |
|---|---|---|---|
| HAIR-SCN-001 *(Example)* | New user opens the first AI meeting summary | Initially | Fresh account, no history |
| HAIR-SCN-002 *(Example)* | User reviews a summary during a busy day and accepts action items | During interaction | Account with 2 weeks of meetings |
| HAIR-SCN-003 *(Example)* | Summary assigns an action item to the wrong person | When wrong | Meeting with two people sharing a first name |
| HAIR-SCN-004 *(Example)* | User wants to stop auto-assignment for one team | When wrong, Over time | Team settings available |
| HAIR-SCN-005 *(Example)* | Model update changes summary style after 3 months of use | Over time | Staging build with new model |
| [ID] | [Scenario] | [Group] | [Setup] |

### 3.1 Evidence log

Store each piece of evidence once and refer to it by ID from the
guideline tables. Evidence must be reproducible: name the build, the
account used and the exact step.

| ID | Type | Description | Scenario and step | Location |
|---|---|---|---|---|
| S-01 *(Example)* | Screenshot | Onboarding card on first summary | HAIR-SCN-001, step 1 | [Link to design review folder] |
| R-01 *(Example)* | Screen recording | Notification during presentation, 04:10 | HAIR-SCN-002, step 3 | [Link] |
| D-01 *(Example)* | Data sample | Assignee role distribution in 50 meetings | HAIR-SCN-002 | [Link to analysis] |
| U-01 *(Example)* | User session note | Participant 3 could not find how to stop auto-assign | HAIR-SCN-004 | [Link to research notes] |
| [ID] | [Type] | [Description] | [Scenario] | [Link] |

### 3.2 Reviewer calibration

Before rating, reviewers rate one guideline together on one scenario and
agree how the scales in section 2 apply. Record disagreements that
remain after reconciliation here, with the final rating and the reason.

| Guideline | Reviewer ratings | Final rating | Reason |
|---|---|---|---|
| G6 *(Example)* | Applied, Partly applied, Violated | Partly applied | Skew seen in sample, cause not yet confirmed |
| [G#] | [Ratings] | [Final] | [Reason] |

## 4. Initially (G1 to G2)

<!-- Guidance: These guidelines set expectations before and at first use. Questions: does a new user know what the AI does and how often it is wrong? Pitfall: marketing copy that promises more than the system does. -->

| ID | Guideline (paraphrased) | How the product applies it | Evidence | Rating | Severity | Finding ID |
|---|---|---|---|---|---|---|
| G1 | **Make clear what the system can do.** Explain the AI's abilities and their boundaries so users form an accurate picture of what to ask for. | *Example:* Onboarding card lists summary, action items and auto-assign; no mention that video content is ignored | HAIR-SCN-001 step 1, screenshot S-01 | Partly applied | S3 | HAIR-F-001 |
| G2 | **Make clear how well the system can do what it can do.** Tell users how reliable the AI is, including that it makes mistakes and roughly how often. | *Example:* No statement that action items may be wrong or missed | HAIR-SCN-001, SCN-003 | Violated | S2 | HAIR-F-002 |

## 5. During interaction (G3 to G6)

<!-- Guidance: These guidelines cover regular use: timing, relevance and social fit. Questions: does the AI interrupt at the right moment, show what matters for the current task, and behave in a way that fits the users' norms and avoids harmful bias? Pitfall: testing only with one language, culture or naming pattern. -->

| ID | Guideline (paraphrased) | How the product applies it | Evidence | Rating | Severity | Finding ID |
|---|---|---|---|---|---|---|
| G3 | **Time services based on context.** Pick the moment for any action or interruption from the user's current task and surroundings. | *Example:* Summary notification arrives while the user is presenting in the next meeting | HAIR-SCN-002, recording 04:10 | Partly applied | S3 | HAIR-F-003 |
| G4 | **Show contextually relevant information.** Surface information that fits the task and situation at hand rather than everything available. | *Example:* Summary highlights decisions and items for the viewing user first | HAIR-SCN-002 | Applied | Not applicable | Not applicable |
| G5 | **Match relevant social norms.** Fit tone, wording and conduct to the conventions of the people using the product and their culture. | *Example:* Summaries use neutral, professional tone; no casual phrasing | HAIR-SCN-002 | Applied | Not applicable | Not applicable |
| G6 | **Mitigate social biases.** Check that outputs and actions treat groups fairly and do not echo harmful stereotypes. | *Example:* Action items more often assigned to junior staff named in passing; under investigation | HAIR-SCN-002, sample of 50 meetings | Partly applied | S2 | HAIR-F-004 |

## 6. When wrong (G7 to G11)

<!-- Guidance: These guidelines cover what happens when the AI is not wanted, is mistaken or is uncertain. They matter most for trust. Questions: can users call the AI up, ignore it, correct it, see why it acted, and does it hold back when unsure? Pitfall: an undo that exists but takes more effort than doing the task by hand. -->

| ID | Guideline (paraphrased) | How the product applies it | Evidence | Rating | Severity | Finding ID |
|---|---|---|---|---|---|---|
| G7 | **Support efficient invocation.** Give users a quick, obvious way to start the AI feature whenever they want it. | *Example:* "Summarise" button available on any meeting page and from the keyboard shortcut | HAIR-SCN-002 | Applied | Not applicable | Not applicable |
| G8 | **Support efficient dismissal.** Let users ignore or close unwanted AI suggestions with little effort and no penalty. | *Example:* Suggested items can be dismissed one by one only; no "dismiss all" | HAIR-SCN-002 | Partly applied | S3 | HAIR-F-005 |
| G9 | **Support efficient correction.** Make fixing a wrong output fast: changing, adjusting or reverting it should cost less than doing the task by hand. | *Example:* Wrong assignee can be changed, but the original assignee was already notified | HAIR-SCN-003 step 4 | Violated | S1 | HAIR-F-006 |
| G10 | **Scope services when in doubt.** When the AI is unsure what the user wants, ask, offer options, or do less rather than guess. | *Example:* With two people named "Sam", the AI picks one silently | HAIR-SCN-003 step 2 | Violated | S2 | HAIR-F-007 |
| G11 | **Make clear why the system did what it did.** Let users find out what led the AI to a given output or action. | *Example:* Each action item links to the transcript moment it came from | HAIR-SCN-003 | Applied | Not applicable | Not applicable |

## 7. Over time (G12 to G18)

<!-- Guidance: These guidelines cover memory, learning, feedback and change over weeks and months. Questions: does the AI remember what helps, learn without surprising people, accept feedback, explain the effects of user actions, offer global settings, and announce changes? Pitfall: a model update shipped silently that changes behaviour users relied on. -->

| ID | Guideline (paraphrased) | How the product applies it | Evidence | Rating | Severity | Finding ID |
|---|---|---|---|---|---|---|
| G12 | **Remember recent interactions.** Keep track of what happened earlier in the session so users can point back to it without repeating themselves. | *Example:* Follow-up question "who owns the budget item?" understood in the same panel | HAIR-SCN-002 | Applied | Not applicable | Not applicable |
| G13 | **Learn from user behavior.** Adapt to each user gradually, using the patterns in what they do. | *Example:* Repeated reassignments of design items to the same person are not learned | HAIR-SCN-004 | Partly applied | S4 | HAIR-F-008 |
| G14 | **Update and adapt cautiously.** Introduce adaptations and updates gradually so behaviour users rely on does not shift abruptly. | *Example:* New model doubles summary length with no transition | HAIR-SCN-005 | Violated | S3 | HAIR-F-009 |
| G15 | **Encourage granular feedback.** Let users express preferences about specific outputs during normal use, not only through a general survey. | *Example:* Per-item thumbs up or down with optional reason | HAIR-SCN-002 | Applied | Not applicable | Not applicable |
| G16 | **Convey the consequences of user actions.** When a user acts, tell them right away whether and how that will shape later AI behaviour. | *Example:* Dismissing items gives no hint whether future suggestions change | HAIR-SCN-004 | Violated | S3 | HAIR-F-010 |
| G17 | **Provide global controls.** Offer product-wide settings for what the AI watches and what it is allowed to do. | *Example:* Auto-assign can be switched off only per meeting, not per team | HAIR-SCN-004 | Partly applied | S2 | HAIR-F-011 |
| G18 | **Notify users about changes.** Announce new or changed AI capabilities where users will notice them. | *Example:* No in-product notice for model updates | HAIR-SCN-005 | Violated | S3 | HAIR-F-012 |

## 8. Summary scorecard

<!-- Guidance: Count ratings per group so the team sees where effort is needed. Compare with the previous review round. Pitfall: averaging into a single score, which hides one critical violation behind many passes. -->

| Group | Guidelines | Applied | Partly applied | Violated | Not applicable | Highest severity |
|---|---|---|---|---|---|---|
| Initially *(Example)* | G1 to G2 | 0 | 1 | 1 | 0 | S2 |
| During interaction *(Example)* | G3 to G6 | 2 | 2 | 0 | 0 | S2 |
| When wrong *(Example)* | G7 to G11 | 2 | 1 | 2 | 0 | S1 |
| Over time *(Example)* | G12 to G18 | 2 | 2 | 3 | 0 | S2 |
| **Total** *(Example)* | 18 | 6 | 6 | 6 | 0 | S1 |

```mermaid
flowchart LR
  I[Initially<br/>G1 to G2] --> D[During interaction<br/>G3 to G6]
  D --> W[When wrong<br/>G7 to G11]
  W --> O[Over time<br/>G12 to G18]
  O -. feedback and updates .-> D
```

**Overall assessment:** [Two to four sentences.] *Example:* The feature
explains itself well and sources its output, but correction is costly
once a wrong assignment has notified a colleague, and the product does
not hold back when unsure. Fix HAIR-F-006 and HAIR-F-007 before widening
the rollout.

### 8.1 Trade-offs between guidelines

Some guidelines pull in different directions. Record each trade-off the
team decided, so the next review does not reopen it without new
evidence.

| Trade-off | Guidelines | Decision | Reason |
|---|---|---|---|
| Asking when unsure versus interrupting less *(Example)* | G10 versus G3 | Ask only for assignee ambiguity; batch questions at the end of the meeting | Wrong assignment costs more than one question |
| Learning from behaviour versus stable behaviour *(Example)* | G13 versus G14 | Learn assignee preferences per team, apply only after 3 consistent corrections | Avoid sudden changes from one correction |
| Remembering context versus privacy expectations *(Example)* | G12 versus G5 | Keep memory within one meeting page; nothing across teams | Team members expect meeting content to stay in the team |
| [Trade-off] | [G# versus G#] | [Decision] | [Reason] |

## 9. Findings and action plan

<!-- Guidance: One row per finding, ordered by severity. Each finding names the guideline, the scenario, a concrete fix and an owner. Re-review closed findings in the next round. Pitfall: fixes phrased as "improve UX" with no testable change. -->

| ID | Guideline | Severity | Finding | Recommended fix | Owner | Due | Status |
|---|---|---|---|---|---|---|---|
| HAIR-F-006 *(Example)* | G9 | S1 | Wrong assignee is notified before the user can correct | Hold notifications for 10 minutes or until the meeting owner confirms; allow recall | [@eng-lead] | YYYY-MM-DD | Open |
| HAIR-F-007 *(Example)* | G10 | S2 | Ambiguous names resolved silently | Show both candidates and ask; leave unassigned if unanswered | [@design-lead] | YYYY-MM-DD | Open |
| HAIR-F-002 *(Example)* | G2 | S2 | No statement of reliability | Add short note on first use and in help: items can be missed or misassigned | [@content-designer] | YYYY-MM-DD | Open |
| HAIR-F-004 *(Example)* | G6 | S2 | Assignment skew toward junior staff | Audit 200 meetings by role; adjust prompt and add evaluation | [@ml-lead] | YYYY-MM-DD | Investigating |
| HAIR-F-011 *(Example)* | G17 | S2 | No team-level control for auto-assign | Add team setting with default off for new teams | [@product-manager] | YYYY-MM-DD | Planned |
| [ID] | [G#] | [S1 to S4] | [Finding] | [Fix] | [@owner] | YYYY-MM-DD | [Status] |

**Release recommendation:** [Ship / Ship after S1 fixes / Do not ship].

### 9.1 Re-review record

After fixes ship, re-run the scenarios linked to each finding and record
the result. A finding closes only when the guideline rating improves on
the same scenario.

| Finding | Fix shipped in build | Re-review date | New rating | Closed |
|---|---|---|---|---|
| HAIR-F-006 *(Example)* | 4.13.0 | YYYY-MM-DD | Applied | Yes |
| HAIR-F-007 *(Example)* | 4.13.0 | YYYY-MM-DD | Partly applied | No: question shown, but unanswered items still auto-assign |
| [ID] | [Build] | YYYY-MM-DD | [Rating] | [Yes / No] |

## 10. Glossary and references

<!-- Guidance: Define terms the team may read differently and list the sources and related documents. -->

### 10.1 Glossary

| Term | Meaning in this review |
|---|---|
| AI-infused system | A product whose features use AI capabilities directly visible to the user |
| Dismissal | Ignoring or closing an AI suggestion |
| Global control | A setting that changes AI behaviour across the product, not one instance |
| Invocation | A user explicitly calling an AI feature |
| Scoping | Reducing what the AI does, or asking, when its confidence about user intent is low |

### 10.2 References

- Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B.,
  Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., Teevan,
  J., Kikin-Gil, R., and Horvitz, E. (2019). Guidelines for Human-AI
  Interaction. Proceedings of CHI 2019, Glasgow, ACM.
  https://doi.org/10.1145/3290605.3300233
- Microsoft HAX Toolkit, Guidelines for Human-AI Interaction and HAX
  Design Library: https://www.microsoft.com/en-us/haxtoolkit/ai-guidelines/
- Microsoft Research project page:
  https://www.microsoft.com/en-us/research/project/guidelines-for-human-ai-interaction/
- Related templates: Heuristic Evaluation (`design/heuristic-evaluation`),
  Copilot and Agent PRD (`ai-data/copilot-prd`), AI Feature PRD
  (`ai-data/ai-feature-prd`), Human Oversight Design
  (`ai-data/human-oversight-design`), AI Launch Review
  (`ai-data/ai-launch-review`), UX Research Plan (`design/ux-research-plan`).

## Standard and licence note

Structure aligned with the 18 Guidelines for Human-AI Interaction
(Amershi et al., CHI 2019) as presented in the Microsoft HAX Toolkit,
including their numbering G1 to G18 and the four groups initially,
during interaction, when wrong and over time. Guideline names are cited
for reference; every description, rating scale, scenario and example in
this template is original text and does not reproduce the paper's
descriptions or examples. Check the current HAX Toolkit for updates and
for the design patterns that accompany each guideline. Template licence:
MIT.