Human-AI Interaction Review: [Product or feature name]
Design review of an AI-powered product experience, structured on the 18
Guidelines for Human-AI Interaction (Amershi et al., CHI 2019, also
published in the Microsoft HAX Toolkit). The guidelines are grouped by
when they apply: initially, during interaction, when the system is
wrong, and over time. For each guideline the review records how the
product applies it, the evidence, any violation, its severity and the
fix. Replace every [bracketed] value and every row marked Example.
Document control
| Field | Value |
|---|
| Document ID | [HAIR-001] |
| Product or feature | [Name] |
| Build or prototype reviewed | [Example: Web app 4.12.0 staging, model release 2026-09] |
| Version | [0.1] |
| Status | [Draft / In review / Final] |
| Owner | [@design-lead] |
| Approvers / reviewers | [@product-manager], [@ml-lead], [@ux-researcher] |
| Review dates | YYYY-MM-DD to YYYY-MM-DD |
| Last updated | YYYY-MM-DD |
| Classification | [Internal / Confidential] |
| Related documents | Copilot and Agent PRD (ai-data/copilot-prd), AI Feature PRD (ai-data/ai-feature-prd) |
Approvals
| Role | Name | Decision | Date |
|---|
| Design lead | [@name] | [Accept findings / Request changes] | YYYY-MM-DD |
| Product manager | [@name] | [Decision] | YYYY-MM-DD |
| Engineering lead | [@name] | [Decision] | YYYY-MM-DD |
Revision history
| Version | Date | Author | Change summary |
|---|
| 0.1 | YYYY-MM-DD | [@name] | First review round |
| 0.2 | YYYY-MM-DD | [@name] | Example: Reconciled ratings from three reviewers |
| 1.0 | YYYY-MM-DD | [@name] | Example: Re-review after fixes; 4 findings closed |
Contents
1. Scope and method
1.1 Scope
| Item | Value |
|---|
| AI capabilities in scope | [Example: Meeting summary, suggested action items, auto-assign to owner] |
| Surfaces in scope | [Example: Web app meeting page, email digest] |
| Out of scope | [Example: Mobile app, admin console] |
| Users represented | [Example: Team members and team leads; novice and frequent users] |
| Autonomy level of the AI | [Example: Suggests summaries; auto-assigns tasks with notification] |
1.2 Method
| Step | Description |
|---|
| Preparation | Reviewers read the PRD and agree the scenarios in section 3 |
| Independent review | Each reviewer walks every scenario and rates each guideline |
| Reconciliation | Group meeting; disagreements resolved with evidence |
| Optional user check | [Example: 5 moderated sessions to confirm the top findings] |
| Reporting | Findings logged in section 9 with owners |
Reviewers: [@reviewer-1] (design), [@reviewer-2] (research),
[@reviewer-3] (engineering).
2. Rating scales
Application rating (how well the product applies the guideline):
| Rating | Meaning |
|---|
| Applied | Clearly applied across the scenarios reviewed |
| Partly applied | Applied in some places or in a weak form |
| Violated | The product does the opposite, or users are harmed by its absence |
| Not applicable | Does not apply to this product; reason recorded |
Severity (for Partly applied or Violated):
| Severity | Meaning |
|---|
| S1 Critical | Can cause harm, wrong decisions with real consequences, or loss of user data or control |
| S2 Major | Users are misled, frustrated or lose trust; workarounds are hard |
| S3 Minor | Friction or confusion with an easy workaround |
| S4 Cosmetic | Polish issue; fix when convenient |
3. Scenarios
| ID | Scenario | Group mainly exercised | Setup |
|---|
| HAIR-SCN-001 (Example) | New user opens the first AI meeting summary | Initially | Fresh account, no history |
| HAIR-SCN-002 (Example) | User reviews a summary during a busy day and accepts action items | During interaction | Account with 2 weeks of meetings |
| HAIR-SCN-003 (Example) | Summary assigns an action item to the wrong person | When wrong | Meeting with two people sharing a first name |
| HAIR-SCN-004 (Example) | User wants to stop auto-assignment for one team | When wrong, Over time | Team settings available |
| HAIR-SCN-005 (Example) | Model update changes summary style after 3 months of use | Over time | Staging build with new model |
| [ID] | [Scenario] | [Group] | [Setup] |
3.1 Evidence log
Store each piece of evidence once and refer to it by ID from the
guideline tables. Evidence must be reproducible: name the build, the
account used and the exact step.
| ID | Type | Description | Scenario and step | Location |
|---|
| S-01 (Example) | Screenshot | Onboarding card on first summary | HAIR-SCN-001, step 1 | [Link to design review folder] |
| R-01 (Example) | Screen recording | Notification during presentation, 04:10 | HAIR-SCN-002, step 3 | [Link] |
| D-01 (Example) | Data sample | Assignee role distribution in 50 meetings | HAIR-SCN-002 | [Link to analysis] |
| U-01 (Example) | User session note | Participant 3 could not find how to stop auto-assign | HAIR-SCN-004 | [Link to research notes] |
| [ID] | [Type] | [Description] | [Scenario] | [Link] |
3.2 Reviewer calibration
Before rating, reviewers rate one guideline together on one scenario and
agree how the scales in section 2 apply. Record disagreements that
remain after reconciliation here, with the final rating and the reason.
| Guideline | Reviewer ratings | Final rating | Reason |
|---|
| G6 (Example) | Applied, Partly applied, Violated | Partly applied | Skew seen in sample, cause not yet confirmed |
| [G#] | [Ratings] | [Final] | [Reason] |
4. Initially (G1 to G2)
| ID | Guideline (paraphrased) | How the product applies it | Evidence | Rating | Severity | Finding ID |
|---|
| G1 | Make clear what the system can do. Explain the AI's abilities and their boundaries so users form an accurate picture of what to ask for. | Example: Onboarding card lists summary, action items and auto-assign; no mention that video content is ignored | HAIR-SCN-001 step 1, screenshot S-01 | Partly applied | S3 | HAIR-F-001 |
| G2 | Make clear how well the system can do what it can do. Tell users how reliable the AI is, including that it makes mistakes and roughly how often. | Example: No statement that action items may be wrong or missed | HAIR-SCN-001, SCN-003 | Violated | S2 | HAIR-F-002 |
5. During interaction (G3 to G6)
| ID | Guideline (paraphrased) | How the product applies it | Evidence | Rating | Severity | Finding ID |
|---|
| G3 | Time services based on context. Pick the moment for any action or interruption from the user's current task and surroundings. | Example: Summary notification arrives while the user is presenting in the next meeting | HAIR-SCN-002, recording 04:10 | Partly applied | S3 | HAIR-F-003 |
| G4 | Show contextually relevant information. Surface information that fits the task and situation at hand rather than everything available. | Example: Summary highlights decisions and items for the viewing user first | HAIR-SCN-002 | Applied | Not applicable | Not applicable |
| G5 | Match relevant social norms. Fit tone, wording and conduct to the conventions of the people using the product and their culture. | Example: Summaries use neutral, professional tone; no casual phrasing | HAIR-SCN-002 | Applied | Not applicable | Not applicable |
| G6 | Mitigate social biases. Check that outputs and actions treat groups fairly and do not echo harmful stereotypes. | Example: Action items more often assigned to junior staff named in passing; under investigation | HAIR-SCN-002, sample of 50 meetings | Partly applied | S2 | HAIR-F-004 |
6. When wrong (G7 to G11)
| ID | Guideline (paraphrased) | How the product applies it | Evidence | Rating | Severity | Finding ID |
|---|
| G7 | Support efficient invocation. Give users a quick, obvious way to start the AI feature whenever they want it. | Example: "Summarise" button available on any meeting page and from the keyboard shortcut | HAIR-SCN-002 | Applied | Not applicable | Not applicable |
| G8 | Support efficient dismissal. Let users ignore or close unwanted AI suggestions with little effort and no penalty. | Example: Suggested items can be dismissed one by one only; no "dismiss all" | HAIR-SCN-002 | Partly applied | S3 | HAIR-F-005 |
| G9 | Support efficient correction. Make fixing a wrong output fast: changing, adjusting or reverting it should cost less than doing the task by hand. | Example: Wrong assignee can be changed, but the original assignee was already notified | HAIR-SCN-003 step 4 | Violated | S1 | HAIR-F-006 |
| G10 | Scope services when in doubt. When the AI is unsure what the user wants, ask, offer options, or do less rather than guess. | Example: With two people named "Sam", the AI picks one silently | HAIR-SCN-003 step 2 | Violated | S2 | HAIR-F-007 |
| G11 | Make clear why the system did what it did. Let users find out what led the AI to a given output or action. | Example: Each action item links to the transcript moment it came from | HAIR-SCN-003 | Applied | Not applicable | Not applicable |
7. Over time (G12 to G18)
| ID | Guideline (paraphrased) | How the product applies it | Evidence | Rating | Severity | Finding ID |
|---|
| G12 | Remember recent interactions. Keep track of what happened earlier in the session so users can point back to it without repeating themselves. | Example: Follow-up question "who owns the budget item?" understood in the same panel | HAIR-SCN-002 | Applied | Not applicable | Not applicable |
| G13 | Learn from user behavior. Adapt to each user gradually, using the patterns in what they do. | Example: Repeated reassignments of design items to the same person are not learned | HAIR-SCN-004 | Partly applied | S4 | HAIR-F-008 |
| G14 | Update and adapt cautiously. Introduce adaptations and updates gradually so behaviour users rely on does not shift abruptly. | Example: New model doubles summary length with no transition | HAIR-SCN-005 | Violated | S3 | HAIR-F-009 |
| G15 | Encourage granular feedback. Let users express preferences about specific outputs during normal use, not only through a general survey. | Example: Per-item thumbs up or down with optional reason | HAIR-SCN-002 | Applied | Not applicable | Not applicable |
| G16 | Convey the consequences of user actions. When a user acts, tell them right away whether and how that will shape later AI behaviour. | Example: Dismissing items gives no hint whether future suggestions change | HAIR-SCN-004 | Violated | S3 | HAIR-F-010 |
| G17 | Provide global controls. Offer product-wide settings for what the AI watches and what it is allowed to do. | Example: Auto-assign can be switched off only per meeting, not per team | HAIR-SCN-004 | Partly applied | S2 | HAIR-F-011 |
| G18 | Notify users about changes. Announce new or changed AI capabilities where users will notice them. | Example: No in-product notice for model updates | HAIR-SCN-005 | Violated | S3 | HAIR-F-012 |
8. Summary scorecard
| Group | Guidelines | Applied | Partly applied | Violated | Not applicable | Highest severity |
|---|
| Initially (Example) | G1 to G2 | 0 | 1 | 1 | 0 | S2 |
| During interaction (Example) | G3 to G6 | 2 | 2 | 0 | 0 | S2 |
| When wrong (Example) | G7 to G11 | 2 | 1 | 2 | 0 | S1 |
| Over time (Example) | G12 to G18 | 2 | 2 | 3 | 0 | S2 |
| Total (Example) | 18 | 6 | 6 | 6 | 0 | S1 |
Overall assessment: [Two to four sentences.] Example: The feature
explains itself well and sources its output, but correction is costly
once a wrong assignment has notified a colleague, and the product does
not hold back when unsure. Fix HAIR-F-006 and HAIR-F-007 before widening
the rollout.
8.1 Trade-offs between guidelines
Some guidelines pull in different directions. Record each trade-off the
team decided, so the next review does not reopen it without new
evidence.
| Trade-off | Guidelines | Decision | Reason |
|---|
| Asking when unsure versus interrupting less (Example) | G10 versus G3 | Ask only for assignee ambiguity; batch questions at the end of the meeting | Wrong assignment costs more than one question |
| Learning from behaviour versus stable behaviour (Example) | G13 versus G14 | Learn assignee preferences per team, apply only after 3 consistent corrections | Avoid sudden changes from one correction |
| Remembering context versus privacy expectations (Example) | G12 versus G5 | Keep memory within one meeting page; nothing across teams | Team members expect meeting content to stay in the team |
| [Trade-off] | [G# versus G#] | [Decision] | [Reason] |
9. Findings and action plan
| ID | Guideline | Severity | Finding | Recommended fix | Owner | Due | Status |
|---|
| HAIR-F-006 (Example) | G9 | S1 | Wrong assignee is notified before the user can correct | Hold notifications for 10 minutes or until the meeting owner confirms; allow recall | [@eng-lead] | YYYY-MM-DD | Open |
| HAIR-F-007 (Example) | G10 | S2 | Ambiguous names resolved silently | Show both candidates and ask; leave unassigned if unanswered | [@design-lead] | YYYY-MM-DD | Open |
| HAIR-F-002 (Example) | G2 | S2 | No statement of reliability | Add short note on first use and in help: items can be missed or misassigned | [@content-designer] | YYYY-MM-DD | Open |
| HAIR-F-004 (Example) | G6 | S2 | Assignment skew toward junior staff | Audit 200 meetings by role; adjust prompt and add evaluation | [@ml-lead] | YYYY-MM-DD | Investigating |
| HAIR-F-011 (Example) | G17 | S2 | No team-level control for auto-assign | Add team setting with default off for new teams | [@product-manager] | YYYY-MM-DD | Planned |
| [ID] | [G#] | [S1 to S4] | [Finding] | [Fix] | [@owner] | YYYY-MM-DD | [Status] |
Release recommendation: [Ship / Ship after S1 fixes / Do not ship].
9.1 Re-review record
After fixes ship, re-run the scenarios linked to each finding and record
the result. A finding closes only when the guideline rating improves on
the same scenario.
| Finding | Fix shipped in build | Re-review date | New rating | Closed |
|---|
| HAIR-F-006 (Example) | 4.13.0 | YYYY-MM-DD | Applied | Yes |
| HAIR-F-007 (Example) | 4.13.0 | YYYY-MM-DD | Partly applied | No: question shown, but unanswered items still auto-assign |
| [ID] | [Build] | YYYY-MM-DD | [Rating] | [Yes / No] |
10. Glossary and references
10.1 Glossary
| Term | Meaning in this review |
|---|
| AI-infused system | A product whose features use AI capabilities directly visible to the user |
| Dismissal | Ignoring or closing an AI suggestion |
| Global control | A setting that changes AI behaviour across the product, not one instance |
| Invocation | A user explicitly calling an AI feature |
| Scoping | Reducing what the AI does, or asking, when its confidence about user intent is low |
10.2 References
- Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B.,
Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., Teevan,
J., Kikin-Gil, R., and Horvitz, E. (2019). Guidelines for Human-AI
Interaction. Proceedings of CHI 2019, Glasgow, ACM.
https://doi.org/10.1145/3290605.3300233
- Microsoft HAX Toolkit, Guidelines for Human-AI Interaction and HAX
Design Library: https://www.microsoft.com/en-us/haxtoolkit/ai-guidelines/
- Microsoft Research project page:
https://www.microsoft.com/en-us/research/project/guidelines-for-human-ai-interaction/
- Related templates: Heuristic Evaluation (
design/heuristic-evaluation),
Copilot and Agent PRD (ai-data/copilot-prd), AI Feature PRD
(ai-data/ai-feature-prd), Human Oversight Design
(ai-data/human-oversight-design), AI Launch Review
(ai-data/ai-launch-review), UX Research Plan (design/ux-research-plan).
Standard and licence note
Structure aligned with the 18 Guidelines for Human-AI Interaction
(Amershi et al., CHI 2019) as presented in the Microsoft HAX Toolkit,
including their numbering G1 to G18 and the four groups initially,
during interaction, when wrong and over time. Guideline names are cited
for reference; every description, rating scale, scenario and example in
this template is original text and does not reproduce the paper's
descriptions or examples. Check the current HAX Toolkit for updates and
for the design patterns that accompany each guideline. Template licence:
MIT.