# Usability Test Plan: [Product or feature name]

> A plan for one usability test, moderated or unmoderated: the decision it informs, the object
> being evaluated, its context of use, participants, task scenarios, measures of effectiveness,
> efficiency and satisfaction (including the System Usability Scale, SUS), the session procedure,
> data handling and the analysis that turns sessions into findings. Measures follow the usability
> framework of ISO 9241-11:2018 (second edition) and the planning items that ISO/IEC 25066:2016
> (withdrawn by ISO in April 2026, see references) expects a usability test report to describe. Both are paid standards: this template cites their
> clause numbers and uses original wording. Replace every `[bracketed]` value and every row marked
> *Example*. Worked examples use Quillmere Libraries, a fictional city library service and its
> borrowing app. Privacy notes are a planning aid, not legal advice: check the current official
> text with your privacy lead.

<!--
Guidance: How to use this template
- Write section 1 (decision) and section 5 (tasks) first. If you cannot name the decision and the
  tasks, the test is not ready to plan.
- Choose formative (find problems to fix) or summative (measure against targets) in section 3.
  The choice drives sample size, measures and how much you can claim in the report.
- Run a pilot session (section 11) with the final build, script and recording set-up before the
  first real participant.
- Pair this plan with design/usability-test-report, which uses the same task IDs, measures and
  severity scale, and with design/research-consent-form for the participant paperwork.
- Common pitfall: task wording that gives away the answer by using the interface labels ("Tap
  Reserve, then choose a Branch"). Describe the goal and the situation, never the controls.
-->

## Document control

<!-- Guidance: Name the lead researcher who owns the plan and the product owner who will act on the findings. The status shows where the study is in its life. -->

| Field | Value |
|---|---|
| Document ID | [UTP-001] |
| Version | [1.0] |
| Status | [Draft / Approved / Piloted / In fieldwork / Closed] |
| Owner (lead researcher) | [@researcher] |
| Approvers / reviewers | [@product-owner], [@design-lead], [@privacy-lead] |
| Last updated | YYYY-MM-DD |
| Classification | [Internal; contains no participant personal data] |
| Object of evaluation | [Product, version or build, prototype fidelity] |
| Test type | [Formative / Summative] and [Moderated in person / Moderated remote / Unmoderated remote] |
| Fieldwork window | YYYY-MM-DD to YYYY-MM-DD |
| Report | [design/usability-test-report, document ID UTR-001] |
| Related | [design/ux-research-plan], [design/research-consent-form], [design/heuristic-evaluation] |

### Approvals

| Role | Name | Decision | Date |
|---|---|---|---|
| Product owner | [@name] | [Approved] | YYYY-MM-DD |
| Lead researcher | [@name] | [Approved] | YYYY-MM-DD |
| Privacy lead | [@name] | [Approved / Changes requested] | YYYY-MM-DD |

### Revision history

| Version | Date | Author | Change |
|---|---|---|---|
| 0.1 | YYYY-MM-DD | [@name] | First draft |
| 0.2 | YYYY-MM-DD | [@name] | Tasks and success criteria revised after pilot |
| 1.0 | YYYY-MM-DD | [@name] | Approved for fieldwork |

## Contents

1. Purpose, object of evaluation and decision
2. Context of use
3. Test type and method
4. Participants and recruitment
5. Task scenarios and success criteria
6. Measures and targets
7. Session procedure and script
8. Environment, tools and data capture
9. Roles and responsibilities
10. Consent, ethics and personal data
11. Pilot and readiness checklist
12. Analysis plan
13. Schedule and deliverables
14. Risks and limitations
15. Glossary and references

## 1. Purpose, object of evaluation and decision

<!-- Guidance: State the decision, who makes it and by when. Describe exactly what will be tested (build, prototype, fidelity) so the report can say what the findings apply to. -->

A usability test is worth running only if its result can change something. Write the decision in
one sentence, name the decision maker, and give the date after which the findings arrive too
late. Then describe the object of evaluation precisely enough that a reader of the report a year
from now knows what was in front of the participants: product name, version or build, whether it
is a live service, a coded prototype or a clickable mock-up, and which parts are in or out of scope.

| Item | Value |
|---|---|
| Decision to inform | [*Example:* ship the redesigned reservation flow in release 5.2, or hold it for another iteration] |
| Decision maker and date | [*Example:* product owner, go or no-go meeting on YYYY-MM-DD] |
| Object of evaluation | [*Example:* Quillmere Libraries app 5.2 beta build, iOS and Android, test catalogue data] |
| In scope | [*Example:* search, reserve, change collection branch, renew loans, branch opening hours] |
| Out of scope | [*Example:* account registration, payments for fines, staff app] |
| Research questions | [*Example:* RQ-1 Can borrowers reserve and redirect a reservation unaided? RQ-2 Where do they hesitate or make errors? RQ-3 How do they rate the app overall?] |

Evaluation purposes differ. ISO/IEC 25066:2016 clause 4.1 distinguishes the purpose of an
evaluation and clause 4.2 the types of usability evaluation, including user observation and
measurement against criteria. State which applies here, because a formative round of six
participants cannot support the claims a summative benchmark can.

## 2. Context of use

<!-- Guidance: Describe who the users are, what goals and tasks they have, with which resources and in which environment. The test is only valid for the context you reproduce. -->

ISO 9241-11:2018 treats usability as an outcome of use for specified users, goals and contexts
(clauses 5.3 and 7). A test result therefore holds only for the context it reproduced. Describe
the intended context here and, in the last column, how faithfully the test reproduces it. Every
gap is a limitation to report.

| Component (ISO 9241-11:2018 clause) | Intended context | Reproduced in the test |
|---|---|---|
| Users (7.2) | [*Example:* adult borrowers, from weekly users to occasional; includes screen reader users and people aged 65 and over] | [*Example:* 8 participants across three segments, see section 4] |
| Goals (7.3) | [*Example:* get a book ready to collect at a convenient branch with little effort] | [*Example:* expressed as four scenarios, see section 5] |
| Tasks (7.4) | [*Example:* search, reserve, redirect a reservation, renew, check opening hours] | [*Example:* T1 to T4] |
| Resources (7.5) | [*Example:* own smartphone, library card number, about five minutes] | [*Example:* participant's own phone; test card numbers supplied] |
| Technical environment (7.6.2) | [*Example:* mobile data or home Wi-Fi, iOS 17 or later, Android 12 or later] | [*Example:* participant's network; beta build via test distribution] |
| Physical environment (7.6.3) | [*Example:* at home, on a bus, in a branch] | [*Example:* at home, remote session] |
| Social, cultural and organisational environment (7.6.4) | [*Example:* often done while commuting, sometimes for a family member] | [*Example:* not reproduced, note as limitation] |

## 3. Test type and method

<!-- Guidance: Choose formative or summative, and moderated or unmoderated, and say why. Each choice has a cost: unmoderated tests scale but cannot probe; moderated tests explain behaviour but take more time. -->

| Choice | Selected | Rationale |
|---|---|---|
| Formative or summative | [*Example:* Formative] | [*Example:* the flow is still changing; the goal is to find and fix problems before release] |
| Moderated or unmoderated | [*Example:* Moderated remote] | [*Example:* need to probe hesitation and support screen reader users live] |
| Think-aloud | [*Example:* Concurrent think-aloud for T1 to T3, silent for T4] | [*Example:* T4 time is compared with a target, and talking slows people down] |
| Rounds | [*Example:* 1 round of 8; a second round after fixes if severity 3 or 4 issues are found] | [*Example:* iterative testing finds more problems than one large round] |
| Comparison | [*Example:* none] | [*Example:* only one design exists; for an A versus B decision see product/ab-test-plan] |

Use this guide to the main choices:

- **Formative testing** finds and explains problems. Small rounds are normal; Nielsen (NN/g, 2000)
  suggests about five users per round for one user group as a rule of thumb, more when user
  groups differ. Report problems and their evidence, not percentages that a small sample cannot
  support.
- **Summative testing** measures against targets or a benchmark. It needs a stable build, a
  fixed protocol and a larger sample; Nielsen suggests about 20 users for quantitative metrics.
  Report confidence intervals, not just point estimates.
- **Moderated sessions** let the moderator ask follow-up questions, recover from technical
  problems and adapt for assistive technology users.
- **Unmoderated sessions** run on a testing platform with written tasks. Write tasks that need
  no clarification, add a clear end condition to each task, and plan for some unusable sessions.

## 4. Participants and recruitment

<!-- Guidance: Recruit by behaviour and need, not by demographics alone. Plan the segments, the screener questions and the incentive, and include people with disabilities in every round. -->

| Segment | Definition | Target | Screener question |
|---|---|---|---|
| S1 *(Example)* | Frequent borrowers: borrowed 6 or more items in the last 6 months | 3 | "How many items have you borrowed in the last 6 months?" |
| S2 *(Example)* | Occasional borrowers: 1 to 5 items in the last 6 months | 3 | Same question |
| S3 *(Example)* | Screen reader or screen magnifier users who borrow at least occasionally | 2 | "Do you use any assistive technology on your phone? Which?" |
| Total | | 8 | |

Recruitment rules:

- Exclude [*Example:* library staff, their families, and anyone who took part in a study in the last 6 months].
- Channel: [*Example:* opt-in research panel, in-branch poster with a screener link].
- Incentive: [*Example:* gift card of a fixed value per session, paid in full even if the participant stops early or withdraws].
- Over-recruit by [*Example:* 1 participant per segment] to cover no-shows.
- Ask about access needs in the screener: captions, extra time, breaks, a support person, use of
  the participant's own device and assistive technology settings.

## 5. Task scenarios and success criteria

<!-- Guidance: Each task is a realistic scenario with a goal, a start state and an unambiguous end state. Define success before the test so that scoring is not decided by the moderator after watching. -->

| Task ID | Scenario (read to the participant) | Start state | Success criterion | Time limit | Research question |
|---|---|---|---|---|---|
| T1 *(Example)* | "A friend recommended *[fictional title]*. Make sure you can pick it up at the library nearest your home." | Logged in, home screen | Reservation exists for the title with the nearest branch as collection point | 3 min | RQ-1, RQ-2 |
| T2 *(Example)* | "You will be near the Central branch next week instead. Arrange to collect the book you reserved there." | One existing reservation at another branch | Collection branch of the existing reservation is Central; no duplicate reservation | 5 min | RQ-1, RQ-2 |
| T3 *(Example)* | "Two of your books are due soon. Keep them for longer." | Two loans due within 3 days | Both loans renewed, new due dates visible | 3 min | RQ-2 |
| T4 *(Example)* | "Find out until what time the Central branch is open this Saturday." | Home screen | Participant states the correct closing time | 1 min | RQ-2 |

Rules for writing tasks:

- Use the participant's language, not interface labels. "Keep them for longer" rather than "Renew".
- Give every task a realistic reason. A reason makes the goal concrete and the behaviour natural.
- Order tasks so that one task does not teach the next, or counterbalance the order between
  participants and record the order used.
- Decide what counts as a partial success, and whether moderator assistance turns a success into
  a failure. Recommended: any assistance that reveals the path counts as failure.

## 6. Measures and targets

<!-- Guidance: Measure each of the three usability components and state targets before fieldwork. A target written after seeing the data is not a target. -->

ISO 9241-11:2018 clause 6 describes usability outcomes as effectiveness (6.2), efficiency (6.3)
and satisfaction (6.4). Choose at least one measure for each and state how it is calculated.

| Measure ID | Component | Measure | How it is calculated | Target |
|---|---|---|---|---|
| M-01 *(Example)* | Effectiveness | Task completion rate | Successful attempts divided by attempts, per task | T1, T3, T4: at least 80%; T2: at least 70% |
| M-02 *(Example)* | Effectiveness | Errors | Count of wrong paths that needed a recovery step, per task | Recorded, no target in a formative round |
| M-03 *(Example)* | Efficiency | Time on task | Seconds from end of task reading to the success state, successful attempts only; report median | T4: median 60 s or less |
| M-04 *(Example)* | Efficiency | Assistance requests | Count of moderator interventions per task | Recorded |
| M-05 *(Example)* | Satisfaction | Single Ease Question (SEQ) | 7-point rating "Overall, this task was..." from very difficult to very easy, after each task | Recorded |
| M-06 *(Example)* | Satisfaction | System Usability Scale (SUS) | 10 items after the last task, scored 0 to 100 | Mean 68 or higher |
| M-07 *(Example)* | Other outcome | Accessibility barriers | Count of tasks blocked or seriously slowed for assistive technology users | 0 blocked tasks |

### 6.1 System Usability Scale scoring

The SUS is a 10-item questionnaire published by John Brooke (1996), answered on a 5-point scale
from strongly disagree (1) to strongly agree (5). Odd-numbered items are worded positively and
even-numbered items negatively. Use the published item wording from Brooke (1996) unchanged, in
the published order, and translate it only with a validated translation.

To score one participant:

1. For each odd item (1, 3, 5, 7, 9), subtract 1 from the response.
2. For each even item (2, 4, 6, 8, 10), subtract the response from 5.
3. Add the ten converted values (range 0 to 40) and multiply by 2.5 (range 0 to 100).

*Example:* responses 4, 2, 4, 1, 4, 2, 5, 2, 4, 2 convert to 3, 3, 3, 4, 3, 3, 4, 3, 3, 3, which
sum to 32, so the SUS score is 32 x 2.5 = 80.0.

A SUS score is not a percentage. An average across many published studies is about 68 (Sauro,
MeasuringU), so treat 68 as a reference point, not a pass mark. With small samples, report the
mean together with a confidence interval, and compare designs only when the protocols match.

### 6.2 Small-sample statistics

- For completion rates, report a 95% confidence interval. The adjusted Wald interval works well
  for small samples (Sauro and Lewis, *Quantifying the User Experience*, 2nd ed., 2016).
- Time on task is usually skewed: report the median or the geometric mean, not the arithmetic mean.
- In a formative round, use numbers to describe what happened, not to generalise to all users.

## 7. Session procedure and script

<!-- Guidance: Script the whole session so that every participant hears the same words. Time each block and leave a buffer; sessions that overrun cut the debrief, which is often the most useful part. -->

```mermaid
flowchart LR
  A["Welcome and consent check"] --> B["Background questions"]
  B --> C["Practice task"]
  C --> D["Tasks T1 to T4 with SEQ after each"]
  D --> E["SUS questionnaire"]
  E --> F["Debrief questions"]
  F --> G["Thank, incentive, withdrawal info"]
```

| Step | Time | Moderator script and actions |
|---|---|---|
| Welcome | 3 min | Introduce yourself and the observer. "We are testing the app, not you. There are no wrong answers." |
| Consent check | 3 min | Confirm the signed consent and each recording choice; start recording only if agreed; repeat consent on the recording. |
| Background | 5 min | "Tell me about the last time you borrowed something from the library." Note device and assistive technology settings. |
| Practice task | 2 min | A short unrelated task to practise thinking aloud. |
| Tasks T1 to T4 | 25 min | Read each scenario, hand over a written copy, start timing when reading ends. Do not help; if asked, reply "What would you do if I were not here?" Ask the SEQ after each task. |
| SUS | 3 min | Present the 10 items on a form; read aloud for participants who prefer it. |
| Debrief | 6 min | "What was the hardest part? What would you change first? Is there anything we did not ask about?" |
| Close | 3 min | Thank, confirm the incentive, repeat how to withdraw and the contact. |
| Total | 50 min | Book 60 minutes per session to leave a buffer. |

Moderator rules: stay neutral, do not explain the design, and avoid "Did you find that easy?"
style questions. When a participant is stuck past the time limit, end the task kindly, record it
as a failure and move on. If a participant becomes distressed, stop the session; follow section 10.

## 8. Environment, tools and data capture

<!-- Guidance: List every tool that will touch participant data, where recordings are stored, and the fallback if a tool fails. -->

| Item | Plan |
|---|---|
| Session tool | [*Example:* video call tool with screen sharing, cloud recording disabled, local recording to an encrypted research laptop] |
| Device | [*Example:* participant's own phone mirrored through the call] |
| Prototype or build | [*Example:* beta build 5.2.0 (build 412) with test catalogue data and test card numbers] |
| Observation | [*Example:* up to 2 observers in a separate channel, cameras off, no direct contact with the participant] |
| Note-taking | [*Example:* shared spreadsheet with one row per participant and task: outcome, time, errors, assistance, SEQ, quotes] |
| Questionnaire | [*Example:* SUS on a form tool that stores only the participant code] |
| Fallback | [*Example:* if mirroring fails, participant describes the screen while the moderator watches a camera view] |

Use participant codes (P01, P02 and so on) in every note, file name and spreadsheet. Keep the
key that links codes to names in a separate restricted location.

## 9. Roles and responsibilities

<!-- Guidance: Every session needs a moderator and a note-taker. Name a backup moderator. -->

| Role | Person | Responsibilities |
|---|---|---|
| Lead researcher | [@name] | Plan, consent process, analysis, report |
| Moderator | [@name] | Runs sessions, follows the script, ensures participant wellbeing |
| Note-taker | [@name] | Records outcomes, times and quotes in the template |
| Observers | [@names] | Watch and add observations; do not interrupt |
| Research operations | [@name] | Recruitment, scheduling, incentives, data deletion |
| Product owner | [@name] | Attends at least 2 sessions, joins the findings workshop |

## 10. Consent, ethics and personal data

<!-- Guidance: Plan consent, recording and retention before recruiting. This is a planning aid, not legal advice: check with your privacy lead and, where required, an ethics committee or institutional review board. -->

- Use design/research-consent-form for the participant information sheet and consent, with a
  separate yes or no for audio, video, screen recording and the use of clips.
- Data minimisation: record only what the analysis needs. A screen recording with audio is often
  enough; face video is rarely needed.
- Retention: [*Example:* raw recordings deleted 6 months after the report is approved; codes and
  pseudonymised notes kept with the report].
- Questions to settle with your privacy lead (dated examples, as of October 2026): in the EU and
  the UK, which GDPR Article 6 lawful basis applies, and will health or disability information
  shared by assistive technology users bring Article 9 into play? In the US, does a state privacy
  law such as the California Consumer Privacy Act apply, and does any partner (for example a
  university or a federally funded programme) require review under the Common Rule (45 CFR part 46)?
- Safeguarding: no participants under [18] unless a separate approved procedure exists.
- Not legal advice; check the current official text and a qualified professional.

## 11. Pilot and readiness checklist

<!-- Guidance: A pilot with a colleague who has not seen the design catches broken tasks, unclear wording and recording problems. Fix them before the first real session. -->

- [ ] Pilot session run on YYYY-MM-DD with the final build, script and tools.
- [ ] Every task has a reachable success state in the test data; test accounts reset between sessions.
- [ ] Task wording checked for interface labels and leading words.
- [ ] Timings confirmed: session fits the booked slot with a buffer.
- [ ] Recording, mirroring and the SUS form tested on iOS and Android.
- [ ] Screen reader path through T1 to T4 checked by an assistive technology user or specialist.
- [ ] Consent forms, incentive process and withdrawal contact ready.
- [ ] Note-taking sheet prepared with participant codes and task columns.

## 12. Analysis plan

<!-- Guidance: Decide the analysis before fieldwork so notes are collected in a usable form. Say how findings get a severity and who rates it. -->

1. After each session, the moderator and note-taker hold a 10-minute debrief and record the top
   observations with timestamps.
2. Score each task attempt against the success criteria in section 5. Disagreements are settled
   from the recording.
3. Calculate the measures in section 6: completion with 95% adjusted Wald intervals, median time
   for successful attempts, SEQ medians, SUS scores per participant, mean and 95% confidence interval.
4. Group observations into problems. Each problem gets an ID, the tasks and participants where it
   occurred, evidence (timestamps, quotes) and a severity on the 0 to 4 scale used in
   design/heuristic-evaluation (0 not a problem, 1 cosmetic, 2 minor, 3 major, 4 catastrophe).
   At least two team members rate severity independently and agree on a final rating.
5. Write recommendations per problem, then hold a findings workshop with the product owner.
6. Write the report in design/usability-test-report. For thematic analysis of the debrief
   answers, use design/research-synthesis.

## 13. Schedule and deliverables

<!-- Guidance: Work back from the decision date in section 1. Leave enough time between the last session and the decision for analysis. -->

| Milestone | Date | Owner |
|---|---|---|
| Plan approved | YYYY-MM-DD | [@product-owner] |
| Recruitment complete | YYYY-MM-DD | [@research-ops] |
| Pilot session | YYYY-MM-DD | [@moderator] |
| Sessions | YYYY-MM-DD to YYYY-MM-DD | [@moderator] |
| Topline summary (top 5 issues) | Within 2 working days of the last session | [@researcher] |
| Full report and findings workshop | YYYY-MM-DD | [@researcher] |
| Decision | YYYY-MM-DD | [@product-owner] |
| Raw recordings deleted | YYYY-MM-DD | [@research-ops] |

## 14. Risks and limitations

<!-- Guidance: Name the threats to validity in advance and what you will do about each. The report repeats the ones that materialised. -->

| ID | Risk or limitation | Effect | Mitigation |
|---|---|---|---|
| RISK-01 *(Example)* | Beta build crashes during a task | Lost data, frustrated participant | Pilot on both platforms; restart procedure in the script |
| RISK-02 *(Example)* | Test catalogue data looks unrealistic | Participants behave differently | Use realistic fictional titles and branch names |
| RISK-03 *(Example)* | Small sample per segment | Findings cannot be generalised per segment | Report as formative findings; no per-segment percentages |
| RISK-04 *(Example)* | Remote sessions miss the commuting context | Distraction effects not observed | Note in the report; consider a diary study later |
| RISK-05 *(Example)* | Observers influence the moderator | Leading questions | Observers in a separate channel; questions passed at the end only |

## 15. Glossary and references

<!-- Guidance: Keep definitions short and in your own words. Point to the official source rather than copying standard definitions. -->

| Term | Meaning in this plan |
|---|---|
| Context of use | The users, goals, tasks, resources and environments in which a product is used |
| Effectiveness | How accurately and completely users reach their goals |
| Efficiency | Resources spent, such as time and effort, in relation to the results achieved |
| Satisfaction | Users' physical, cognitive and emotional responses to the use |
| Formative test | A test to find and explain problems so the design can be improved |
| Summative test | A test to measure usability against targets or a benchmark |
| SEQ | Single Ease Question, a one-item post-task difficulty rating |
| SUS | System Usability Scale, a 10-item post-test questionnaire scored 0 to 100 |

References:

- ISO 9241-11:2018, Ergonomics of human-system interaction, Part 11: Usability: definitions and
  concepts, second edition, 2018-03 (confirmed at the 2023 systematic review). Clauses used:
  5.3 specified circumstances, 6.2 to 6.4 effectiveness, efficiency, satisfaction, 7 context of use.
  Official text: ISO, purchase or licensed access: https://www.iso.org/standard/63500.html
- ISO/IEC 25066:2016, Systems and software engineering, SQuaRE, Common Industry Format (CIF) for
  usability: evaluation report, first edition, published 2016-06-13. Clauses used: 4.1 and 4.2
  purpose and types of evaluation, 5.2 report content elements. ISO withdrew this edition on
  2026-04-24 (stage 95.99) without naming a replacement, as shown on ISO's committee catalogue
  page and in ISO open data, checked 2026-10-30; it is still a useful structure for reports, but
  do not claim conformance to a current standard. Official text: ISO, purchase or licensed
  access (withdrawn editions may be available on request): https://www.iso.org/standard/63831.html
- Brooke, J. (1996). SUS: a "quick and dirty" usability scale. In Jordan, P. W. et al. (eds.),
  *Usability Evaluation in Industry*. Taylor and Francis. Brooke, J. (2013). SUS: a retrospective.
  *Journal of Usability Studies*, 8(2), 29 to 40.
- Sauro, J. Measuring usability with the System Usability Scale (SUS), MeasuringU,
  https://measuringu.com/sus/
- Sauro, J. and Lewis, J. R. (2016). *Quantifying the User Experience*, 2nd edition. Morgan Kaufmann.
- Nielsen, J. (2000). Why you only need to test with 5 users. Nielsen Norman Group,
  https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/
- Related templates: design/ux-research-plan, design/research-consent-form,
  design/usability-test-report, design/heuristic-evaluation, design/accessibility-checklist,
  product/persona, product/ab-test-plan.

## Standard and licence note

Structure aligned with ISO 9241-11:2018 (usability outcomes and context of use) and with the
planning items that ISO/IEC 25066:2016 expects a usability evaluation report to describe; the SUS
scoring method is attributed to Brooke (1996) and the reference average to Sauro (MeasuringU).
ISO standards are paid publications: this template cites their names, editions and clause numbers
only, and all wording is original. Check the current official versions before relying on them.
Privacy notes are not legal advice. Template licence: MIT.