Usability Test Plan Template (ISO 9241-11, SUS)

Usability Test Plan: [Product or feature name]

A plan for one usability test, moderated or unmoderated: the decision it informs, the object being evaluated, its context of use, participants, task scenarios, measures of effectiveness, efficiency and satisfaction (including the System Usability Scale, SUS), the session procedure, data handling and the analysis that turns sessions into findings. Measures follow the usability framework of ISO 9241-11:2018 (second edition) and the planning items that ISO/IEC 25066:2016 (withdrawn by ISO in April 2026, see references) expects a usability test report to describe. Both are paid standards: this template cites their clause numbers and uses original wording. Replace every [bracketed] value and every row marked Example. Worked examples use Quillmere Libraries, a fictional city library service and its borrowing app. Privacy notes are a planning aid, not legal advice: check the current official text with your privacy lead.

Document control

Approvals

Revision history

Contents

1. Purpose, object of evaluation and decision
2. Context of use
3. Test type and method
4. Participants and recruitment
5. Task scenarios and success criteria
6. Measures and targets
7. Session procedure and script
8. Environment, tools and data capture
9. Roles and responsibilities
10. Consent, ethics and personal data
11. Pilot and readiness checklist
12. Analysis plan
13. Schedule and deliverables
14. Risks and limitations
15. Glossary and references

1. Purpose, object of evaluation and decision

A usability test is worth running only if its result can change something. Write the decision in one sentence, name the decision maker, and give the date after which the findings arrive too late. Then describe the object of evaluation precisely enough that a reader of the report a year from now knows what was in front of the participants: product name, version or build, whether it is a live service, a coded prototype or a clickable mock-up, and which parts are in or out of scope.

Evaluation purposes differ. ISO/IEC 25066:2016 clause 4.1 distinguishes the purpose of an evaluation and clause 4.2 the types of usability evaluation, including user observation and measurement against criteria. State which applies here, because a formative round of six participants cannot support the claims a summative benchmark can.

2. Context of use

ISO 9241-11:2018 treats usability as an outcome of use for specified users, goals and contexts (clauses 5.3 and 7). A test result therefore holds only for the context it reproduced. Describe the intended context here and, in the last column, how faithfully the test reproduces it. Every gap is a limitation to report.

3. Test type and method

Use this guide to the main choices:

• Formative testing finds and explains problems. Small rounds are normal; Nielsen (NN/g, 2000) suggests about five users per round for one user group as a rule of thumb, more when user groups differ. Report problems and their evidence, not percentages that a small sample cannot support.
• Summative testing measures against targets or a benchmark. It needs a stable build, a fixed protocol and a larger sample; Nielsen suggests about 20 users for quantitative metrics. Report confidence intervals, not just point estimates.
• Moderated sessions let the moderator ask follow-up questions, recover from technical problems and adapt for assistive technology users.
• Unmoderated sessions run on a testing platform with written tasks. Write tasks that need no clarification, add a clear end condition to each task, and plan for some unusable sessions.

4. Participants and recruitment

Recruitment rules:

• Exclude [Example: library staff, their families, and anyone who took part in a study in the last 6 months].
• Channel: [Example: opt-in research panel, in-branch poster with a screener link].
• Incentive: [Example: gift card of a fixed value per session, paid in full even if the participant stops early or withdraws].
• Over-recruit by [Example: 1 participant per segment] to cover no-shows.
• Ask about access needs in the screener: captions, extra time, breaks, a support person, use of the participant's own device and assistive technology settings.

5. Task scenarios and success criteria

Rules for writing tasks:

• Use the participant's language, not interface labels. "Keep them for longer" rather than "Renew".
• Give every task a realistic reason. A reason makes the goal concrete and the behaviour natural.
• Order tasks so that one task does not teach the next, or counterbalance the order between participants and record the order used.
• Decide what counts as a partial success, and whether moderator assistance turns a success into a failure. Recommended: any assistance that reveals the path counts as failure.

6. Measures and targets

ISO 9241-11:2018 clause 6 describes usability outcomes as effectiveness (6.2), efficiency (6.3) and satisfaction (6.4). Choose at least one measure for each and state how it is calculated.

6.1 System Usability Scale scoring

The SUS is a 10-item questionnaire published by John Brooke (1996), answered on a 5-point scale from strongly disagree (1) to strongly agree (5). Odd-numbered items are worded positively and even-numbered items negatively. Use the published item wording from Brooke (1996) unchanged, in the published order, and translate it only with a validated translation.

To score one participant:

1. For each odd item (1, 3, 5, 7, 9), subtract 1 from the response.
2. For each even item (2, 4, 6, 8, 10), subtract the response from 5.
3. Add the ten converted values (range 0 to 40) and multiply by 2.5 (range 0 to 100).

Example: responses 4, 2, 4, 1, 4, 2, 5, 2, 4, 2 convert to 3, 3, 3, 4, 3, 3, 4, 3, 3, 3, which sum to 32, so the SUS score is 32 x 2.5 = 80.0.

A SUS score is not a percentage. An average across many published studies is about 68 (Sauro, MeasuringU), so treat 68 as a reference point, not a pass mark. With small samples, report the mean together with a confidence interval, and compare designs only when the protocols match.

6.2 Small-sample statistics

• For completion rates, report a 95% confidence interval. The adjusted Wald interval works well for small samples (Sauro and Lewis, Quantifying the User Experience, 2nd ed., 2016).
• Time on task is usually skewed: report the median or the geometric mean, not the arithmetic mean.
• In a formative round, use numbers to describe what happened, not to generalise to all users.

7. Session procedure and script

Moderator rules: stay neutral, do not explain the design, and avoid "Did you find that easy?" style questions. When a participant is stuck past the time limit, end the task kindly, record it as a failure and move on. If a participant becomes distressed, stop the session; follow section 10.

8. Environment, tools and data capture

Use participant codes (P01, P02 and so on) in every note, file name and spreadsheet. Keep the key that links codes to names in a separate restricted location.

9. Roles and responsibilities

10. Consent, ethics and personal data

• Use design/research-consent-form for the participant information sheet and consent, with a separate yes or no for audio, video, screen recording and the use of clips.
• Data minimisation: record only what the analysis needs. A screen recording with audio is often enough; face video is rarely needed.
• Retention: [Example: raw recordings deleted 6 months after the report is approved; codes and pseudonymised notes kept with the report].
• Questions to settle with your privacy lead (dated examples, as of October 2026): in the EU and the UK, which GDPR Article 6 lawful basis applies, and will health or disability information shared by assistive technology users bring Article 9 into play? In the US, does a state privacy law such as the California Consumer Privacy Act apply, and does any partner (for example a university or a federally funded programme) require review under the Common Rule (45 CFR part 46)?
• Safeguarding: no participants under [18] unless a separate approved procedure exists.
• Not legal advice; check the current official text and a qualified professional.

11. Pilot and readiness checklist

• Pilot session run on YYYY-MM-DD with the final build, script and tools.
• Every task has a reachable success state in the test data; test accounts reset between sessions.
• Task wording checked for interface labels and leading words.
• Timings confirmed: session fits the booked slot with a buffer.
• Recording, mirroring and the SUS form tested on iOS and Android.
• Screen reader path through T1 to T4 checked by an assistive technology user or specialist.
• Consent forms, incentive process and withdrawal contact ready.
• Note-taking sheet prepared with participant codes and task columns.

12. Analysis plan

1. After each session, the moderator and note-taker hold a 10-minute debrief and record the top observations with timestamps.
2. Score each task attempt against the success criteria in section 5. Disagreements are settled from the recording.
3. Calculate the measures in section 6: completion with 95% adjusted Wald intervals, median time for successful attempts, SEQ medians, SUS scores per participant, mean and 95% confidence interval.
4. Group observations into problems. Each problem gets an ID, the tasks and participants where it occurred, evidence (timestamps, quotes) and a severity on the 0 to 4 scale used in design/heuristic-evaluation (0 not a problem, 1 cosmetic, 2 minor, 3 major, 4 catastrophe). At least two team members rate severity independently and agree on a final rating.
5. Write recommendations per problem, then hold a findings workshop with the product owner.
6. Write the report in design/usability-test-report. For thematic analysis of the debrief answers, use design/research-synthesis.

13. Schedule and deliverables

14. Risks and limitations

15. Glossary and references

References:

• ISO 9241-11:2018, Ergonomics of human-system interaction, Part 11: Usability: definitions and concepts, second edition, 2018-03 (confirmed at the 2023 systematic review). Clauses used: 5.3 specified circumstances, 6.2 to 6.4 effectiveness, efficiency, satisfaction, 7 context of use. Official text: ISO, purchase or licensed access: https://www.iso.org/standard/63500.html
• ISO/IEC 25066:2016, Systems and software engineering, SQuaRE, Common Industry Format (CIF) for usability: evaluation report, first edition, published 2016-06-13. Clauses used: 4.1 and 4.2 purpose and types of evaluation, 5.2 report content elements. ISO withdrew this edition on 2026-04-24 (stage 95.99) without naming a replacement, as shown on ISO's committee catalogue page and in ISO open data, checked 2026-10-30; it is still a useful structure for reports, but do not claim conformance to a current standard. Official text: ISO, purchase or licensed access (withdrawn editions may be available on request): https://www.iso.org/standard/63831.html
• Brooke, J. (1996). SUS: a "quick and dirty" usability scale. In Jordan, P. W. et al. (eds.), Usability Evaluation in Industry. Taylor and Francis. Brooke, J. (2013). SUS: a retrospective. Journal of Usability Studies, 8(2), 29 to 40.
• Sauro, J. Measuring usability with the System Usability Scale (SUS), MeasuringU, https://measuringu.com/sus/
• Sauro, J. and Lewis, J. R. (2016). Quantifying the User Experience, 2nd edition. Morgan Kaufmann.
• Nielsen, J. (2000). Why you only need to test with 5 users. Nielsen Norman Group, https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/
• Related templates: design/ux-research-plan, design/research-consent-form, design/usability-test-report, design/heuristic-evaluation, design/accessibility-checklist, product/persona, product/ab-test-plan.

Standard and licence note

Structure aligned with ISO 9241-11:2018 (usability outcomes and context of use) and with the planning items that ISO/IEC 25066:2016 expects a usability evaluation report to describe; the SUS scoring method is attributed to Brooke (1996) and the reference average to Sauro (MeasuringU). ISO standards are paid publications: this template cites their names, editions and clause numbers only, and all wording is original. Check the current official versions before relying on them. Privacy notes are not legal advice. Template licence: MIT.