Usability Test Plan Template (ISO 9241-11, SUS)

Usability test plan: decision, context of use, participants, task scenarios, effectiveness, efficiency, satisfaction, SUS scoring, script, analysis

डिज़ाइन और UXपूर्णISO 9241-11:2018; ISO/IEC 25066:2016usability test planअपडेट किया गया 29 अक्टू॰ 2026

एडिटर में खोलें.md डाउनलोड करें

उपयोग कैसे करें: Use it when you are about to run a moderated or unmoderated usability test. Fill the decision, the object of evaluation and the task scenarios with success criteria first, then the measures and targets. The main pitfall is task wording that names the interface controls and gives the answer away.

प्रीव्यू

Usability Test Plan: [Product or feature name]

A plan for one usability test, moderated or unmoderated: the decision it informs, the object being evaluated, its context of use, participants, task scenarios, measures of effectiveness, efficiency and satisfaction (including the System Usability Scale, SUS), the session procedure, data handling and the analysis that turns sessions into findings. Measures follow the usability framework of ISO 9241-11:2018 (second edition) and the planning items that ISO/IEC 25066:2016 (withdrawn by ISO in April 2026, see references) expects a usability test report to describe. Both are paid standards: this template cites their clause numbers and uses original wording. Replace every [bracketed] value and every row marked Example. Worked examples use Quillmere Libraries, a fictional city library service and its borrowing app. Privacy notes are a planning aid, not legal advice: check the current official text with your privacy lead.

Document control

Field	Value
Document ID	[UTP-001]
Version	[1.0]
Status	[Draft / Approved / Piloted / In fieldwork / Closed]
Owner (lead researcher)	[@researcher]
Approvers / reviewers	[@product-owner], [@design-lead], [@privacy-lead]
Last updated	YYYY-MM-DD
Classification	[Internal; contains no participant personal data]
Object of evaluation	[Product, version or build, prototype fidelity]
Test type	[Formative / Summative] and [Moderated in person / Moderated remote / Unmoderated remote]
Fieldwork window	YYYY-MM-DD to YYYY-MM-DD
Report	[design/usability-test-report, document ID UTR-001]
Related	[design/ux-research-plan], [design/research-consent-form], [design/heuristic-evaluation]

Approvals

Role	Name	Decision	Date
Product owner	[@name]	[Approved]	YYYY-MM-DD
Lead researcher	[@name]	[Approved]	YYYY-MM-DD
Privacy lead	[@name]	[Approved / Changes requested]	YYYY-MM-DD

Revision history

Version	Date	Author	Change
0.1	YYYY-MM-DD	[@name]	First draft
0.2	YYYY-MM-DD	[@name]	Tasks and success criteria revised after pilot
1.0	YYYY-MM-DD	[@name]	Approved for fieldwork

Purpose, object of evaluation and decision
Context of use
Test type and method
Participants and recruitment
Task scenarios and success criteria
Measures and targets
Session procedure and script
Environment, tools and data capture
Roles and responsibilities
Consent, ethics and personal data
Pilot and readiness checklist
Analysis plan
Schedule and deliverables
Risks and limitations
Glossary and references

1. Purpose, object of evaluation and decision

A usability test is worth running only if its result can change something. Write the decision in one sentence, name the decision maker, and give the date after which the findings arrive too late. Then describe the object of evaluation precisely enough that a reader of the report a year from now knows what was in front of the participants: product name, version or build, whether it is a live service, a coded prototype or a clickable mock-up, and which parts are in or out of scope.

Item	Value
Decision to inform	[Example: ship the redesigned reservation flow in release 5.2, or hold it for another iteration]
Decision maker and date	[Example: product owner, go or no-go meeting on YYYY-MM-DD]
Object of evaluation	[Example: Quillmere Libraries app 5.2 beta build, iOS and Android, test catalogue data]
In scope	[Example: search, reserve, change collection branch, renew loans, branch opening hours]
Out of scope	[Example: account registration, payments for fines, staff app]
Research questions	[Example: RQ-1 Can borrowers reserve and redirect a reservation unaided? RQ-2 Where do they hesitate or make errors? RQ-3 How do they rate the app overall?]

Evaluation purposes differ. ISO/IEC 25066:2016 clause 4.1 distinguishes the purpose of an evaluation and clause 4.2 the types of usability evaluation, including user observation and measurement against criteria. State which applies here, because a formative round of six participants cannot support the claims a summative benchmark can.

2. Context of use

ISO 9241-11:2018 treats usability as an outcome of use for specified users, goals and contexts (clauses 5.3 and 7). A test result therefore holds only for the context it reproduced. Describe the intended context here and, in the last column, how faithfully the test reproduces it. Every gap is a limitation to report.

Component (ISO 9241-11:2018 clause)	Intended context	Reproduced in the test
Users (7.2)	[Example: adult borrowers, from weekly users to occasional; includes screen reader users and people aged 65 and over]	[Example: 8 participants across three segments, see section 4]
Goals (7.3)	[Example: get a book ready to collect at a convenient branch with little effort]	[Example: expressed as four scenarios, see section 5]
Tasks (7.4)	[Example: search, reserve, redirect a reservation, renew, check opening hours]	[Example: T1 to T4]
Resources (7.5)	[Example: own smartphone, library card number, about five minutes]	[Example: participant's own phone; test card numbers supplied]
Technical environment (7.6.2)	[Example: mobile data or home Wi-Fi, iOS 17 or later, Android 12 or later]	[Example: participant's network; beta build via test distribution]
Physical environment (7.6.3)	[Example: at home, on a bus, in a branch]	[Example: at home, remote session]
Social, cultural and organisational environment (7.6.4)	[Example: often done while commuting, sometimes for a family member]	[Example: not reproduced, note as limitation]

3. Test type and method

Choice	Selected	Rationale
Formative or summative	[Example: Formative]	[Example: the flow is still changing; the goal is to find and fix problems before release]
Moderated or unmoderated	[Example: Moderated remote]	[Example: need to probe hesitation and support screen reader users live]
Think-aloud	[Example: Concurrent think-aloud for T1 to T3, silent for T4]	[Example: T4 time is compared with a target, and talking slows people down]
Rounds	[Example: 1 round of 8; a second round after fixes if severity 3 or 4 issues are found]	[Example: iterative testing finds more problems than one large round]
Comparison	[Example: none]	[Example: only one design exists; for an A versus B decision see product/ab-test-plan]

Use this guide to the main choices:

Formative testing finds and explains problems. Small rounds are normal; Nielsen (NN/g, 2000) suggests about five users per round for one user group as a rule of thumb, more when user groups differ. Report problems and their evidence, not percentages that a small sample cannot support.
Summative testing measures against targets or a benchmark. It needs a stable build, a fixed protocol and a larger sample; Nielsen suggests about 20 users for quantitative metrics. Report confidence intervals, not just point estimates.
Moderated sessions let the moderator ask follow-up questions, recover from technical problems and adapt for assistive technology users.
Unmoderated sessions run on a testing platform with written tasks. Write tasks that need no clarification, add a clear end condition to each task, and plan for some unusable sessions.

4. Participants and recruitment

Segment	Definition	Target	Screener question
S1 (Example)	Frequent borrowers: borrowed 6 or more items in the last 6 months	3	"How many items have you borrowed in the last 6 months?"
S2 (Example)	Occasional borrowers: 1 to 5 items in the last 6 months	3	Same question
S3 (Example)	Screen reader or screen magnifier users who borrow at least occasionally	2	"Do you use any assistive technology on your phone? Which?"
Total		8

Recruitment rules:

Exclude [Example: library staff, their families, and anyone who took part in a study in the last 6 months].
Channel: [Example: opt-in research panel, in-branch poster with a screener link].
Incentive: [Example: gift card of a fixed value per session, paid in full even if the participant stops early or withdraws].
Over-recruit by [Example: 1 participant per segment] to cover no-shows.
Ask about access needs in the screener: captions, extra time, breaks, a support person, use of the participant's own device and assistive technology settings.

5. Task scenarios and success criteria

Task ID	Scenario (read to the participant)	Start state	Success criterion	Time limit	Research question
T1 (Example)	"A friend recommended [fictional title]. Make sure you can pick it up at the library nearest your home."	Logged in, home screen	Reservation exists for the title with the nearest branch as collection point	3 min	RQ-1, RQ-2
T2 (Example)	"You will be near the Central branch next week instead. Arrange to collect the book you reserved there."	One existing reservation at another branch	Collection branch of the existing reservation is Central; no duplicate reservation	5 min	RQ-1, RQ-2
T3 (Example)	"Two of your books are due soon. Keep them for longer."	Two loans due within 3 days	Both loans renewed, new due dates visible	3 min	RQ-2
T4 (Example)	"Find out until what time the Central branch is open this Saturday."	Home screen	Participant states the correct closing time	1 min	RQ-2

Rules for writing tasks:

Use the participant's language, not interface labels. "Keep them for longer" rather than "Renew".
Give every task a realistic reason. A reason makes the goal concrete and the behaviour natural.
Order tasks so that one task does not teach the next, or counterbalance the order between participants and record the order used.
Decide what counts as a partial success, and whether moderator assistance turns a success into a failure. Recommended: any assistance that reveals the path counts as failure.

6. Measures and targets

ISO 9241-11:2018 clause 6 describes usability outcomes as effectiveness (6.2), efficiency (6.3) and satisfaction (6.4). Choose at least one measure for each and state how it is calculated.

Measure ID	Component	Measure	How it is calculated	Target
M-01 (Example)	Effectiveness	Task completion rate	Successful attempts divided by attempts, per task	T1, T3, T4: at least 80%; T2: at least 70%
M-02 (Example)	Effectiveness	Errors	Count of wrong paths that needed a recovery step, per task	Recorded, no target in a formative round
M-03 (Example)	Efficiency	Time on task	Seconds from end of task reading to the success state, successful attempts only; report median	T4: median 60 s or less
M-04 (Example)	Efficiency	Assistance requests	Count of moderator interventions per task	Recorded
M-05 (Example)	Satisfaction	Single Ease Question (SEQ)	7-point rating "Overall, this task was..." from very difficult to very easy, after each task	Recorded
M-06 (Example)	Satisfaction	System Usability Scale (SUS)	10 items after the last task, scored 0 to 100	Mean 68 or higher
M-07 (Example)	Other outcome	Accessibility barriers	Count of tasks blocked or seriously slowed for assistive technology users	0 blocked tasks

6.1 System Usability Scale scoring

The SUS is a 10-item questionnaire published by John Brooke (1996), answered on a 5-point scale from strongly disagree (1) to strongly agree (5). Odd-numbered items are worded positively and even-numbered items negatively. Use the published item wording from Brooke (1996) unchanged, in the published order, and translate it only with a validated translation.

To score one participant:

For each odd item (1, 3, 5, 7, 9), subtract 1 from the response.
For each even item (2, 4, 6, 8, 10), subtract the response from 5.
Add the ten converted values (range 0 to 40) and multiply by 2.5 (range 0 to 100).

Example: responses 4, 2, 4, 1, 4, 2, 5, 2, 4, 2 convert to 3, 3, 3, 4, 3, 3, 4, 3, 3, 3, which sum to 32, so the SUS score is 32 x 2.5 = 80.0.

A SUS score is not a percentage. An average across many published studies is about 68 (Sauro, MeasuringU), so treat 68 as a reference point, not a pass mark. With small samples, report the mean together with a confidence interval, and compare designs only when the protocols match.

6.2 Small-sample statistics

For completion rates, report a 95% confidence interval. The adjusted Wald interval works well for small samples (Sauro and Lewis, Quantifying the User Experience, 2nd ed., 2016).
Time on task is usually skewed: report the median or the geometric mean, not the arithmetic mean.
In a formative round, use numbers to describe what happened, not to generalise to all users.

7. Session procedure and script

Step	Time	Moderator script and actions
Welcome	3 min	Introduce yourself and the observer. "We are testing the app, not you. There are no wrong answers."
Consent check	3 min	Confirm the signed consent and each recording choice; start recording only if agreed; repeat consent on the recording.
Background	5 min	"Tell me about the last time you borrowed something from the library." Note device and assistive technology settings.
Practice task	2 min	A short unrelated task to practise thinking aloud.
Tasks T1 to T4	25 min	Read each scenario, hand over a written copy, start timing when reading ends. Do not help; if asked, reply "What would you do if I were not here?" Ask the SEQ after each task.
SUS	3 min	Present the 10 items on a form; read aloud for participants who prefer it.
Debrief	6 min	"What was the hardest part? What would you change first? Is there anything we did not ask about?"
Close	3 min	Thank, confirm the incentive, repeat how to withdraw and the contact.
Total	50 min	Book 60 minutes per session to leave a buffer.

Moderator rules: stay neutral, do not explain the design, and avoid "Did you find that easy?" style questions. When a participant is stuck past the time limit, end the task kindly, record it as a failure and move on. If a participant becomes distressed, stop the session; follow section 10.

8. Environment, tools and data capture

Item	Plan
Session tool	[Example: video call tool with screen sharing, cloud recording disabled, local recording to an encrypted research laptop]
Device	[Example: participant's own phone mirrored through the call]
Prototype or build	[Example: beta build 5.2.0 (build 412) with test catalogue data and test card numbers]
Observation	[Example: up to 2 observers in a separate channel, cameras off, no direct contact with the participant]
Note-taking	[Example: shared spreadsheet with one row per participant and task: outcome, time, errors, assistance, SEQ, quotes]
Questionnaire	[Example: SUS on a form tool that stores only the participant code]
Fallback	[Example: if mirroring fails, participant describes the screen while the moderator watches a camera view]

Use participant codes (P01, P02 and so on) in every note, file name and spreadsheet. Keep the key that links codes to names in a separate restricted location.

9. Roles and responsibilities

Role	Person	Responsibilities
Lead researcher	[@name]	Plan, consent process, analysis, report
Moderator	[@name]	Runs sessions, follows the script, ensures participant wellbeing
Note-taker	[@name]	Records outcomes, times and quotes in the template
Observers	[@names]	Watch and add observations; do not interrupt
Research operations	[@name]	Recruitment, scheduling, incentives, data deletion
Product owner	[@name]	Attends at least 2 sessions, joins the findings workshop

10. Consent, ethics and personal data

Use design/research-consent-form for the participant information sheet and consent, with a separate yes or no for audio, video, screen recording and the use of clips.
Data minimisation: record only what the analysis needs. A screen recording with audio is often enough; face video is rarely needed.
Retention: [Example: raw recordings deleted 6 months after the report is approved; codes and pseudonymised notes kept with the report].
Questions to settle with your privacy lead (dated examples, as of October 2026): in the EU and the UK, which GDPR Article 6 lawful basis applies, and will health or disability information shared by assistive technology users bring Article 9 into play? In the US, does a state privacy law such as the California Consumer Privacy Act apply, and does any partner (for example a university or a federally funded programme) require review under the Common Rule (45 CFR part 46)?
Safeguarding: no participants under [18] unless a separate approved procedure exists.
Not legal advice; check the current official text and a qualified professional.

11. Pilot and readiness checklist

Pilot session run on YYYY-MM-DD with the final build, script and tools.
Every task has a reachable success state in the test data; test accounts reset between sessions.
Task wording checked for interface labels and leading words.
Timings confirmed: session fits the booked slot with a buffer.
Recording, mirroring and the SUS form tested on iOS and Android.
Screen reader path through T1 to T4 checked by an assistive technology user or specialist.
Consent forms, incentive process and withdrawal contact ready.
Note-taking sheet prepared with participant codes and task columns.

12. Analysis plan

After each session, the moderator and note-taker hold a 10-minute debrief and record the top observations with timestamps.
Score each task attempt against the success criteria in section 5. Disagreements are settled from the recording.
Calculate the measures in section 6: completion with 95% adjusted Wald intervals, median time for successful attempts, SEQ medians, SUS scores per participant, mean and 95% confidence interval.
Group observations into problems. Each problem gets an ID, the tasks and participants where it occurred, evidence (timestamps, quotes) and a severity on the 0 to 4 scale used in design/heuristic-evaluation (0 not a problem, 1 cosmetic, 2 minor, 3 major, 4 catastrophe). At least two team members rate severity independently and agree on a final rating.
Write recommendations per problem, then hold a findings workshop with the product owner.
Write the report in design/usability-test-report. For thematic analysis of the debrief answers, use design/research-synthesis.

13. Schedule and deliverables

Milestone	Date	Owner
Plan approved	YYYY-MM-DD	[@product-owner]
Recruitment complete	YYYY-MM-DD	[@research-ops]
Pilot session	YYYY-MM-DD	[@moderator]
Sessions	YYYY-MM-DD to YYYY-MM-DD	[@moderator]
Topline summary (top 5 issues)	Within 2 working days of the last session	[@researcher]
Full report and findings workshop	YYYY-MM-DD	[@researcher]
Decision	YYYY-MM-DD	[@product-owner]
Raw recordings deleted	YYYY-MM-DD	[@research-ops]

14. Risks and limitations

ID	Risk or limitation	Effect	Mitigation
RISK-01 (Example)	Beta build crashes during a task	Lost data, frustrated participant	Pilot on both platforms; restart procedure in the script
RISK-02 (Example)	Test catalogue data looks unrealistic	Participants behave differently	Use realistic fictional titles and branch names
RISK-03 (Example)	Small sample per segment	Findings cannot be generalised per segment	Report as formative findings; no per-segment percentages
RISK-04 (Example)	Remote sessions miss the commuting context	Distraction effects not observed	Note in the report; consider a diary study later
RISK-05 (Example)	Observers influence the moderator	Leading questions	Observers in a separate channel; questions passed at the end only

15. Glossary and references

Term	Meaning in this plan
Context of use	The users, goals, tasks, resources and environments in which a product is used
Effectiveness	How accurately and completely users reach their goals
Efficiency	Resources spent, such as time and effort, in relation to the results achieved
Satisfaction	Users' physical, cognitive and emotional responses to the use
Formative test	A test to find and explain problems so the design can be improved
Summative test	A test to measure usability against targets or a benchmark
SEQ	Single Ease Question, a one-item post-task difficulty rating
SUS	System Usability Scale, a 10-item post-test questionnaire scored 0 to 100

References:

ISO 9241-11:2018, Ergonomics of human-system interaction, Part 11: Usability: definitions and concepts, second edition, 2018-03 (confirmed at the 2023 systematic review). Clauses used: 5.3 specified circumstances, 6.2 to 6.4 effectiveness, efficiency, satisfaction, 7 context of use. Official text: ISO, purchase or licensed access: https://www.iso.org/standard/63500.html
ISO/IEC 25066:2016, Systems and software engineering, SQuaRE, Common Industry Format (CIF) for usability: evaluation report, first edition, published 2016-06-13. Clauses used: 4.1 and 4.2 purpose and types of evaluation, 5.2 report content elements. ISO withdrew this edition on 2026-04-24 (stage 95.99) without naming a replacement, as shown on ISO's committee catalogue page and in ISO open data, checked 2026-10-30; it is still a useful structure for reports, but do not claim conformance to a current standard. Official text: ISO, purchase or licensed access (withdrawn editions may be available on request): https://www.iso.org/standard/63831.html
Brooke, J. (1996). SUS: a "quick and dirty" usability scale. In Jordan, P. W. et al. (eds.), Usability Evaluation in Industry. Taylor and Francis. Brooke, J. (2013). SUS: a retrospective. Journal of Usability Studies, 8(2), 29 to 40.
Sauro, J. Measuring usability with the System Usability Scale (SUS), MeasuringU, https://measuringu.com/sus/
Sauro, J. and Lewis, J. R. (2016). Quantifying the User Experience, 2nd edition. Morgan Kaufmann.
Nielsen, J. (2000). Why you only need to test with 5 users. Nielsen Norman Group, https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/
Related templates: design/ux-research-plan, design/research-consent-form, design/usability-test-report, design/heuristic-evaluation, design/accessibility-checklist, product/persona, product/ab-test-plan.

Standard and licence note

Structure aligned with ISO 9241-11:2018 (usability outcomes and context of use) and with the planning items that ISO/IEC 25066:2016 expects a usability evaluation report to describe; the SUS scoring method is attributed to Brooke (1996) and the reference average to Sauro (MeasuringU). ISO standards are paid publications: this template cites their names, editions and clause numbers only, and all wording is original. Check the current official versions before relying on them. Privacy notes are not legal advice. Template licence: MIT.