Test objective
Write the specific question this test will answer. 'Can first-time users complete onboarding without help?' is a test objective. 'Understand the user experience' is not. The objective determines everything else — the tasks, the success criteria, and the metrics.
Tasks
Write three to five tasks participants will attempt. Each task should be: written in the participant's language (not product language), scenario-framed ('You've just joined a new project. Find where your team's previous boards are stored.'), and completable without the moderator's help. Write the tasks in the order participants will attempt them — logical progression reduces cognitive load.
Success criteria
For each task: what counts as success? Completion without assistance? Completion in under 90 seconds? Correct navigation path? Write the success condition explicitly. Without this, observers will disagree about what they saw.
Metrics
What will you measure? Task completion rate, time on task, number of errors, and System Usability Scale (SUS) score are the most common. Write the metric and the method for capturing it. If someone needs to take notes during the session, assign a note-taker.
Observer protocol
Who is observing? What are they capturing? How do they communicate questions to the moderator without interrupting? Write the observer roles and the communication protocol. Observers who don't have a defined capture role generate unhelpful ad-hoc commentary.