User Testing Roles
Test participant:
The user who is interacting with the system.
Operator:
In paper prototyping, the operator is responsible for changing the prototype based on the participant’s action, usually by bringing up new pages as they are requested. The operator needs to have a firm understanding of the IA of the site in order to know which pages to bring up – a sitemap can help with this.
In other prototyping methods, the operator is responsible for resetting the test environment between tests.
The operator should avoid talking and should not reveal how many pages there are.
Moderator:
The moderator is the only one on the team who is allowed to talk. The moderator’s job is to make the participant feel at home, explain the procedure, the task goal and to answer any questions the participant has.
During the test, the moderator should smile and try to keep the session relaxed. If the participant is struggling, the moderator can offer words of encouragement (eg. “You’re doing well” or “You’re helping us discover problems we never realised we had”), but should avoid leading the participant (eg. “Why don’t you try clicking on this?”). The moderator is also responsible for ending the session if the participant is becoming too stressed.
Observer / notetaker:
The observers should be completely silent and should avoid influencing the participant in any way – avoid any outward signs of frustration, exasperation or happiness. Ideally the observer/s are located out of sight of the participant. In the event that the observers want to ask a question or get a more detailed response from the participant, these questions should be written down and given to the moderator to ask after the test session.
The observer’s primary role is to take notes about how the user is interacting with the system.
Metrics (what are we testing for?)
Successful Task Completion:
In user testing, a task goal is what you want your participant to do. This might be to complete some task (eg. post a comment) or to find the answer to a question (eg. what time does the store open?). You may want to give participants multiple-choice questions. This metric measures whether your participant completes the task goal correctly.
Critical Errors:
Errors that prevent the participant from completing the task successfully. For example, this might mean the participant is unable to find the required information, or reports the incorrect information. The participant may or may not be aware that the task goal is incorrect or incomplete.
Non-Critical Errors:
Errors that are recovered by the participant and do not prevent the participant being able to complete the task. These errors result in the task being completed less efficiently. For example, exploratory behaviors such as opening the wrong navigation menu item or using a control incorrectly are non-critical errors.
Error-Free Rate:
The percentage of test participants who complete the task without any errors.
Time On Task:
The amount of time it takes the participant to complete the task.
Subjective Measures:
For example, satisfaction, ease of use, ease of finding information. Self-reported by the participant, usually on a 5 or 7 point Likert scale.
Likes, Dislikes and Recommendations:
Participants provide what they liked most about the site, what they liked least about the site, and recommendations for improving the site.
No comments:
Post a Comment