Usability Testing
One-on-one usability testing sessions with users work best when the participant actually uses the document to find and understand information.
Usability testing is the best technique for documents where people have to find the information before understanding it. (See National Cancer Institute testing success)
When should I test?
You can conduct usability testing at any time that you have a draft. After you make changes based on the first round of usability testing, you can conduct a second round to see if your changes solved the problems you found without introducing new problems.
Who should I test?
You need to find three people to test your website or document.
Identify who your intended readers are. For example, individuals searching for medical information; taxpayers and tax professionals looking for forms; travelers wanting a passport.
Develop simple criteria and find three people who match them. For example, for travelers, the criteria might be: Adult U.S. citizens who haven’t applied for or renewed a passport lately. The criteria don’t need to be complicated.
Using your network of colleagues, friends, and family, find three people who, more or less, meet the criteria and will give you an hour of their time. Don’t use members of your own team, but employees from a different team down the hall may be fine.
You’re not required to get any special permission to do a usability test with only three people. Set aside a morning to conduct your test, and give each of the volunteers an appointment, one hour apart from each other.
What happens in a typical session?
A typical usability test session lasts about one hour with these parts:
- Introduction. You make the participant comfortable, explain what will happen, and ask a few questions about the person to understand their relevant experience.
- Scenarios. You give the participant very short stories suggesting they have a need for specific information and then you watch and listen as they find that information and tell you what they understand from what they found.
An example of a scenario for the FAA web site might be:
You have a private pilot's license and you just moved to a new city. Find out if you need to tell FAA about your new address. If you do, find out how to do that.
You can also ask participants for their own scenarios. What would they come to the document you are testing to find out? Then watch and listen as they look for and try to understand the information.
Typically, you ask people to "think aloud" as they work so you hear their words for what they are looking for and you hear how they understand what they find. - Debriefing. At the end, you can ask neutral questions about the experience and follow up about any specific words or phrases.
What variations are there?
Variations on the one-on-one usability test:
- Two people working together (co-discovery). Their discussion is an easy form of think aloud.
- Several people working independently at the same time followed by a group discussion. This speeds up the time you spend in usability test sessions, but it only works if you have several usability test note-takers so you have someone watching and listening to each participant before you bring all the participants together for the discussion.
- Comparative usability tests. You can include different versions of your document. Because you have a small number of people, it is best to have each person work with both versions. You have to alternate which version people start with.(See Controlled Comparative Studies)
- Remote moderated usability testing. With web-based tools, you do not have to be in the same place as the participant. These tools allow you to draw participants from a wide geographic range without travel costs.
- Remote unmoderated usability testing. You can have large numbers of people participate through remote testing tools. (For federal agencies, this may require clearance through OMB.)
Where can I learn more?
Almost anyone can conduct a simple usability test and fix problems that you see the volunteers encounter. Use these resources to help learn how.
Books
Barnum, Carol, Usability Testing Essentials: Ready, Set...Test!, Morgan-Kaufmann/Elsevier, 2011
Chisnell, Dana, and Rubin, Jeff. The Handbook of Usability Testing, 2nd edition, Wiley, May 2008.
Krug, Steve. Rocket Surgery Made Easy, New Riders, 2009.
Courses
Web Manager University (WMU) offers webinars, seminars, and one- and two-day courses in usability topics.
"Conducting Usability Testing in the Wild"
presented by Dana Chisnell (free archived WMU webinar)
Conferences
Usability Professionals' Association Annual Conference
Nielsen Norman Group conferences
User Interface Engineering holds an annual conference and training events
Additional resources:
Usability Professionals Association has local chapters which offer training and networking events