Usability Testing

Introduction | Test Plan | Preparation & Testing | Data Analyses & Report

Test Plan

Topics on This Page

• Develop the Test Plan
• Plan Elements

Develop the Test Plan

The first step in each round of usability testing is to develop a plan for the test. The purpose of the plan is to document what you are going to do, how you are going to conduct the test, what metrics you are going to capture, number of participants you are going to test, and what scenarios you will use.

Typically, the usability specialist meets with the rest of the team to decide on the major elements of the plan. Often, the usability specialist then drafts the plan, which circulates to management and the rest of the team. Once everyone has commented and a final plan is negotiated, the usability specialist revises the written plan to reflect the final decisions.

Plan Elements

You will need to include these elements in the usability test plan.

Scope

Indicate what you are testing: Give the name of the Web site, Web application, or other product. Also specify how much of the product the test will cover (e.g. the prototype as of a specific date; the navigation; navigation and content).

Purpose

Identify the concerns, questions, and goals for this test. These can be quite broad; for example, "Can users navigate to important information from the prototype's home page?" They can be quite specific; for example, "Will users easily find the search box in its present location?"

In each round of testing, you will probably have several general and several specific concerns to focus on. Your concerns should drive the scenarios you choose for the usability test.

Schedule & Location

Indicate when and where you will do the test. You may want to be specific about the schedule. For example, you may want to indicate how many sessions you will hold in a day and exactly what times the sessions will be. (You will need that information to recruit participants, so you must decide the schedule early on.)

Sessions

You will want to describe the sessions, the length of the sessions (typically one hour to 90 minutes). When scheduling participants, remember to leave yourself a little time between session to reset the environment and to briefly review the session with observer.

Equipment

Indicate the type of equipment you will be using in the test. Include information about the computer system, monitor size and resolution, operating system, browser etc. Also indicate if you are planning on videotaping or audio taping the test sessions or using any special usability testing tools.

Participants

Indicate the number and types of participants to be tested you will be recruiting. Describe how these participants are recruited.

Scenarios

Indicate the number and types of tasks included in testing. Typically you will end up with approximately 10 scenarios. You may want to include more in the test plan so the team can choose the appropriate tasks.

Metrics

Subjective metrics: Include the questions you are going to ask the participants prior to the sessions (e.g., background questionnaire), after each task scenario is completed (satisfaction questions about interface facilitations of the task), and overall satisfaction questions when the sessions is completed.

Quantitative metrics: Indicate the quantitative data you will be measuring in your test (e.g., successful completion rates, error rates, time on task).

Roles

Include a list of the staff who will participate in the usability testing and what role each will have. The usability specialist should be the facilitator of the sessions. The usability team may also provide the primary note-taker. Other team members should be expected to participate as observers and, perhaps, as note-takers.