1. "Is This a Toolkit?" Checklist

2. Toolkit Content Checklist

3. Toolkit Usability Checklist

4. Tool Checklist

√

√

1. "Is This a Toolkit?" Checklist

A "toolkit" is an action-oriented compilation of related information, resources, or tools that together can guide users to develop a plan or organize efforts to conform to evidence-based recommendations or meet evidence-based specific practice standards.

Toolkits are effective for presenting action-oriented recommendations, but they are not appropriate for all research. At the beginning of your project, and throughout the process, address these checklist questions to determine if a toolkit is the proper way to share your research findings.

To proceed with a toolkit format, make sure you have answered "yes" to each item in the checklist. If you answer "no" to any of the items, brainstorm with your team about how to resolve any concerns. You may also want to consider pursuing another stand-alone product, such as a research paper or a fact sheet.

1. What behavior or action are you trying to promote?
      _____________________________________________________________________________
      _____________________________________________________________________________
      _____________________________________________________________________________

2. Why is a package of tools the best way to attain your goal (as opposed to a research paper, fact sheet, or other stand-alone product?
      _____________________________________________________________________________
      _____________________________________________________________________________
      _____________________________________________________________________________

3. Have you verified that a product like this does not already exist?

4. Did your research generate multiple action-oriented tools (such as specific procedures, protocols, or other structured activities) that, working together, can help users develop a plan or organize efforts to conform to evidence-based practice?

___ Yes ___ No

5. Have you spoken to potential users to determine the demand for a product like this one? How do you know there's a demand? What research or data supports this demand?
      _________________________________________________________________________
      _________________________________________________________________________
      _________________________________________________________________________

___ Yes ___ No

2. Toolkit Content Checklist

Toolkits should have a standard format and look as well as similar types of information to easily guide users through a process of change. Answer the content checklist questions for your toolkit and tools to determine if you have provided users with sufficient information to implement the changes in behavior that your research recommends.

To proceed, make sure you have answered "yes" to each item in the checklist. If you answer "no" to any of the items, brainstorm with your team about how to resolve any concerns.

1. Are the toolkit and tools based on tasks?

___ Yes  ___ No

• Does the toolkit provide sequential steps users should follow?

___ Yes  ___ No

• Does the toolkit provide examples of how others have used the toolkit or tools successfully?

___ Yes  ___ No

2. Does the first page of the toolkit state its purpose?

3. Does the first page of the toolkit explain how to use the toolkit?

4. Does the first page of the toolkit list each tool and its purpose?

5. Does the first page of the toolkit describe target users and address their differing goals in using the toolkit (Go to the Toolkit Overview Checklist)?

___ Yes  ___ No

___ Yes  ___ No

___ Yes  ___ No

___ Yes  ___ No

6. Do the tools provide necessary information regarding what users need to complete tasks, such as:

• Staff time.
• Staff skills.
• Materials.
• Equipment.
• Administrative clearances and approvals.

___ Yes  ___ No

7. Does the toolkit organize resources to achieve a goal through specific actions (assign responsibilities, create a schedule, document progress, and ensure accountability)?

___ Yes  ___ No

8. Does each tool give adequate instruction on how to use it (e.g., collect and analyze data, interpret results, implement suggestions, and assess impact)?

___ Yes  ___ No

9. Does the toolkit provide users with additional resources for more information?

___ Yes  ___ No

3. Toolkit Usability Checklist

Think deliberately about the toolkit and its components: the potential users, the users' goals, the toolkit's look, expert validation, and measures of success. This checklist will help you and your team to plan a well-designed, usable toolkit.

To proceed, make sure you have answered "yes" to each item in the checklist. If you answer "no" to any of the items, brainstorm with your team about how to resolve any concerns or consider pursuing an alternative method for disseminating your research results.¹

Users

1. Have you identified your target group of users and taken them into account when designing the toolkit? How?
      _________________________________________________________________________
      _________________________________________________________________________
      _________________________________________________________________________

___ Yes  ___ No

2. Is there more than one target group of users?

___ Yes  ___ No

• Will they have different goals?
• What are some of those goals?

      _________________________________________________________________________
      _________________________________________________________________________

___ Yes  ___ No

3. Have you explained how different users can adapt the toolkit to suit their needs?

___ Yes ___No

4. What tasks do you want users to accomplish with each tool?

5. Are target users familiar with the toolkit's concepts and terminology?

___ Yes  ___ No

Style

1. How is this toolkit presented?

___ Web site                           ___ Printed Document

___ Video (CD/DVD)             ___ Audio (Digital, Web-based)

___ Other:                              

• AHRQ has style guides for many different kinds of presentations that include information elements, such as font size, typeface, and color. Does the toolkit comply with AHRQ Publishing and Communications Guidelines for that presentation style?

___ Yes  ___ No

2. Layering, or page sequence, in a Web-based environment reflects the hierarchy of organization in a document. Primary information is on the first level, and secondary information is on the second level of the Web site.

• If your toolkit is Web-based, does your critical information appear on the primary level?

___ Yes  ___ No

___ Not Applicable

• Is your secondary information linked so it is supplementary to the first level of information?

___ Yes  ___ No

___ Not Applicable

3. Does the toolkit as a whole have a cohesive, AHRQ-branded look and follow the design specifications laid out in the AHRQ Publishing and Communications Guidelines?

___ Yes  ___ No

Testing and Evaluation

1. Will the toolkit be tested before it is published?

___ Yes  ___ No

• How will the toolkit be tested?

___ Expert review                  ___ Focus groups with users

___ Usability testing            ___ Other:

2. How can users measure the impact of the toolkit within their organization? What are the measures of success? (Suggest 3-5 methods of impact measurement.)

      _____________________________________________________________________________
      _____________________________________________________________________________
      _____________________________________________________________________________

Implementation

1. If the toolkit requires updates, will you, the developers, perform those updates?

___ Yes  ___ No
___ Not Applicable

2. If the toolkit is a Web site and has the capacity to serve as a "live" resource for users (e.g., communities of practice for those with shared interests, bulletin boards, networking sites), which organization will provide ongoing quality oversight and technical support?
      _________________________________________________________________________
      _________________________________________________________________________
      _________________________________________________________________________

4. Tool Checklist

This checklist addresses three areas: organization, design, and language.

To proceed, make sure you have answered "yes" to each item in the checklist. If you answer "somewhat" or "no" to any of the items, brainstorm with your team about how to resolve any concerns.

Organization Considerations

Cohesive and logical organization helps users navigate the tool, improves comprehension, and encourages use. Answer the questions in this section to determine how successfully your toolkit aids users in finding and using the information presented.

1. Does the tool have an advance organizer, such as a table of contents or site map?

___ Yes  ___ No

• Does the advance organizer provide a coherent, complete "big picture" view of the tool?

___ Yes  ___ No

___ Somewhat

2. Does the tool have an organizational hierarchy and a clear structure of main topics and sub-topics?

___ Yes  ___ No

___ Somewhat

• Restricting your hierarchy to fewer than five levels makes it easier for users to navigate the tool. Does the tool have fewer than five levels in the hierarchy?

___ Yes  ___ No

• Is the hierarchy maintained throughout the tool?

___ Yes  ___ No

___ Somewhat

3. Is the tool structure based on tasks?

___ Yes  ___ No

___ Somewhat

4. Does the tool have headings (this can apply to printed documents, Web-based documents, presentations,
etc.)?

___ Yes  ___ No

• Are sections or headings arranged in a logical order? Do they clearly describe the contents of the sections they cover?

___ Yes  ___ No

___ Somewhat

• Do the paragraphs relate to the headings? (Do they contain information users would expect to find under each heading?)

___ Yes  ___ No

___ Somewhat

• Does each section identify the appropriate user?

___ Yes  ___ No

___ Somewhat

5. Are there clear cross references to other sections, research, tools, or toolkits (e.g. Web hyperlinks)?

___ Yes  ___ No

___ Somewhat

Design Considerations

Each tool should conform to AHRQ Publishing and Communications Guidelines. These ensure a consistent look and feel across materials. You may want to consider consulting with a graphic designer who can assist with layout and incorporating graphic elements. Examine the questions in this section to determine if you can improve the tool's look and usability.

1. Does each tool's design adhere to AHRQ Publishing and Communications Guidelines and have the same branding
   and style?

___ Yes  ___ No

2. Does each tool address 508 compliance issues?²

___ Yes  ___ No

3. Does the tool appear accessible and easy to use?

___ Yes  ___ No

4. Does the tool have a visual focal point (logo, title, or
   design element)?

___ Yes  ___ No

5. Does the tool have an intentional and predictable grid with elements lining up vertically on the page? For example, do paragraphs start at 1" and all bulleted lists start at 2" from the border?

___ Yes  ___ No

6. Does the tool use common and easily readable fonts, such as Arial, Verdana, Tahoma, Garamond, or Times Roman?

___ Yes  ___ No

7. Are different fonts or sizes used to denote different levels of the organizational hierarchy?

___ Yes  ___ No

8. Does the tool use emphasis typeface techniques, such as bold and italics, without overusing them?

___ Yes  ___ No

9. Are the fill-in spaces on the tool large enough for users to comfortably enter information? If the toolkit is Web based, do the fields expand to accommodate any number of characters?

___ Yes  ___ No

10. Are pages, items, or questions numbered to help users navigate?

___ Yes  ___ No

11. Does the tool use bullets or numbers to list important information?

___ Yes  ___ No

12. Does the tool use white space—the area not used for text, such as borders and the space between lines—to visually organize sections and items and make the tool more reader friendly?

___ Yes  ___ No

___ Somewhat

13. Does the tool use color and shading to help users navigate?

___ Yes  ___ No

14. Does the tool use visual displays in addition to text, such as tables, lists, and graphics?

___ Yes  ___ No

___ Somewhat

• Do the visuals have descriptive titles?

___ Yes  ___ No

• Do the visuals support the text and help communicate the message to users?

___ Yes  ___ No

___ Somewhat

• Are these visuals 508 compliant, i.e., do they contain text to make them accessible to individuals with disabilities?

___ Yes  ___ No

___ Somewhat

Language Considerations

Clear and concise language aids in communicating your message. Use this section to ensure your grammar and word choice are appropriate for the tool. You may also consider having an editor review the product to ensure these considerations are addressed.

1. Does the tool use clear and concise language that's free of jargon?

___ Yes  ___ No

2. Does the tool use positive or simple negative sentence construction whenever possible? (for example: Always include or never include not don't include)

___ Yes  ___ No

3. Does the tool use the active voice (for example: consult with stakeholders not stakeholders should be consulted)?

___ Yes  ___ No

4. Does the tool use personal pronouns (for example: your evaluation team)?

___ Yes  ___ No

5. Does the tool use action verbs (for example: assess not make an assessment)?

___ Yes  ___ No

6. Does the tool use gender-neutral words?

___ Yes  ___ No

7. Does the tool use words and terms consistently?

___ Yes  ___ No

8. Does the tool use lists or tables for several items or conditional statements (i.e., if X, then Y)?

___ Yes  ___ No

9. Does the tool use correct spelling, grammar, and punctuation?

___ Yes  ___ No