Section 6: Toolkit Guidance

AHRQ Publishing and Communications Guidelines

Contents

What Is a Toolkit?
Checklists
How Do the Checklists Help Me?
Acknowledgments
Checklists
   "Is This a Toolkit?" Checklist
   Toolkit Content Checklist
   Toolkit Usability Checklist
   Tool Checklist

What Is a Toolkit?

To translate research findings into policy and practice, the Agency for Healthcare Research and Quality (AHRQ) and its grantees or contractors occasionally decide to create toolkits based on their work. An AHRQ workgroup developed the following definitions:

  • A "toolkit" is an action-oriented compilation of related information, resources, or tools that together can guide users to develop a plan or organize efforts to conform to evidence-based recommendations or meet evidence-based specific practice standards.
  • A "tool" is an instrument (e.g., survey, guidelines, or checklist) that helps users accomplish a specific task that contributes to meeting a specific evidence-based recommendation or practice standard.

Toolkits provide action-oriented guidance for practitioners or policymakers to apply the research to their work. This document outlines considerations for developing effective toolkits, and includes a series of checklists described below.

Return to Section Contents

Checklists

The checklists provided should help the toolkit developer(s) and AHRQ reviewers in designing and checking the final product under a grant or contract with AHRQ. Three checklists will help you with the overall toolkit:

  • The first checklist is used to determine if a toolkit is the right method to disseminate your research. Throughout your project, revisit this checklist to see if the results of the research continue to fit the toolkit frame.
  • The second checklist will help you develop the toolkit's contents once you decide on the toolkit format. This list looks at the type of information you should present in a toolkit and how to highlight critical versus supplementary information.
  • The third checklist will assist your team in thinking about the framework for your research-based tools—target users, toolkit style, and the function of each tool. The checklist also prompts your team to consider validating the toolkit with its intended audience, steering users to additional information, and measuring the impact of the toolkit.
  • The fourth checklist is useful for the entire toolkit but is intended for individual tools. This checklist addresses:
    • Organization.
    • Design.
    • Language.
    This checklist provides more detail and will aid your team in creating a set of tools that is consistent, making them easier to navigate and use.

In the planning stages and during the production process, review each of the four checklists with your team to ensure that you provide users with a product that includes all critical information. Fill out the four checklists for the entire toolkit, and for each individual tool contained in the checklist, complete checklist four (Table 1). To proceed, make sure you have answered "yes" to each item in the checklists. If you answer "somewhat" or "no" to any of the items, brainstorm with your team about how to resolve any concerns.

Table 1. Checklists to use with your toolkit and tools

 

Toolkit

Tool

  1. "Is This a Toolkit?" Checklist
 
  1. Toolkit Content Checklist
 
  1. Toolkit Usability Checklist
 
  1. Tool Checklist

Return to Section Contents

How Do the Checklists Help Me?

"Is This a Toolkit?" Checklist

Toolkits are effective for presenting action-oriented recommendations, but they are not appropriate for all research. At the beginning of your project, and throughout the process, answer the checklist questions to determine if a toolkit is the proper way to circulate your research findings.

Toolkit Content Checklist

Toolkits should have a standard format and look as well as similar types of information to easily guide users through a process of change. Answer the content checklist questions for your toolkit and tools to determine whether you have provided users with sufficient information to implement the changes in behavior that your research recommends.

Toolkit Usability Checklist

Think deliberately about the toolkit and its components: the potential users, the users' goals, the toolkit's look, expert validation, and measures of success. This checklist will help you and your team to plan a well-designed, usable toolkit.

Tool Checklist

This checklist addresses three areas: organization, design, and language.

Cohesive and logical organization helps users navigate the tool, improves comprehension, and encourages use. Answer the questions in the Organization Considerations section to determine how successfully your toolkit aids users in finding and using the information presented.

Each tool should conform to AHRQ Publishing and Communications Guidelines. The guidelines ensure a consistent look and feel across materials. You may want to consider consulting with a graphic designer who can assist with layout and incorporating graphic elements into the toolkit. Examine the questions in the Design Considerations section to determine if you can improve the tool's look and usability.

Clear and concise language aids in communicating your message. Use the Language Considerations section to ensure your grammar and word choice are appropriate for the tool. You may also consider having an editor review the product to ensure these considerations are addressed.

1. "Is This a Toolkit?" Checklist

A "toolkit" is an action-oriented compilation of related information, resources, or tools that together can guide users to develop a plan or organize efforts to conform to evidence-based recommendations or meet evidence-based specific practice standards.

Toolkits are effective for presenting action-oriented recommendations, but they are not appropriate for all research. At the beginning of your project, and throughout the process, address these checklist questions to determine if a toolkit is the proper way to share your research findings.

To proceed with a toolkit format, make sure you have answered "yes" to each item in the checklist. If you answer "no" to any of the items, brainstorm with your team about how to resolve any concerns. You may also want to consider pursuing another stand-alone product, such as a research paper or a fact sheet.

1. What behavior or action are you trying to promote?
      _____________________________________________________________________________
      _____________________________________________________________________________
      _____________________________________________________________________________

 

2. Why is a package of tools the best way to attain your goal (as opposed to a research paper, fact sheet, or other stand-alone product?
      _____________________________________________________________________________
      _____________________________________________________________________________
      _____________________________________________________________________________

 

3. Have you verified that a product like this does not already exist?
 

___ Yes
___ No

4. Did your research generate multiple action-oriented tools (such as specific procedures, protocols, or other structured activities) that, working together, can help users develop a plan or organize efforts to conform to evidence-based practice?

___ Yes
___ No

5. Have you spoken to potential users to determine the demand for a product like this one? How do you know there's a demand? What research or data supports this demand?
      _________________________________________________________________________
      _________________________________________________________________________
      _________________________________________________________________________

 

___ Yes
___ No

 

2. Toolkit Content Checklist

Toolkits should have a standard format and look as well as similar types of information to easily guide users through a process of change. Answer the content checklist questions for your toolkit and tools to determine if you have provided users with sufficient information to implement the changes in behavior that your research recommends.

To proceed, make sure you have answered "yes" to each item in the checklist. If you answer "no" to any of the items, brainstorm with your team about how to resolve any concerns.

1. Are the toolkit and tools based on tasks?

___ Yes
___ No

  • Does the toolkit provide sequential steps users should follow?

___ Yes
___ No

  • Does the toolkit provide examples of how others have used the toolkit or tools successfully?

___ Yes
___ No

2. Does the first page of the toolkit state its purpose?

3. Does the first page of the toolkit explain how to use the toolkit?

4. Does the first page of the toolkit list each tool and its purpose?

5. Does the first page of the toolkit describe target users and address their differing goals in using the toolkit (Go to the Toolkit Overview Checklist)?

___ Yes
___ No

___ Yes
___ No

___ Yes
___ No

___ Yes
___ No

6. Do the tools provide necessary information regarding what users need to complete tasks, such as:

  • Staff time.
  • Staff skills.
  • Materials.
  • Equipment.
  • Administrative clearances and approvals.

___ Yes
___ No

7. Does the toolkit organize resources to achieve a goal through specific actions (assign responsibilities, create a schedule, document progress, and ensure accountability)?

___ Yes
___ No

8. Does each tool give adequate instruction on how to use it (e.g., collect and analyze data, interpret results, implement suggestions, and assess impact)?

___ Yes
___ No

9. Does the toolkit provide users with additional resources for more information?

___ Yes
___ No

 

3. Toolkit Usability Checklist

Think deliberately about the toolkit and its components: the potential users, the users' goals, the toolkit's look, expert validation, and measures of success. This checklist will help you and your team to plan a well-designed, usable toolkit.

To proceed, make sure you have answered "yes" to each item in the checklist. If you answer "no" to any of the items, brainstorm with your team about how to resolve any concerns or consider pursuing an alternative method for disseminating your research results.1

Users

1. Have you identified your target group of users and taken them into account when designing the toolkit? How?
      _________________________________________________________________________
      _________________________________________________________________________
      _________________________________________________________________________

 

___ Yes
___ No

2. Is there more than one target group of users?

___ Yes
___ No

  • Will they have different goals?
  • What are some of those goals?

      _________________________________________________________________________
      _________________________________________________________________________

 

___ Yes
___ No

3. Have you explained how different users can adapt the toolkit to suit their needs?

___ Yes
___No

4. What tasks do you want users to accomplish with each tool?

Tool

Goal

A.

 

B.

 

C.

 

D.

 

E.

 

F.

 

5. Are target users familiar with the toolkit's concepts and terminology?

___ Yes
___ No

Style

1. How is this toolkit presented?

___ Web site                           ___ Printed Document

___ Video (CD/DVD)             ___ Audio (Digital, Web-based)

___ Other:                              

  • AHRQ has style guides for many different kinds of presentations that include information elements, such as font size, typeface, and color. Does the toolkit comply with AHRQ Publishing and Communications Guidelines for that presentation style?

___ Yes
___ No

2. Layering, or page sequence, in a Web-based environment reflects the hierarchy of organization in a document. Primary information is on the first level, and secondary information is on the second level of the Web site.

  • If your toolkit is Web-based, does your critical information appear on the primary level?

___ Yes
___ No

___ Not Applicable

  • Is your secondary information linked so it is supplementary to the first level of information?

___ Yes
___ No

___ Not Applicable

3. Does the toolkit as a whole have a cohesive, AHRQ-branded look and follow the design specifications laid out in the AHRQ Publishing and Communications Guidelines?

___ Yes
___ No

Testing and Evaluation

1. Will the toolkit be tested before it is published?

___ Yes
___ No

  • How will the toolkit be tested?

___ Expert review                  ___ Focus groups with users

___ Usability testing            ___ Other:

2. How can users measure the impact of the toolkit within their organization? What are the measures of success? (Suggest 3-5 methods of impact measurement.)

      _____________________________________________________________________________
      _____________________________________________________________________________
      _____________________________________________________________________________

 

Implementation

1. If the toolkit requires updates, will you, the developers, perform those updates?

___ Yes
___ No
___ Not Applicable

2. If the toolkit is a Web site and has the capacity to serve as a "live" resource for users (e.g., communities of practice for those with shared interests, bulletin boards, networking sites), which organization will provide ongoing quality oversight and technical support?
      _________________________________________________________________________
      _________________________________________________________________________
      _________________________________________________________________________

 

1 For question 2, if there is only one target group, the answer will be "no" and the subsequent two questions are not applicable.

 

4. Tool Checklist

This checklist addresses three areas: organization, design, and language.

To proceed, make sure you have answered "yes" to each item in the checklist. If you answer "somewhat" or "no" to any of the items, brainstorm with your team about how to resolve any concerns.

Organization Considerations

Cohesive and logical organization helps users navigate the tool, improves comprehension, and encourages use. Answer the questions in this section to determine how successfully your toolkit aids users in finding and using the information presented.

1. Does the tool have an advance organizer, such as a table of contents or site map?

___ Yes
___ No

  • Does the advance organizer provide a coherent, complete "big picture" view of the tool?

___ Yes
___ No

___ Somewhat

2. Does the tool have an organizational hierarchy and a clear structure of main topics and sub-topics?

___ Yes
___ No

___ Somewhat

  • Restricting your hierarchy to fewer than five levels makes it easier for users to navigate the tool. Does the tool have fewer than five levels in the hierarchy?

___ Yes
___ No

  • Is the hierarchy maintained throughout the tool?

___ Yes
___ No

___ Somewhat

3. Is the tool structure based on tasks?

___ Yes
___ No

___ Somewhat

4. Does the tool have headings (this can apply to printed documents, Web-based documents, presentations,
etc.)?

___ Yes
___ No

  • Are sections or headings arranged in a logical order? Do they clearly describe the contents of the sections they cover?

___ Yes
___ No

___ Somewhat

  • Do the paragraphs relate to the headings? (Do they contain information users would expect to find under each heading?)

___ Yes
___ No

___ Somewhat

  • Does each section identify the appropriate user?

___ Yes
___ No

___ Somewhat

5. Are there clear cross references to other sections, research, tools, or toolkits (e.g. Web hyperlinks)?

___ Yes
___ No

___ Somewhat

Design Considerations

Each tool should conform to AHRQ Publishing and Communications Guidelines. These ensure a consistent look and feel across materials. You may want to consider consulting with a graphic designer who can assist with layout and incorporating graphic elements. Examine the questions in this section to determine if you can improve the tool's look and usability.

  1. Does each tool's design adhere to AHRQ Publishing and Communications Guidelines and have the same branding
    and style?

___ Yes
___ No

  1. Does each tool address 508 compliance issues?2

___ Yes
___ No

  1. Does the tool appear accessible and easy to use?

___ Yes
___ No

  1. Does the tool have a visual focal point (logo, title, or
    design element)?

___ Yes
___ No

  1. Does the tool have an intentional and predictable grid with elements lining up vertically on the page? For example, do paragraphs start at 1" and all bulleted lists start at 2" from the border?

___ Yes
___ No

  1. Does the tool use common and easily readable fonts, such as Arial, Verdana, Tahoma, Garamond, or Times Roman?

___ Yes
___ No

  1. Are different fonts or sizes used to denote different levels of the organizational hierarchy?

___ Yes
___ No

  1. Does the tool use emphasis typeface techniques, such as bold and italics, without overusing them?

___ Yes
___ No

  1. Are the fill-in spaces on the tool large enough for users to comfortably enter information? If the toolkit is Web based, do the fields expand to accommodate any number of characters?

___ Yes
___ No

  1. Are pages, items, or questions numbered to help users navigate?

___ Yes
___ No

  1. Does the tool use bullets or numbers to list important information?

___ Yes
___ No

  1. Does the tool use white space—the area not used for text, such as borders and the space between lines—to visually organize sections and items and make the tool more reader friendly?

___ Yes
___ No

___ Somewhat

  1. Does the tool use color and shading to help users navigate?

___ Yes
___ No

  1. Does the tool use visual displays in addition to text, such as tables, lists, and graphics?

___ Yes
___ No

___ Somewhat

  • Do the visuals have descriptive titles?

___ Yes
___ No

  • Do the visuals support the text and help communicate the message to users?

___ Yes
___ No

___ Somewhat

  • Are these visuals 508 compliant, i.e., do they contain text to make them accessible to individuals with disabilities?

___ Yes
___ No

___ Somewhat

Language Considerations

Clear and concise language aids in communicating your message. Use this section to ensure your grammar and word choice are appropriate for the tool. You may also consider having an editor review the product to ensure these considerations are addressed.

  1. Does the tool use clear and concise language that's free of jargon?

___ Yes
___ No

  1. Does the tool use positive or simple negative sentence construction whenever possible? (for example: Always include or never include not don't include)

___ Yes
___ No

  1. Does the tool use the active voice (for example: consult with stakeholders not stakeholders should be consulted)?

___ Yes
___ No

  1. Does the tool use personal pronouns (for example: your evaluation team)?

___ Yes
___ No

  1. Does the tool use action verbs (for example: assess not make an assessment)?

___ Yes
___ No

  1. Does the tool use gender-neutral words?

___ Yes
___ No

  1. Does the tool use words and terms consistently?

___ Yes
___ No

  1. Does the tool use lists or tables for several items or conditional statements (i.e., if X, then Y)?

___ Yes
___ No

  1. Does the tool use correct spelling, grammar, and punctuation?

___ Yes
___ No

2 Section 508 Amendment to the Rehabilitation Act of 1973 requires Federal agencies to make their electronic and information technology accessible to people with disabilities. Go to http://www.section508.gov and Section 2 of the AHRQ Publishing and Communications Guidelines for more information and additional resources.

Return to Section Contents

Acknowledgments

AcademyHealth and Kleimann Communication Group, Inc. prepared this document and presented it to the Agency for Healthcare Research and Quality in November 2008 under AHRQ Knowledge Transfer and Applications Support Contract No. 290-04-0001.

Return to Section Contents
Return to Guidelines Contents
Proceed to Next Section

Current as of February 2013
Internet Citation: Section 6: Toolkit Guidance: AHRQ Publishing and Communications Guidelines. February 2013. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/research/publications/pubcomguide/pcguide6.html