The Effective Health Care Program Stakeholder Guide

Chapter 3: Getting Involved in the Research Process

AHRQ relies on stakeholder involvement to ensure that research is relevant to, and useful for, decisionmakers. Stakeholder involvement also increases transparency in the research process, which is critical for maintaining the scientific integrity and credibility of AHRQ's work. Finally, once research is produced, it is hoped that stakeholders who are involved in the process are more likely to actively use and disseminate the information that they helped produce.

This chapter describes the different types of stakeholders that AHRQ works with and why they are important. Then it outlines the opportunities for involvement in different EHC activities and what that involvement entails. Not all types of stakeholders are involved in all stages of research; instead they are targeted where they can have the most impact. If you have been asked to join a specific project activity, you can use this chapter to find a step-by-step description of the project and what is expected of you. If you are looking for opportunities to be involved, the shaded boxes in each section explain the different opportunities and which types of stakeholders are needed.

Who Are Stakeholders and Why Are Their Views Important?

AHRQ has defined "stakeholders" as persons or groups that have a vested interest in a clinical decision and the evidence that supports that decision. Stakeholders may be patients, caregivers, clinicians, researchers, advocacy groups, professional societies, businesses, policymakers, or others. Each group has a unique and valuable perspective (Table 1).

Table 1. Stakeholder groups

Stakeholders Stakeholders' Perspective
Consumers, patients, caregivers, and patient advocacy organizations It is vital that research answer the questions of greatest importance to those experiencing the situation that the research addresses. Which aspects of an illness are of most concern? Which features of a treatment make the most difference? Which kinds of presentation of research results are easiest to understand and act upon?
Clinicians and their professional associations Clinicians are at the heart of medical decisionmaking. Where is lack of good data about diagnostic or treatment choices causing the most harm to patients? What information is needed to make better recommendations to patients? What evidence is required to support guidelines or practice pathways that would improve the quality of care?
Health care institutions, such as hospital systems and medical clinics, and their associations Many health care decisions are structured by the choices of institutional health care providers, and institutional health care providers often have a broad view of what is causing problems. What information would support better decisions at an institutional level to improve health outcomes?
Purchasers and payers, such as employers and public and private insurers Coverage by public or private purchasers of health care plays a large role in shaping individual decisions about diagnostic and treatment choices. Where does unclear or conflicting evidence cause difficulty in making the decision of what to pay for? Where is new technology or new uses of technology raising questions about what constitutes a standard of care? What research is or could be funded?
Health care industry and industry associations The manufacturers of treatments and devices often have unique information about their products.
Health care policymakers at the Federal, State, and local levels Policymakers at all levels want to make health care decisions based on the best available evidence about what works well and what does not. Comparative effectiveness research/patient-centered outcomes research can help decisionmakers plan public health programs, design health insurance coverage, and initiate wellness or advocacy programs that provide people with the best possible information about different health care treatment options.
Health care researchers and research institutions Researchers gather and analyze the evidence from multiple sources on currently available treatment options.

 

Return to Contents

Identification of Evidence Needs

Identifying a need for evidence is the beginning of any comparative effectiveness research/patient-centered outcomes research process. This presents an opportunity for stakeholders to have a significant impact by nominating a topic for research. Once a topic has been nominated, it is further clarified though a process of topic development, which may allow further opportunities for input from the nominator and possibly other stakeholders.

Topic Nomination

The EHC Program accepts topic nominations from all individuals or organizations (Box 2). Nominations can be submitted by anyone using the Web site (www.effectivehealthcare.ahrq.gov) and selecting the "Submit a Suggestion for Research" tab. For those who do not have access to a computer or the Internet, nominations may be submitted by mail (Appendix B). The nomination form requests information about the importance of the topic being proposed, patients affected by the issue, and specific questions that research could help answer.

 

Box 2. Get involved as a nominator

Any individual or organization can nominate a topic at any time using the EHC Web site: www.effectivehealthcare.ahrq.gov. Nominations may be submitted anonymously. 

The nominator is asked to— 

  • Describe a well-defined question related to the topic of interest (go to Appendix C for tips on developing strong research questions). 
  • Provide as much information as possible in order to guide the process for selecting which nominations will go forward for research.

 Be as specific as possible, particularly regarding the health care intervention or service of interest, the population of interest, and how you expect this research to affect health care. Providing more information initially will help guide the process for selecting research topics. 

Submit the nomination by mail or Email, or by completing the nomination form through the Web site (www.effectivehealthcare.ahrq.gov/index.cfm/submit-a-suggestion-for-research/). Include any supporting documentation. 

It is not mandatory that you provide your contact information, but it is often helpful to contact the nominator when clarification about the research question is needed. This is known as topic development. Clinical or other experts may also be consulted during this process. 

Consider serving in other roles (described below) if the topic goes forward for research.

 

Topic Development

When nominations are under review, the nominator may be asked to provide further information for clarification. Clinical or other experts may also be consulted to ensure that the context of the nomination is accurately considered. This process of clarification, known as topic development, is conducted by the EHC Program and typically takes 3 to 4 months. Once the topic nomination is fully developed, the EHC Program selects topics based on a set of predefined criteria (Appendix A).

Anyone can check the status of a nomination at any time on the EHC Web site.

Return to Contents

Evidence Synthesis

The EHC Program produces two types of evidence synthesis reports: systematic reviews and Technical Briefs. Both offer opportunities for stakeholder involvement, although each report follows a slightly different process, as described below.

Systematic Reviews

Topic Refinement

If a nominated topic has been selected to move forward as a systematic review, AHRQ will assign the topic to one of the EPCs for topic refinement. Topic refinement is the process of clarifying the scope of a topic and defining the questions so that it is ready to undergo research. Refinement requires several steps.

Kick-Off Call

Once an EPC is assigned a topic for refinement, a kickoff call is scheduled for key staff from the EPC, AHRQ, Eisenberg Center, and Scientific Resource Center to organize and discuss the research plan. The kickoff call is facilitated by the EPC with guidance from a staff person assigned by AHRQ, known as a Task Order Officer. The kickoff call is intended to help develop a common understanding of the task at hand, as well as to establish agreement on the methods, plans, and timeline for completing the research. The nominator may be asked to participate in this call to help clarify the intent of the topic and to communicate important contextual information.

Developing the Key Questions

Good research requires a good set of research questions. The key research questions for each systematic review are formulated and refined with the help of Key Informants to ensure that research addresses the questions important to decisionmakers, represents an accurate scope of issues, and produces the most valuable product (Box 3). Key Informants often include the nominator as well as other decisionmakers who can contribute to defining the scope and Key Questions of a research report. Key Informants may include patients and caregivers, clinicians (including both generalists and experts in relevant specialties), representatives of relevant professional and consumer organizations, insurers and health plan representatives who make coverage and benefit decisions, public policymakers, and others with experience in making health care decisions relevant to the topic. During the topic refinement process, the focus of the original nomination may be narrowed, expanded, or shifted depending on the input received from the Key Informants.

This input is gathered through Key Informant calls, which are scheduled and coordinated by the EPC assigned to do the research. One or several calls may be held. The "Guide for Key Informants," including roles, responsibilities, and what to expect, is shown as Appendix D.

 

Box 3. Get involved as a Key Informant for a systematic review

Key Informants are stakeholders with direct experience with the topic being researched as patients or caregivers, clinicians, policymakers, insurers, or other health care decisionmakers. Key Informants offer unique perspectives that help to refine Key Questions before the research begins. They also provide context, as well as help direct questions for specific considerations such as side effects, benefits, harms, and quality-of-life issues.

The EPC assigned to a research topic will invite approximately six to eight individuals to participate in the topic refinement process as Key Informants. The nominator of the topic will often be invited to participate to address the original intent of the nomination and to increase the likelihood that the end products will meet the originating need. All Key Informants must complete disclosure-of-interest forms, and they may be asked to submit a short description of their experience with the topic. The requirement of disclosure bolsters transparency, assists in mitigating bias, and helps create a balance of perspectives among the Key Informant group.

Once Key Informants are approved by AHRQ to participate, the EPC is responsible for scheduling and coordinating conference calls and/or other opportunities for input. The number of calls or methods used to collect input will vary depending on the complexity of the topic. It is recommended that calls include as many of the identified Key Informants as possible to foster more robust discussions. In the event that this is not possible, it may be necessary to schedule individual calls, schedule calls with subsets of the identified Key Informants, or use other methods for soliciting input from Key Informants.

The topic refinement process is expected to take 4 months. Key Informants should expect to—

  • Submit a completed disclosure-of-interest form.
  • Submit a brief description of their experience with the topic.
  • Participate in at least one, and possibly several, phone calls with the EPC and other Key Informants. Typically, calls last 1 hour and use a toll-free number. Often it is necessary to schedule multiple calls.
  • Be available to answer questions if the research team requires additional information.
  • Help guide the formation of Key Questions, which may involve the use of a patient, intervention, comparison, outcome (treatment and setting) [PICO(TS)] format. For more information on formulating questions using PICO(TS), please go to Appendix C.
  • Have their names and contact information shared with the EHC Program for potential consultation on future work, unless otherwise requested.

Key Informants who are also nominators should anticipate that the original nomination may be altered or changed during the topic refinement process in order to ensure the greatest possible application and relevancy.

The EHC Program assists EPCs in identifying and supporting Key Informants in the topic refinement process. To indicate interest in participating as a key informant, contact the Effective Health Care Program at (301) 427-1502 or EffectiveHealthCare@ahrq.hhs.gov.

 

Once Key Informants have provided input into the development of research questions, the EPC develops a draft set of Key Questions. The draft questions are then posted on the EHC Web site for public comment (Box 4). The final key research questions, along with an analytic framework and research abstract, will guide the research process.

 

Box 4. Get involved by providing public comments on the Key Questions

Anyone may comment on draft Key Questions, and the assigned EPC will consider incorporating feedback in the final Key Questions. It is critical that the questions posed for these documents reflect the concerns and dilemmas of consumers, clinicians, policymakers, and other health care decisionmakers. Public posting is another opportunity for involvement of the whole range of stakeholders and a way to ensure that the research report has the broadest possible relevancy.

The public comment period lasts 3 weeks. Those who wish to comment on the Key Questions should expect to—

You can sign up at www.effectivehealthcare.ahrq.gov/index.cfm/join-the-email-list1 to receive alerts when Key Questions are posted for clinical areas that interest you.

 

Conducting a Systematic Review

A systematic review involves carefully reviewing the literature and analyzing the quality of existing studies and data. This process can take up to 12 months to complete, during which time scientific investigators thoroughly and methodically examine information. During the conduct of a report, there will not be opportunity for communication with the institution conducting the research. All communication regarding the topic at this phase must go through AHRQ in order to ensure that the investigators remain as objective as possible. The process does include an opportunity for stakeholder involvement through Technical Expert Panels and during peer and public review.

Technical Expert Panels

Technical Expert Panels provide expert advice about the clinical specialty being studied as well as advice about research methods. Therefore, Technical Expert Panels are primarily comprised of clinical, research, and methodological experts who can provide information and guidance on technical aspects of the review (Box 5). Technical Experts are assembled by the EPC assigned to the report. The AHRQ Task Order Officer is available to help identify participants for Technical Expert Panels if needed.

 

Box 5. Get involved as part of a Technical Expert Panel

Participation on Technical Expert Panels is usually limited to researchers, clinical experts, statisticians, and specialists who can help ensure the methodological rigor of the research report. Generally comprised of five to eight members, Technical Expert Panels help focus the literature search, identify inclusion/exclusion criteria, and assist in the evaluation of available evidence. The size and composition of the Technical Expert Panel are intended to create a balance between content and methodological expertise. The assigned EPC is responsible for convening the Technical Expert Panel, with approval from the AHRQ Task Order Officer.

The research process is expected to take up to 12 months. Technical Experts should expect to—

  • Submit a completed disclosure-of-interest form.
  • Submit a brief description of their experience with the topic.
  • Participate in at least one, and possibly several, phone calls with the EPC and other Technical Experts. Typically, calls last 1 hour and use a toll-free number. Often it is necessary to schedule multiple calls.
  • Be available to answer questions if the research team requires additional information.

Have their names and contact information shared with the EHC Program for potential consultation on future work, unless otherwise requested.

Peer and Public Review

Once the EPC has completed a draft of the systematic review, a peer review panel is assembled to provide additional and technical review of the report (Box 6). The peer review process is coordinated through the Scientific Resource Center.

 

Box 6. Get involved in the peer review process of a systematic review

Research products undergo a peer review process to ensure the scientific integrity and quality of research reports. AHRQ, the Scientific Resource Center, and the assigned EPC identify Peer Reviewers for specific topics. Decisionmaker organizations (such as professional societies) are encouraged to suggest experts to participate in peer review. Suggestions for Peer Reviewers can be made during the topic nomination, topic development, and topic refinement processes by contacting EffectiveHealthCare@ahrq.hhs.gov. The Scientific Resource Center coordinates the peer review process, which typically lasts 3 months. The EPC considers all peer review comments and modifies the final report as appropriate. 

Peer Reviewers should expect to— 

  • Complete their review of the draft research review within 4 weeks. 
  • Have their names and contact information shared with the Scientific Research Center for potential consultation on future work, unless otherwise requested.

 

While the draft report is undergoing peer review, it is posted on the EHC Web site for public comment (Box 7). An announcement is sent through the EHC Program listservTM that the draft is available for comment. Reports are typically available online for public comment for 4 weeks. 

To sign up for EHC Program listserv notification, go to the EHC Web site http://www.effectivehealthcare.ahrq.gov) and select "Join the E-mail List" in the lower left corner. 

 

Box 7. Get involved by providing public comments on the draft report

Anyone may comment on the draft report, and the assigned EPC will consider incorporating feedback in the final report. It is critical that the questions posed by these documents reflect the concerns and dilemmas of consumers, clinicians, policymakers, and other health care decisionmakers. Public posting is another opportunity for involvement of the whole range of stakeholders and a way to ensure that the research report has the broadest possible relevancy. Those who wish to comment on the draft report should expect to— 

You can sign up at www.effectivehealthcare.ahrq.gov/index.cfm/join-the-email-list1/ to receive alerts when draft reports are posted for clinical areas that interest you.

 

After all public and peer review comments have been received, the final report is prepared. The process of responding to and addressing Peer Reviewer, Technical Expert, and public comments can take up to 3 months. The final report is then posted on the EHC Web site at www.effectivehealthcare.ahrq.gov. A notice of availability is also sent to individuals and organizations who have signed up through the AHRQ listserv to receive announcements. 

Disposition of Comments

AHRQ supports and is committed to the transparency of its review processes. Therefore, within 3 months after a final report is posted, all comments received from Peer Reviewers, Technical Experts, and the public, and all the responses made by the authors of a draft report to the public comments (i.e., the "disposition of comments") are posted on the EHC Web site. Each comment is listed with the name and affiliation of the commentator, if such information is provided. Public commentators are not required to provide their names or affiliations to submit suggestions or comments, but contact information will be used to communicate with commentators if there are questions about submitted comments.

Future Research Needs

A Future Research Needs paper is a document produced by an EPC, usually the one preparing the main research report. After completing a research review, including identification of evidence gaps, the EPC convenes a group of stakeholders, including investigators, funders, and others, to prioritize future research needs as they relate to the research topic (Box 8). The results of these discussions and prioritization are summarized in a separate Future Research Needs paper.

 

Box 8. Get involved identifying needed research

Methods of involving stakeholders in the development of the Future Research Needs papers are being tested. Research institutions consult with decisionmakers regarding how and what type of research should be prioritized to meet the identified evidence gaps. 

The role of a stakeholder at this point is to participate in discussions to describe and prioritize research needs. Stakeholders involved in identifying research needs should expect to— 

  • Read and review portions of the research report.
  • Review suggestions and draft language regarding the prioritization of research gaps and needs for additional evidence. 
  • Provide comments in individual conversations or in group settings, such as dedicated meetings or conference calls. 
  • Have the process take up to 2 months. 

In a transparent and systematic formal process, all stakeholders, including clinicians, funding agencies, and researchers, consider the gaps between available medical knowledge and the needs of clinical practice that are identified in the systematic reviews. Participants in the discussion include the researchers who worked on the individual review where the gap was first identified, stakeholders with interest in the topic, clinicians with particular expertise in the topic area, and agencies with funds for potential future research. Also involved are researchers with expertise in the clinical area and in study design, who can help identify evidence needs and develop new research projects based on the findings of the systematic review. It is hoped that this process will help shape future research plans and set priorities for a national investment in new research.

Inputs to the evidence gap identification process include nominations and recommendations of stakeholders by groups such as the Federal Coordinating Council for Comparative Effectiveness Research and the Institute of Medicine's project on Priority Setting for Comparative Effectiveness Research, as well as AHRQ's systematic review process.

 

Technical Briefs

A Technical Brief is a rapid report, typically on an emerging medical technology, strategy or intervention. It provides an overview of key issues related to the intervention—for example, current indications, relevant patient populations and subgroups of interest, outcomes measured, and contextual factors that may affect decisions regarding the intervention. Although Technical Briefs generally focus on interventions for which there are limited published data and too few completed protocol-driven studies to support definitive conclusions, the decision to request a Technical Brief is not solely based on the availability of clinical studies. 

The goals of the Technical Brief are to provide an early objective description of the state of the science, a potential framework for assessing the applications and implications of the intervention, a summary of ongoing research, and information on future research needs. In particular, through the Technical Brief, AHRQ hopes to gain insight on the appropriate conceptual framework and critical issues that will inform future comparative effectiveness research/patient-centered outcomes research.

Key Informants

Since Technical Briefs are assessments of the current state and implications of new medical technologies, Key Informant interviews are an important resource for identifying how the technology in question is currently used, the major issues and controversies surrounding the technology, and strategies for acquiring information on the technology. Therefore, an integral part of the research process for Technical Briefs is interviewing subject matter experts and end-users of the technology, such as patients and caregivers, practicing clinicians, relevant professional and consumer organizations, purchasers of health care, and others with experience in making health care decisions relevant to the topic.

This input may be gathered through Key Informant calls, which are scheduled and coordinated by the EPC assigned to do the research, or the EPC may carry out individual interviews, depending on the topic (Box 9).

 

Box 9. Get involved as a Key Informant for a Technical Brief

Key Informants are stakeholders with direct experience with the topic being researched as patients or caregivers, clinicians, policymakers, insurers, or other health care decisionmakers. Key Informants offer unique perspectives that help to refine Key Questions before the research begins. They also provide context, as well as help direct questions for specific considerations such as side effects, benefits, harms, and quality-of-life issues. They may provide information on how a technology, strategy, or intervention is used; the clinical context in which it is used; an and so on.

The EPC assigned to a research topic will invite approximately six to eight individuals to participate in the topic refinement process as Key Informants. The nominator of the topic will often be invited to participate to address the original intent of the nomination and to increase the likelihood that the end products will meet the originating need. All Key Informants must complete disclosure-of-interest forms, and they may be asked to submit a short description of their experience with the topic. The requirement of disclosure bolsters transparency, assists in mitigating bias, and helps create a balance of perspectives among the Key Informant group.

Once Key Informants are approved by AHRQ to participate, the EPC is responsible for scheduling and coordinating conference calls and/or other opportunities for input. The number of calls or methods used to collect input will vary depending on the complexity of the topic. It is recommended that calls include as many of the identified Key Informants as possible to foster more robust discussions. In the event that this is not possible, it may be necessary to schedule individual calls, schedule calls with subsets of the identified Key Informants, or use other methods for soliciting input from Key Informants.

The topic refinement process is expected to take 4 months. Key Informants should expect to—

  • Submit a completed disclosure-of-interest form.
  • Submit a brief description of their experience with the topic.
  • Participate in at least one, and possibly several, phone calls with the EPC and other Key Informants. Typically, calls last 1 hour and use a toll-free number. Often it is necessary to schedule multiple calls.
  • Be available to answer questions if the research team requires additional information.
  • Help guide the formation of Key Questions, which may involve the use of a patient, intervention, comparison, outcome (treatment and setting) [PICO(TS)] format. For more information on formulating questions using PICO(TS), please go to Appendix C.
  • Have their names and contact information shared with the EHC Program for potential consultation on future work, unless otherwise requested.

Key Informants who are also nominators should anticipate that the original nomination may be altered or changed during the topic refinement process in order to ensure the greatest possible application and relevancy.

The EHC Program assists EPCs in identifying and supporting Key Informants in the topic refinement process. To indicate interest in participating as a Key Informant, contact the EHC Program at (301) 427-1502 or at EffectiveHealthCare@ahrq.hhs.gov.

 

Peer and Public Review

Once the EPC has completed a draft of the Technical Brief, a peer review panel is assembled to provide an independent and unbiased technical review of the report. Peer Reviewers agree to be named in the report, although the report is not intended to represent their opinions. Peer review comments are published with a disposition of comments, although individual comments of invited Peer Reviewers are not directly attributed. The peer review process is coordinated through the Scientific Resource Center (Box 10).

 

Box 10. Get involved in the peer review process for a Technical Brief

Research products undergo a peer review process to ensure the scientific integrity and quality of research reports. AHRQ, the Scientific Resource Center, and the assigned EPC identify Peer Reviewers for specific topics. Decisionmaker organizations (such as professional societies) are encouraged to suggest experts to participate in peer review. Suggestions for Peer Reviewers can be made during the topic nomination and topic development processes by contacting EffectiveHealthCare@ahrq.hhs.gov

The Scientific Resource Center coordinates the peer review process, which typically lasts 3 months. The EPC considers all peer review comments and modifies the final report as appropriate.

Peer Reviewers should expect to—

  • Complete their review of the draft research review within 4 weeks.
  • Have their names and contact information shared with the Scientific Research Center for potential consultation on future work, unless otherwise requested.

 

While the draft report is undergoing peer review, it is posted on the EHC Web site for public comment. An announcement is sent through the EHC Program listserv that the draft is available for comment. Reports are typically available online for public comment for 4 weeks. To sign up for EHC Program listserv notification, go to the EHC Web site (www.effectivehealthcare.ahrq.gov) and select "Join the Email List" in the lower left corner.

Preparation of Final Report

After all public and peer review comments have been received, the final report is prepared. The process of responding to and addressing public comments can take up to 3 months. The final report is then posted on the EHC Web site at www.effectivehealthcare.ahrq.gov. A notice of availability is also sent to individuals and organizations who have signed up through the AHRQ listserv to receive announcements.

AHRQ supports and is committed to the transparency of its review processes. Therefore, within 3 months after a final report is posted, all comments received from the public and all the responses made by the authors of a draft report to the public comments (i.e., the "disposition of comments") are posted on the EHC Web site. Each comment is listed with the name and affiliation of the commentator, if such information is provided. Public commentators are not required to provide their names or affiliations to submit suggestions or comments, but contact information will be used to communicate with commentators if there are questions about submitted comments.

Return to Contents

Product Translation and Dissemination

The John M. Eisenberg Center for Clinical Decisions and Communications Science produces and disseminates user-friendly actionable summaries of systematic reviews and other reports for health care consumers, clinicians, and policymakers. These guides are designed to facilitate effective communication and decisionmaking about test or treatment choices between clinicians and patients, and to provide evidence-based decision tools for policymakers.

Translation

To ensure that translations of EHC Program research accurately reflect the needs of audience members as well as the science itself, the Eisenberg Center is involved throughout the systematic review process, listening carefully to Key Informants and Technical Expert panelists as they interact with EPCs, and interacting with investigators as they review public comments and refine reports. The Eisenberg Center also engages its own medical content experts to better understand the factors that both patients and clinicians must consider when making medical decisions on specific topics and to assist the Center in producing scientifically accurate translations of research findings.

Translation of research into practical decisionmaking tools for consumers, clinicians, and policymakers is a pivotal aspect of AHRQ's overarching goal to support the practice of evidence-based medicine. Effective translation of research is critical to ensuring that decisionmakers have access to high-quality evidence and tools for making informed decisions.

Input from patients, their caregivers, clinicians, and policymakers guides the creation of summaries and decision tools by providing a context for decisionmaking as well as feedback on the tools themselves. For each topic, a consumer panel, clinician panel, and policymaker panel are created early in the process. Panel members may participate in several group and individual conversations conducted by the Eisenberg Center or its associates.

Providing Information on the Context for Decisions

Consumer, clinician, and policymaker panelists can first participate in focus groups or an individual interview that explores the context of a specific topic currently under systematic review. During these interviews, panelists may be asked to share their experiences with the condition or certain treatment choices, their values and preferences in information-seeking and decisionmaking, and their challenges in deciding the best choice for themselves or others. This information is used to guide the Eisenberg Center in developing contextually relevant materials that speak directly to decisionmaking needs and situations among a broad audience base. The Center is careful to ensure that all panelist information is kept confidential. Patient/caregiver panelists who have personal experiences with the condition being studied are sought for these conversations. Clinician and policymaker panelists who have experience treating or setting policies on the test or treatment being studied are also sought for these conversations. Chosen panelists are paid for their time (See Box 11 for information on decision-context groups).

 

Box 11. Get involved in groups describing the context for decisions

The role of a panelist is to participate in interviews or group discussions to provide context and experience related to a specific health condition, and to test product messages. Panelists should expect to—

  • Participate in a 30- to 45-minute conversation with an Eisenberg Center associate, either in person or over the telephone and either alone or with other consumers, clinicians, or policymakers. This conversation is recorded and transcribed, although individual panelist names are not included in the transcript or reported.
  • Discuss their own health and medical experiences, their values and preferences, and their habits of information collection, whether it be from brochures, magazines, television, radio, Internet, or other sources.
  • Discuss the challenges faced and strategies used concerning the specific condition being studied as a patient, provider of clinical treatment, or policymaker.
  • Provide informed consent for their participation following a full disclosure of possible risks and benefits of participating in the interview.
  • Receive compensation for their time spent in conversation.
  • Have their names and contact information shared with the EHC Program for potential consultation on future work, unless otherwise requested.
Summary Guide Review

The Eisenberg Center consults with AHRQ and the Scientific Research Center to identify and invite individual representatives of decisionmaker organizations to review and provide feedback on draft information products and decision tools. The purpose of these reviews is to ensure the scientific accuracy of the products and to confirm the contextual relevancy of the content. Reviewers include individuals who have been involved throughout the research process, as well as those who have not been involved and can provide a "fresh eye" on decisions made throughout development. Feedback received from reviewers of these products is used to revise and improve the content or graphics of the guides. Product reviews generally take less than an hour for reviewers to complete (Box 12).

 

Box 12. Get involved in summary guide review

The role of a product reviewer is to provide review and comment on specific draft products. Product reviewers should expect to—

  • Receive information products and decision tools by mail or electronic delivery for review.
  • Receive a set of instructions and a formal review form to assist in the product review process.
  • Receive a clear timeline and return path for the submission of comments.
  • Receive compensation for their time spent reviewing products.
  • Have their names and contact information shared with the EHC Program for potential consultation on future work, unless otherwise requested.

 

Product User Feedback

Once draft summary guides are developed and reviewed, the Eisenberg Center invites consumers, clinicians, policymakers, and other decisionmakers who are potential users of the products to provide feedback on their ease of comprehension, usefulness, and actionability. Feedback received from potential users of these products is used to revise and improve the content or graphics of the guides. Often, panelists may encounter several "rounds" of products to confirm that changes are leading to a more understandable and useful information product or decision tool. Length of time involved in these interviews is between 60 and 90 minutes (Box 13).

 

Box 13. Get involved by providing user feedback

The role of a user panelist is to participate in interviews or group discussions to test products. Panelists should expect to—

  • Receive samples of draft information products/decision tools in either print or electronic version to read, or attend a session where Eisenberg Center associates can observe the panelists' interaction with the information product/decision tool to understand their initial interaction process with these products.
  • Answer specific questions about their interaction with the guides as they experience them.
  • Provide honest feedback and suggestions on making the guides more understandable or useful.
  • Provide informed consent of their participation following a full disclosure of possible risks and benefits of participating in the interview.
  • Receive compensation for their time spent in conversation.
  • Have their names and contact information shared with the EHC Program for potential consultation on future work, unless otherwise requested.

 

Dissemination

Having stakeholders distribute and talk about EHC Program research products and summary guides, model their use, and report outcomes from the use is critical to maximizing the understanding of how the work of the EHC Program improves the quality of health care decisionmaking. AHRQ and the Eisenberg Center employ a variety of strategies to disseminate products, including distribution of resources through consumer and professional organizations, societies, and associations, and through databases such as the National Library of Medicine and electronic clinical decision support services (Box 14).

 

Box 14. Get involved in product dissemination

Stakeholders can get involved in dissemination by—

  • Distributing products to their organization's members or clients.
  • Sharing information about the EHC Program and its products in their organization's newsletters or other communication.
  • Participating in efforts to measure the use and impact of the products, programs, or policies derived from EHC reports.
  • Linking to the EHC Web site and/or the EHC products.
  • Making presentations to their organization or other audiences regarding the EHC Program or any of its products, including successes they have had from using them.

To obtain EHC products or tools, visit the Web site at www.effectivehealthcare.ahrq.gov or request copies by calling 1–800–358–9295.

Return to Contents

Current as of February 2014
Internet Citation: The Effective Health Care Program Stakeholder Guide: Chapter 3: Getting Involved in the Research Process. February 2014. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/research/findings/evidence-based-reports/stakeholderguide/chapter3.html