RE-AIM Plus To Evaluate Effective Dissemination of AHRQ CER Products
On September 19, 2011, Michele Heisler made this presentation at the 2011 Annual Conference. Select to access the PowerPoint® presentation (2 MB). Plugin Software Help.
Slide 1
RE-AIM Plus To Evaluate Effective Dissemination of AHRQ Comparative Effectiveness Research (CER) Products
Michele Heisler, MD, MPA
September, 2011
Slide 2
Web-Based Tool to Facilitate Diabetes Medications Decision-making (AHRQ R18, Heisler)
iDecide:
- Assessing the use of a Web-based, interactive, tailored decision tool in improving diabetes health outcomes.
Decido:
- Evaluando el uso de un herramieta de decision en mejorar los resultados de la salud en pacientes con diabetes.
Slide 3
Definition of Tailoring
- Assess an individual's characteristics relevant to the behavior.
- Use assessment data to generate messages relevant to that individual's specific needs.
- Deliver these messages in a clear, vivid—and interactive—format.
Slide 4
iDecide
- Project description:
- Community health worker (CHW)-delivered intervention to improve knowledge of diabetes medications among low-literacy, low-income adults (Latinos and African Americans) with poor glycemic control/reported medication problems.
- 2 arms (n=210):
- Control group: reviews the static, paper AHRQ patient guide with a CHW.
- Intervention group: reviews the tailored, Web-based program with a CHW.
Slide 5
Goals of the Study
Aim 1: (Work together to) Build an interactive, tailored diabetes medication decision aid to help people:
- Assess their treatment goals.
- Identify personal preferences and concerns.
- Understand options for improving their diabetes care (when appropriate).
Slide 6
Goals of the Study (continued)
Aim 2: Compare the tailored tool with the print AHRQ CER guides on participant:
- Knowledge of medications.
- Satisfaction with information.
- Decisional conflict.
Slide 7
Goals of the Study (continued)
Aim 3: Look at how the tool affects:
- Participant changes to medication.
- Self-reported medication adherence.
- Participant beliefs about medication.
- A1c levels.
Slide 8
What Do Qualitative Methods Add to Implementation Evaluation?
- Use open-ended techniques, e.g., interviews, observation:
- Goal is understanding, rather than measurement.
- Get into the "black box":
- Uncover the whys and hows behind quantitative measures.
- Identify and understand implementation processes, how influenced by context and how affect implementation success.
Slide 9
Development of RE-AIM Plus
- Systematically review each RE-AIM dimension and create open-ended questions.
- Add information sources and data analyses to address:
Dimension | Quantitative Measures | Qualitative Inquiry |
---|---|---|
Reach | ||
Effectiveness | ||
Adoption | ||
Implementation | ||
Maintenance |
Slide 10
RE-AIM Plus: Reach
Quantitative Measures:
How many and what proportion of target population is being contacted and participating?
- Measures:
- # eligibles contacted/# eligibles.
- # eligibles participating/# eligibles.
- Use measures to track patient contact and participation.
- Look at variation across sites.
- Data Source:
- Clinic data, recruitment, and activity data sets.
Qualitative Inquiry:
What explains variation across sites?
What are factors and processes underlying barriers to contacting patients and patient participation, and how do we address them?
Data Sources:
- Semi-structured interviews with Key Informants.
- Site visits and observations.
- E-mails between stakeholders and research staff.
Slide 11
RE-AIM Plus: Effectiveness
Quantitative Measures:
Evaluate effects of intervention on outcomes:
- Measure:
- Changes between baseline and follow-up in survey measures and A1cs.
- Data Source:
- Surveys and point-of-service A1cs.
Qualitative Inquiry:
Explain summative outcomes:
- What are the conditions and mechanisms that lead to effectiveness?
- Why did the intervention work? Why not?
- What explains variation across sites?
- What are factors and processes underlying barriers to implementation, and how did we address them?
Data Sources: All
Slide 12
RE-AIM PLUS: Implementation
Quantitative Measures:
Implementation:
Did CHWs use the decision aid with participants as anticipated?
- Measures:
- % of enrolled patients with decision aid session.
- % of enrolled patients who received follow-up call.
- % of enrolled patients who discussed questions/concerns with MDs.
- Data Source:
- Activity reports/Surveys.
Qualitative Inquiry:
Pre-implementation:
What are site-specific issues that might influence implementation? What modifications do we need to make and how do we make them?
Implementation:
- What were the problems with key implementation processes?
- What are the barriers to conducting the sessions, follow-up calls, MD discussions.
Data Sources: all
Slide 13
RE-AIM PLUS: Maintenance
Quantitative Measures:
Is the program maintained after the study period?
Qualitative Inquiry:
- Can the program be sustained after the study period?
- What is the feasibility of retaining program-specific staff to continue the intervention?
- What is the perception of program value among stakeholders?
Slide 14
Important Methods Considerations
Methods:
- Effective procedures and instruments to systematically collect data.
- Multiple data sources.
- Data management system to facilitate data retrieval:
- Including real time.
Slide 15
Concluding Thoughts
RE-AIM Plus:
- Useful in generating context-specific and generalizable information on implementation.
- Allows understanding of implementation mechanisms behind both process and outcome measures.
- May improve ways to adapt implementation in real time and increase likelihood of success when interventions are disseminated.