Evaluating Dissemination of AHRQ CER Products (Text Version)
On September 19, 2011, Darren Mays made this presentation at the 2011 Annual Conference. Select to access the PowerPoint® presentation (400 KB). Plugin Software Help.
Slide 1
Evaluating Dissemination of AHRQ Comparative Effectiveness Research (CER) Products
Darren Mays, PhD, MPH
Department of Oncology
Georgetown University Medical Center
Lombardi Comprehensive Cancer Center
Washington, DC
Slide 2
Research to Practice Gap
Image: A rope-bridge over water is shown.
Bernhardt, Mays, & Kreuter, 2011.
Slide 3
How will iADAPT help?
- Poised to make progress:
- What approaches work? For whom? In what settings/conditions?
- Presents an evaluation challenge:
- Creative methods/approaches.
- Diverse populations.
- Different clinical areas.
- Need for a flexible evaluation framework.
Slide 4
What is RE-AIM?
- Evaluate public health impact.
- Focus on dissemination.
- Barriers include design, setting, approach.
- Impact assessed on multiple domains.
- RE-AIM domains:
- Reach, Efficacy/Effectiveness, Adoption, Implementation, Maintenance.
Glasgow, Vogt, & Boles, 1999; Glasgow, Lichtenstein, & Marcus, 2003.
Slide 5
RE-AIM Domains
- Reach:
- Did the CER products reach the intended population(s)?
- Participation rate(s), characteristics, baseline "risk"
- Did the CER products reach the intended population(s)?
- Efficacy/effectiveness:
- What is the impact on intended outcomes?
- Clinical outcomes, CER product utilization, occurrence of harms/unintended consequences.
- What is the impact on intended outcomes?
Refer to RE-AIM domains handout; Glasgow et al., 2006.
Slide 6
RE-AIM Domains
- Adoption:
- Did the intended units use the CER product(s)?
- Participation and characteristics of setting(s), delivery agents, barriers to adoption.
- Did the intended units use the CER product(s)?
- Implementation:
- Were the CER products implemented as intended?
- Adherence, fidelity, technical success.
- Were the CER products implemented as intended?
- Maintenance:
- What is the long-term impact of CER products?
- Long-term efficacy/effectiveness, sustained implementation, barriers to long-term use.
- What is the long-term impact of CER products?
Refer to RE-AIM domains handout; Glasgow et al., 2006.
Slide 7
Determining Impact
- Quantitatively determining impact.
- Original application:
- Reach x Efficacy = Impact.
- RE-AIM overall impact:
- Product of all 5 domains.
- Requires quantifiable measures.
Glasgow, Vogt, Boles, 1999; Glasgow et al. 2006.
Slide 8
Application to iADAPT?
R | E | A | I | M | Audience(s) | Clinical Area(s) | |
---|---|---|---|---|---|---|---|
In Person CHW Outreach Acad. Detailing | Med Med | Med Med | Low Low | Low Low | Low Med | Patient Provider | Diabetes Diabetes |
Group CER Training School Group Therapy | Med High Low | Low Low Med | Med Med Med | Med Low Low | Med Low Med | Policy Patient Patient | Multiple Heart Disease Diabetes |
eHealth Clinic Kiosk Web Patient Portal | Med Med | Med Med | Low Med | High High | Med High | Patient Patient & Provider | Diabetes Diabetes |
Print/Media Targeted Video | Med | Med | Med | Med | Med | Patient | Heart Disease |
Adapted from Glasgow, et al., 2001
Slide 9
A Closer Look
Clinic Kiosk:
- R: n = 200 patients.
Well-controlled diabetes. - E: Small effect size.
200 CERSGs (~1/pt.). - A: 75% of clinics.
- I: 50% completion rate.
Technical problems. - M: Few support resources.
Limited patient interest.
Web Portal:
- R: n = 1000 patients.
Poor diabetes control. - E: Medium effect size.
2,500 CERSGs (~2.5/pt.). - A: 100% of clinics.
- I: 75% completion rate.
Few technical problems. - M: Minimal maintenance.
Low-direct cost to patients.
Slide 10
Conclusions
- A flexible evaluation framework.
- Multi-domain evaluation approach.
- Identify facilitators, barriers, and future directions.
- Creative approaches may be needed!
Slide 11
RE-AIM Resources
- NCI DCCPS web site for RE-AIM:
- Resources include:
- Figures/graphics illustrating key concepts.
- Checklists and planning tools.
- Example measures.
- Publications, presentation.
Slide 12
References
Bernhardt, JM, Mays, D, & Kreuter, MW. (2011). Dissemination 2.0: Closing the gap between knowledge and practice with new media. J Health Comm, 16(S1), 32-44.
Glasgow, RE, Vogt, TM, & Boles, SM. (1999). Evaluating the public health impact of health promotion interventions: The RE-AIM framework. AJPH, 89(9), 1322-1327.
Glasgow, RE, et al. (2001). The RE-AIM framework for evaluating interventions: What can it tell us about approaches to chronic illness management. Patient Ed. & Counsel, 44, 119-127.
Glasgow, RE, Lichtenstein, E, & Marcus, AC. (2003). Why don't we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. AJPH, 93(8), 1261-1267.
Glasgow, RE, et al. (2006). Using RE-AIM metrics to evaluate diabetes self-management support interventions. AJPM, 30(1), 67-73.