Evaluating Dissemination of AHRQ CER Products (Text Version)

Slide presentation from the AHRQ 2011 conference.

On September 19, 2011, Darren Mays made this presentation at the 2011 Annual Conference. Select to access the PowerPoint® presentation (400 KB). Plugin Software Help.


Slide 1

Evaluating Dissemination of AHRQ CER Products

Evaluating Dissemination of AHRQ Comparative Effectiveness Research (CER) Products

Darren Mays, PhD, MPH

Department of Oncology
Georgetown University Medical Center
Lombardi Comprehensive Cancer Center
Washington, DC

Slide 2

Research to Practice Gap

Research to Practice Gap

Image: A rope-bridge over water is shown.

Bernhardt, Mays, & Kreuter, 2011.

Slide 3

How will iADAPT help?

How will iADAPT help?

  • Poised to make progress:
    • What approaches work? For whom? In what settings/conditions?
  • Presents an evaluation challenge:
    • Creative methods/approaches.
    • Diverse populations.
    • Different clinical areas.
  • Need for a flexible evaluation framework.

Slide 4

What is RE-AIM?

What is RE-AIM?

  • Evaluate public health impact.
  • Focus on dissemination.
  • Barriers include design, setting, approach.
  • Impact assessed on multiple domains.
  • RE-AIM domains:
    • Reach, Efficacy/Effectiveness, Adoption, Implementation, Maintenance.

Glasgow, Vogt, & Boles, 1999; Glasgow, Lichtenstein, & Marcus, 2003.

Slide 5

RE-AIM Domains

RE-AIM Domains

  • Reach:
    • Did the CER products reach the intended population(s)?
      • Participation rate(s), characteristics, baseline "risk"
  • Efficacy/effectiveness:
    • What is the impact on intended outcomes?
      • Clinical outcomes, CER product utilization, occurrence of harms/unintended consequences.

Refer to RE-AIM domains handout; Glasgow et al., 2006.

Slide 6

RE-AIM Domains

RE-AIM Domains

  • Adoption:
    • Did the intended units use the CER product(s)?
      • Participation and characteristics of setting(s), delivery agents, barriers to adoption.
  • Implementation:
    • Were the CER products implemented as intended?
      • Adherence, fidelity, technical success.
  • Maintenance:
    • What is the long-term impact of CER products?
      • Long-term efficacy/effectiveness, sustained implementation, barriers to long-term use.

Refer to RE-AIM domains handout; Glasgow et al., 2006.

Slide 7

Determining Impact

Determining Impact

  • Quantitatively determining impact.
  • Original application:
    • Reach x Efficacy = Impact.
  • RE-AIM overall impact:
    • Product of all 5 domains.
    • Requires quantifiable measures.

Glasgow, Vogt, Boles, 1999; Glasgow et al. 2006.

Slide 8

Application to iADAPT?

Application to iADAPT?

 REAIMAudience(s)Clinical Area(s)
In Person
CHW Outreach
Acad. Detailing
Med
Med
Med
Med
Low
Low
Low
Low
Low
Med
Patient
Provider
Diabetes
Diabetes
Group
CER Training
School
Group Therapy
Med
High
Low
Low
Low
Med
Med
Med
Med
Med
Low
Low
Med
Low
Med
Policy
Patient
Patient
Multiple
Heart Disease
Diabetes
eHealth
Clinic Kiosk
Web Patient Portal
Med
Med
Med
Med
Low
Med
High
High
Med
High
Patient
Patient &
Provider
Diabetes
Diabetes
Print/Media
Targeted Video
MedMedMedMedMedPatientHeart Disease

Adapted from Glasgow, et al., 2001

Slide 9

A Closer Look

A Closer Look

Clinic Kiosk:

  • R: n = 200 patients.
    Well-controlled diabetes.
  • E: Small effect size.
    200 CERSGs (~1/pt.).
  • A: 75% of clinics.
  • I: 50% completion rate.
    Technical problems.
  • M: Few support resources.
    Limited patient interest.

Web Portal:

  • R: n = 1000 patients.
    Poor diabetes control.
  • E: Medium effect size.
    2,500 CERSGs (~2.5/pt.).
  • A: 100% of clinics.
  • I: 75% completion rate.
    Few technical problems.
  • M: Minimal maintenance.
    Low-direct cost to patients.

Slide 10

Conclusions

Conclusions

  • A flexible evaluation framework.
  • Multi-domain evaluation approach.
  • Identify facilitators, barriers, and future directions.
  • Creative approaches may be needed!

Slide 11

RE-AIM Resources

RE-AIM Resources

  • NCI DCCPS web site for RE-AIM:
  • Resources include:
    • Figures/graphics illustrating key concepts.
    • Checklists and planning tools.
    • Example measures.
    • Publications, presentation.

Slide 12

References

References

Bernhardt, JM, Mays, D, & Kreuter, MW. (2011). Dissemination 2.0: Closing the gap between knowledge and practice with new media. J Health Comm, 16(S1), 32-44.

Glasgow, RE, Vogt, TM, & Boles, SM. (1999). Evaluating the public health impact of health promotion interventions: The RE-AIM framework. AJPH, 89(9), 1322-1327.

Glasgow, RE, et al. (2001). The RE-AIM framework for evaluating interventions: What can it tell us about approaches to chronic illness management. Patient Ed. & Counsel, 44, 119-127.

Glasgow, RE, Lichtenstein, E, & Marcus, AC. (2003). Why don't we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. AJPH, 93(8), 1261-1267.

Glasgow, RE, et al. (2006). Using RE-AIM metrics to evaluate diabetes self-management support interventions. AJPM, 30(1), 67-73.

Current as of March 2012
Internet Citation: Evaluating Dissemination of AHRQ CER Products (Text Version). March 2012. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/news/events/conference/2011/mays/index.html