Closing the Quality Gap: Revisiting the State of the Science

Select to access the PowerPoint® presentation (360 KB).

Slide 1

Closing the Quality Gap: Revisiting the State of the Science

Kathryn McDonald
Stanford University

Slide 2

Acknowledgements

Christine Chang, AHRQ
Ellen Schultz, Stanford University
The Topic Review Teams from each EPC

• Minnesota EPC (Disability Outcomes).
• Johns Hopkins EPC (Palliative Care).
• RAND EPC (Bundled Payment).
• Duke EPC (PCMH).
• Oregon EPC (Public Reporting).
• BCBSA EPC (HAI).
• Vanderbilt EPC (Disparities).
• RTI International–Univ. of North Carolina EPC (Medication Adherence).

This work supported by the Agency for Healthcare Research and Quality (AHRQ).

The views expressed here do not necessarily reflect those of the Agency for Healthcare Research and Quality.

Slide 3

Outline

• CQG Series Goals.
• Topics and Framework.
• Results.
  • Highlights.
  • Intervention features, context, harms.
• Messages to Key Audiences.
  • Micro, macro, research.
• Challenges Across Topics.
• Next Steps to Advance the Field.

Slide 4

CQG Series Goals

• Assemble evidence to "close the quality gap" for 8 high-priority topics.
• Synthesize lessons learned across topics about quality improvement (QI) research.
• Identify research gaps and make recommendations to improve future research and evidence reviews.

Slide 5

CQG Series Topics

• QI measurement of outcomes for people with disabilities.
• Effects of bundled payment systems on health care spending and quality of care.
• Public reporting as a quality improvement strategy.
• Quality improvement interventions to address health disparities.
• Interventions to improve health care and palliative care for advanced and serious illness.
• Patient-Centered Medical Home (PCMH).
• Prevention of health care-associated infections (HAI).
• Comparative effectiveness of medication adherence interventions.

Slide 6

Series Key Questions & Framework

• KQ1: What is the quality gap targeted by the review? How might the gap be approached to make improvements?
• KQ2: Who are the likely stakeholders who could act upon the gap? What evidence will they need? At what level (micro, macro) are changes likely needed?
• KQ3: What is the state of the science for the topic/quality gap? What are PICOTS, logic models, context? [aka, "PLICCOTS"].

PICOTS = populations, interventions, comparators, outcomes, timing and setting.
PLICCOTS = population, logic model, interventions, comparators, context, outcomes, timing, setting.

Slide 7

CQG Series Framework

Image: An oval is captioned "Topic emphasis to close quality gap?" Three arrows point from this oval to three squares, captioned "Information," "Incentives," and "Infrastructure/Delivery Systems." Three arrows point from these three squares to a second oval, captioned "Target Audience(s)?" Four arrows point from "Target Audience(s)?" to four smaller squares, captioned "Policy," "Delivery Org," "Clinician," and "Patient." "Policy" and "Delivery Org" are paired by a bracket captioned "macro." "Clinician" and "Patient" are paired by a bracket captioned "micro."

Slide 8

CQG Series Framework

The "3 I's" of Improvement:

Real reform "requires changes in the organization and delivery of care that provide physicians with the information, infrastructure, and incentives they need to improve quality and control costs."

-Victor Fuchs

Slide 9

Topics and Framework Overlay

Slide 10

Results Highlights by Quality Lever

Slide 11

Results Highlights: QI Strategy Effectiveness by Topic

O = strategy studied for topic.
X = evidence of benefit for this strategy.

Disability Outcomes omitted because it did not study interventions.

Slide 12

Intervention Features

4 reviews looked for evidence of effectiveness by intervention features.

• Little or no evidence available by feature for Bundled Payment, PCMH, Medication Adherence.
• Qualitative research suggests patient use of Public Reporting varies by relevance, readability and clarity.

Slide 13

The Role of Context

Context examined in 3 reviews:

• Reductions in spending with Bundled Payment magnified for for-profit providers and hospitals under greater financial pressure.
• Public Reporting stimulates improvements more readily in competitive markets and among low performers (high strength of evidence).
• HAI review found great variety in reporting and use of contextual factors; did not examine association with outcomes.
• Overall, few studies examined impact of context on intervention effectiveness.

Slide 14

Potential Harms from QI Activities

4 reviews examined potential harms.

• Harms rarely investigated in literature.
  • Little or no evidence for Disparities or Medication Adherence.
• Single-setting Bundled Payment programs shifted care to other settings; current trend is use of multi-setting bundled payment.
• Much discussion of harms for Public Reporting, but rarely investigated.
  • Available evidence mixed, but overall does not support harms concern.

Slide 15

Messages to Key Audiences: Patients, Caregivers and Clinicians

Slide 16

Messages to Key Audiences: Health Delivery Organizations and Policymakers

Slide 17

Messages to Research Community

• Poor study quality and extreme heterogeneity limited evidence synthesis for all topics.
• Economic outcomes rarely explored (PCMH, HAI, Medication Adherence).
• Evidence missing from key settings.
  • Examine Public Reporting beyond hospital cardiac care and Nursing Home Compare
  • Investigate Palliative Care in hospice and LTC settings.
  • Improve HAI prevention in ambulatory surgery and dialysis centers.
• Effectiveness unknown in key patient sub-groups or vulnerable populations.
  • Include mixed populations of Disabled and non-disabled patients.
  • Investigate Palliative Care for populations other than cancer patients.
  • Test PCMH in pediatric and general adult populations.
  • Explore reducing Disparities beyond racial/ethnic minorities.
  • Improve Medication Adherence in racial/ethnic minorities, low-income patients, un- or under-insured, patients with low literacy.

Slide 18

Common Challenges Across Topics

• Traditional systematic review methods poor fit for complex, system-based interventions.
  • Difficulty defining "doable" topic scopes.
• Lack of common terminology, framework for QI interventions.
• Inconsistent use of outcomes measures limited synthesis across studies.
  • Extreme study heterogeneity in design, populations, outcomes.
• Poor study quality.
  • RCTs not necessarily best design, often not practical.
  • Lack of guidance for assessing study quality and strength of evidence for complex interventions.

Slide 19

Recommendations to Improve Quality of Literature

* More studies needed that include both disabled and non-disabled patients.

Slide 20

Next Steps to Advance the Field

• Develop core set of outcomes measures for use in QI research.
• Develop a lexicon and framework for describing QI interventions.
• Develop measures of context.
• Adapt or develop new methods for evaluating effectiveness and comparative synthesis of complex, context-dependent, systems-level QI interventions.

Slide 21

Extra material

Slide 22

Series Key Questions & Framework

KQ [Key Question] 1. Quality Gap Addressed by:

• Information Data:
  • Measuring.
• Incentives:
  • Influencing.
• Infrastructure & Processes:
  • Improving.

KQ2. Target Audience(s):

• Level(s) of implementation:
  • Macro:
    • Policy/Market.
    • Organization.
  • Micro:
    • Clinician.
    • Patient.

KQ3. Evidence Available:

• State of science:
  • Concepts.
  • Content:
    • PICOTS.
  • Methods.
  • Context-sensitivity.

Slide 23

Results: Information

Disability Outcomes Review:

• 71 measures identified covering participation, body function, functional status, depression, HRQoL, and health status.
• No evidence for using modifiers or case-mix adjustment of general population measures applied to disabled populations.
• >100 measures of care coordination in context of community-based care for people with disabilities.
  • Health, level of functioning, costs, healthcare utilization most common concepts
• Key measure sources identified:
  • Rehabilitation Outcomes Database (http://www.rehabmeasures.org/). Measures for use patients with stroke or spinal cord injuries. Soon also for traumatic brain injury.
  • National Core Indicators (NCI) collaborative. Standard set of measures for evaluating quality of developmental disability services.

Slide 24

Results: Incentives

Bundled Payment:

• Small (<10%) decreases in spending vs. FFS models.
• 5-15% lower utilization, especially hospital LOS.
• Mixed impact on quality.

Public Reporting:

• Most studies show improvements in quality with public reporting, especially mortality.
• Physicians and health delivery organizations made changes in response to reporting; patients generally did not.

Medication Adherence:*

• Medication adherence improved when patients' out-of-pocket costs were reduced.

*Although this topic is part of Infrastructure, one Key Question addressed effectiveness of policy interventions with an Incentive component, specifically reducing patient costs.

Slide 25

Results: Infrastructure

Disparities:

• Only one study showed reduced disparity with QI intervention: black but not white patients increased HbA1c testing following patient education.
• Other suggestive results (but no reduction in disparities):
  • Amplified effects of patient education, collaborative care in minority populations.
  • Language- and literacy-concordant education more effective for non-English speakers.

Palliative Care:

• Several intervention strategies improved some, but not all, outcomes:
  • Pain improved when targeted, but not QOL.
  • Targeting communication/decision-making improved utilization but not satisfaction.
  • Targeting coordination improved satisfaction, but not QOL, symptoms or utilization.
• Patient-centered interventions were generally more effective.

Slide 26

Results: Infrastructure

PCMH:

• PCMH interventions showed small to moderate improvements in patient experience, delivery of preventive care services, staff experience, and use of ED by older adults. No evidence of cost savings.
• Amount of evidence on PCMH expected to double in next few years.

HAI:

• Most interventions use organizational change and/or provider education (base strategies), but alone these are not effective.
• Adherence and infection rates improved with combination of base strategies + audit and feedback; or base + audit & feedback + provider reminder systems.
• Base + provider reminders might also be effective.

Slide 27

Results: Infrastructure

Medication Adherence:

• Many interventions have potential to be effective.
• Patient education and case management showed most promise.
• Improved adherence does not necessarily mean improved outcomes.