Expanding the Uses of AHRQ's Prevention Quality Indicators: Validity from the Clinician Perspective

Slide presentation from the AHRQ 2010 conference.

On September 27, 2010, Sheryl Davies made this presentation at the 2010 Annual Conference. Select to access the PowerPoint® presentation (180 KB). Free PowerPoint® Viewer (Plugin Software Help).


Slide 1

 Expanding the Uses of AHRQ's Prevention Quality Indicators: Validity from the Clinician Perspective

Expanding the Uses of AHRQ's Prevention Quality Indicators: Validity from the Clinician Perspective

Presented by:

Sheryl Davies, MA
Stanford University
Center for Primary Care and Outcomes Research

AHRQ Annual Meeting
September 26 – 29, 2010
Bethesda, MD

Slide 2

 Acknowledgements

Acknowledgements

Project team:

  • Sheryl Davies, MA (Stanford)
  • Kathryn McDonald, MM (Stanford)
  • Eric Schmidt, BA (Stanford)
  • Ellen Schultz, MS (Stanford)
  • Olga Saynina, MS (Stanford)
  • Jeffrey Geppert JD (Battelle)
  • Patrick Romano, MS, MD (UC Davis)

AHRQ Project Officer: Mamatha Pancholi

  • This project was funded by a contract from the Agency for Healthcare Research and Quality (#290-04-0020)

Slide 3

 Potentially Avoidable Hospitalizations

Potentially Avoidable Hospitalizations

  • Admissions for diagnoses that may have been prevented or ameliorated with currently recommended outpatient care.
  • Two independently developed measure sets primarily used in the literature:
    • John Billings
    • Joel Weissman
  • Strong independent negative correlations between self-rated access and avoidable hospitalization.
  • Correlations between avoidable hospitalization and:
    • Household income at zip code level (neg)
    • Uninsured or Medicaid enrolled (pos)
    • Maternal education (neg)
    • Physician to population ratio (neg)
    • Weaker associations for Medicare populations

Slide 4

Prevention Quality Indicators: Background  

Prevention Quality Indicators: Background

  • Developed in early 2000s.
  • Numerator: Number of admissions within a geographic area.
  • Denominator: Population.
  • Some admissions are excluded if considered relatively less preventable.
  • Conditions selected had adequate variation, signal ratio, and literature based evidence supporting use.

Slide 5

Prevention Quality Indicators  

Prevention Quality Indicators

  • Diabetes related indicators:
    • Diabetes, short-term complications (PQI 1)
    • Diabetes, long-term complications (PQI 3)
    • Lower extremity amputations among patients with diabetes (PQI 16)
  • Chronic disease indicators:
    • Chronic obstructive pulmonary disease (PQI 5)
    • Hypertension (PQI 7)
    • Congestive heart failure (PQI 8)
    • Angina without procedure (PQI 13)
    • Adult asthma (PQI 15)
  • Acute disease indicators:
    • Perforated appendicitis (PQI 2)
    • Dehydration (PQI 10)
    • Bacterial pneumonia (PQI 11)
    • Urinary infections (PQI 12)

Slide 6

Potential uses of PQIs  

Potential uses of PQIs

 QIComp ReportP4P
Area X
(Current application)
 
Payor X
(Extended application)
X
(Extended application)
ProviderX
(Extended application)
X
(Extended application)
X
(Extended application)
LTCX
(Extended application proposed by panel)
X
(Extended application proposed by panel)
X
(Extended application proposed by panel)

Note: We initially assessed the internal quality improvement application for large provider groups. Following our initial rating period, panelists expressed interest in applying select indicators to the long term care setting and these applications were added to our panel questionnaire.

Slide 7

Scenarios of use  

Scenarios of use

  • Area level—Publish maps of rates by county. Target areas with higher rates
  • Payors (SCHIP, Medicare Advantage, private plans):
    • CR: Publicly report payor rates to improve consumer choice
    • P4P: Medicaid agencies implementing P4P for contracted payor groups
  • Provider (large provider groups)/LTC:
    • QI: Analyze rates to identify potential intervention targets (e.g., care coordination, education)
    • CR: Publicly report provider rates to improve consumer choice
    • P4P: Payors implementing P4P programs for contracted provider groups

Slide 8

 Methods

Methods

  • Clinical Panel review using new hybrid Delphi/Nominal Group technique
  • Two groups: Core and Specialist:
    • Core assesses all; Specialist only applicable
  • Three indicator groups: Acute, Chronic, Diabetes
  • Two panels:
    • Delphi
    • Nominal Group

Slide 9

Delphi vs. Nominal  

Delphi vs. Nominal

  • Delphi group:
    • Advantages: Better reliability, more points of view, less chance for one panelist to pull the group
    • Disadvantage: Less communication and cross-pollination across panelists, less ability to discuss and refine details of indicators/evaluation.
  • Nominal group:
    • Advantages: Can discuss details, facilitate sharing of ideas.
    • Disadvantages: Limited in size and therefore representation, one strong panelist can flavor group and therefore poorer reliability.

Slide 10

 Panel Process: Exchange of Information

Panel Process: Exchange of Information

Image: A diagram shows the exchange of information between panels, outlining the hybrid methodology used in this study. The nominal group and delphi group each individually rated the indicators via an E-mailed questionnaire. They both had opportunity to comment on the initial results, and those initial results and comments were distributed to the nominal group prior to a series of 3 conference calls. The conference calls were grouped by clinical area, namely diabetes, acute disease, and chronic disease. Summaries from those called were distributed to panels who re-rated the indicators again using an E-mailed questionnaire. Ratings were combined to obtain the final ratings.
 

Slide 11

Quality Improvement Applications  

Quality Improvement Applications

IndicatorProvider(Delphi/Nominal)
COPD and Asthma (40 yrs +)*****
Asthma ( < 39 yrs)******
Hypertension*****
Angina****
CHF******
Perforated Appendix***
Diabetes Short Term Complications******
Diabetes Long-Term Complications*****
Lower Extremity Amputation*****
Bacterial Pneumonia****
UTI****
Dehydration***

* Major Concern Regarding Use
** Some Concern
*** Majority Support
**** Full Support

Slide 12

Comparative Reporting Applications  

Comparative Reporting Applications

IndicatorAreaPayorProvider
COPD**/****/****/***
Asthma ( < 39 yrs)**/*****/*****/***
Hypertension**/*****/*****/**
Angina**/****/***/*
CHF**/*****/******/***
Perforated Appendix**/***/***/*
Diabetes Short Term Complications**/****/*****/***
Diabetes Long-Term Complications**/*****/****/**
Lower Extremity Amputation***/*****/*****/**
Bacterial Pneumonia**/****/****/**
UTI**/****/****/**
Dehydration**/****/***/*

* Major Concern Regarding Use
** Some Concern
*** Majority Support
**** Full Support

Slide 13

 Pay for Performance Applications

Pay for Performance Applications

IndicatorPayorProvider
COPD**/****/***
Asthma ( < 39 yrs)**/****/***
Hypertension**/*****/**
Angina**/****/*
CHF**/****/**
Perforated Appendix**/***/*
Diabetes Short Term Complications**/****/**
Diabetes Long-Term Complications**/****/**
Lower Extremity Amputation**/****/**
Bacterial Pneumonia**/****/**
UTI**/***/*
Dehydration**/**/*
* Major Concern Regarding Use
** Some Concern
*** Majority Support
**** Full Support

Slide 14

 Concordance Between Panels

Concordance Between Panels

NGDelphi Full supportDelphi Some ConcernDelphi Major Concern
NG Full support861 (6)10
NG Some concern0340
NG Major Concern012 (5)13

1Numbers in parentheses are the number of instances in that cell where Median (Delphi) - Median (NG) > 1.

  • Majority of combinations rated the same (56%).
  • Three combinations had one rating of "majority support" which requires disagreement within one panel (not shown on table).
  • Of remaining differences, all were within one level. Of those about 2/3 had a difference in medians of one or less.
  • Delphi panel always more moderate than NG.

Slide 15

 What feeds into the ratings?

What feeds into the ratings?

  • Linear regression on usefulness ratings:
    • Mixed model: panelist random effect (nested)
    • Fixed effects:
      • Delphi vs. NG (N.S.)
      • Generalist vs. Specialist (F=32.3, p<.0001)
      • Public Health vs. Other (F=20.0, p<.0001)
      • Quality vs. Other (F=54.7, p<.0001)
      • Denominator Level (F=24.4, p<.0001)
      • Use (F=23.2, p<.0001)
      • Indicator (F=8.5, p<.0001)

Slide 16

 Potential interventions to reduce hospitalizations

Potential interventions to reduce hospitalizations

 AcuteChronic
Area
  • Access to primary care/urgent care
  • Access to care
  • Lifestyle modifications
Payor
  • Coverage of medications
  • Coverage of auxiliary health services (e.g. at home nursing)
  • Access to primary care/urgent care
  • Coverage of medications
  • Coverage of comprehensive care programs
  • Coverage of auxiliary health services (e.g. at home nursing)
  • Disease management programs
  • Lifestyle modification incentives
Provider
  • Quality nursing triage
  • Patient education
  • Accurate/rapid diagnosis and treatment
  • Appointment availability
  • Outpatient treatment of complications
  • Education, disease management
  • Lifestyle medication interventions
  • Comprehensive care programs, care coordination, auxiliary health services

 

Slide 17

 So you want to adapt the PQI?

So you want to adapt the PQI?

  • Selecting indicators:
    • Stability of denominator group improves validity for long-term complications.
  • Defining the numerator:
    • One admission per patient per year.
    • Using related principal dx with target secondary dx.
    • Including first hospitalization before chronic condition dxed.
  • Defining the denominator:
    • Identifying patients with chronic diseases (multiple dx, population rates, pharmaceutical data).
    • Requiring minimum tenure with payor or provider.

Slide 18

 Risk adjustment

Risk adjustment

  • Demographics:
    • Age and gender highly rated as important.
    • Race depending on indicator.
  • Disease severity:
    • Historical vs. current data.
  • Comorbidity:
    • Highly rated as important.
  • Lifestyle associated risk and compliance:
    • Smoking, obesity.
    • Pharmacy records.
    • Can interventions help reduce impact of these factors?
  • Socioeconomic status:
    • Highly rated as important.
    • May mask true disparities in access to care.
    • Panel felt benefits of inclusion outweighed problems.

Slide 19

 Policy implications

Policy implications

  • Ensuring true quality improvement:
    • Case mix shifting, coding
  • Cost/burden of data collection.
  • Does avoiding hospitalization really reflect the best:
    • Quality?
    • Value?

Slide 20

 Next steps: Understanding stakeholder perspectives

Next steps: Understanding stakeholder perspectives

  • Results represent clinical perspective.
  • Other stakeholders may be more attuned to public health, access to care, quality uses.
  • Other important perspectives:
    • Public health
    • Long term Care
    • Policymakers
    • Quality stakeholders
  • Why are there differences in perspectives?

Slide 21

 Next steps

Next steps

  • Investigate multiple definitions.
  • Investigate risk adjustment approaches.
  • Continue to learn from user experience.
  • Identify interventions and link usefulness of indicators with true quality improvement.
Current as of December 2010
Internet Citation: Expanding the Uses of AHRQ's Prevention Quality Indicators: Validity from the Clinician Perspective. December 2010. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/news/events/conference/2010/davies/index.html