Expanding the Uses of AHRQ's Prevention Quality Indicators: Validity from the Clinician Perspective

On September 27, 2010, Sheryl Davies made this presentation at the 2010 Annual Conference. Select to access the PowerPoint® presentation (180 KB). Free PowerPoint® Viewer (Plugin Software Help).

Slide 1

Expanding the Uses of AHRQ's Prevention Quality Indicators: Validity from the Clinician Perspective

Presented by:

Sheryl Davies, MA
Stanford University
Center for Primary Care and Outcomes Research

AHRQ Annual Meeting
September 26 – 29, 2010
Bethesda, MD

Slide 2

Acknowledgements

Project team:

• Sheryl Davies, MA (Stanford)
• Kathryn McDonald, MM (Stanford)
• Eric Schmidt, BA (Stanford)
• Ellen Schultz, MS (Stanford)
• Olga Saynina, MS (Stanford)
• Jeffrey Geppert JD (Battelle)
• Patrick Romano, MS, MD (UC Davis)

AHRQ Project Officer: Mamatha Pancholi

• This project was funded by a contract from the Agency for Healthcare Research and Quality (#290-04-0020)

Slide 3

Potentially Avoidable Hospitalizations

• Admissions for diagnoses that may have been prevented or ameliorated with currently recommended outpatient care.
• Two independently developed measure sets primarily used in the literature:
  • John Billings
  • Joel Weissman
• Strong independent negative correlations between self-rated access and avoidable hospitalization.
• Correlations between avoidable hospitalization and:
  • Household income at zip code level (neg)
  • Uninsured or Medicaid enrolled (pos)
  • Maternal education (neg)
  • Physician to population ratio (neg)
  • Weaker associations for Medicare populations

Slide 4

Prevention Quality Indicators
Background

• Developed in early 2000s.
• Numerator: Number of admissions within a geographic area.
• Denominator: Population.
• Some admissions are excluded if considered relatively less preventable.
• Conditions selected had adequate variation, signal ratio, and literature based evidence supporting use.

Slide 5

Prevention Quality Indicators

• Diabetes related indicators:
  • Diabetes, short-term complications (PQI 1)
  • Diabetes, long-term complications (PQI 3)
  • Lower extremity amputations among patients with diabetes� (PQI 16)
• Chronic disease indicators:
  • Chronic obstructive pulmonary disease (PQI 5)
  • Hypertension (PQI 7)
  • Congestive heart failure (PQI 8)
  • Angina without procedure (PQI 13)
  • Adult asthma (PQI 15)
• Acute disease indicators:
  • Perforated appendicitis (PQI 2)
  • Dehydration (PQI 10)
  • Bacterial pneumonia (PQI 11)
  • Urinary infections (PQI 12)

Slide 6

Potential uses of PQIs

Note: We initially assessed the internal quality improvement application for large provider groups. Following our initial rating period, panelists expressed interest in applying select indicators to the long term care setting and these applications were added to our panel questionnaire.

Slide 7

Scenarios of use

• Area level—Publish maps of rates by county. Target areas with higher rates
• Payors (SCHIP, Medicare Advantage, private plans):
  • CR: Publicly report payor rates to improve consumer choice
  • P4P: Medicaid agencies implementing P4P for contracted payor groups
• Provider (large provider groups)/LTC:
  • QI: Analyze rates to identify potential intervention targets (e.g., care coordination, education)
  • CR: Publicly report provider rates to improve consumer choice
  • P4P: Payors implementing P4P programs for contracted provider groups

Slide 8

Methods

• Clinical Panel review using new hybrid Delphi/Nominal Group technique
• Two groups: Core and Specialist:
  • Core assesses all; Specialist only applicable
• Three indicator groups: Acute, Chronic, Diabetes
• Two panels:
  • Delphi
  • Nominal Group

Slide 9

Delphi vs. Nominal

• Delphi group:
  • Advantages: Better reliability, more points of view, less chance for one panelist to pull the group
  • Disadvantage: Less communication and cross-pollination across panelists, less ability to discuss and refine details of indicators/evaluation.
• Nominal group:
  • Advantages: Can discuss details, facilitate sharing of ideas.
  • Disadvantages: Limited in size and therefore representation, one strong panelist can flavor group and therefore poorer reliability.

Slide 10

Panel Process: Exchange of Information

Image: A diagram shows the exchange of information between panels, outlining the hybrid methodology used in this study. The nominal group and delphi group each individually rated the indicators via an E-mailed questionnaire. They both had opportunity to comment on the initial results, and those initial results and comments were distributed to the nominal group prior to a series of 3 conference calls. The conference calls were grouped by clinical area, namely diabetes, acute disease, and chronic disease. Summaries from those called were distributed to panels who re-rated the indicators again using an E-mailed questionnaire. Ratings were combined to obtain the final ratings.
 

Slide 11

Quality Improvement Applications

* Major Concern Regarding Use
** Some Concern
*** Majority Support
**** Full Support

Slide 12

Comparative Reporting Applications

* Major Concern Regarding Use
** Some Concern
*** Majority Support
**** Full Support

Slide 13

Pay for Performance Applications

Slide 14

Concordance Between Panels

¹Numbers in parentheses are the number of instances in that cell where Median (Delphi) - Median (NG) > 1.

• Majority of combinations rated the same (56%).
• Three combinations had one rating of "majority support" which requires disagreement within one panel (not shown on table).
• Of remaining differences, all were within one level. Of those about 2/3 had a difference in medians of one or less.
• Delphi panel always more moderate than NG.

Slide 15

What feeds into the ratings?

• Linear regression on usefulness ratings:
  • Mixed model: panelist random effect (nested)
  • Fixed effects:
    • Delphi vs. NG (N.S.)
    • Generalist vs. Specialist (F=32.3, p<.0001)
    • Public Health vs. Other (F=20.0, p<.0001)
    • Quality vs. Other (F=54.7, p<.0001)
    • Denominator Level (F=24.4, p<.0001)
    • Use (F=23.2, p<.0001)
    • Indicator (F=8.5, p<.0001)

Slide 16

Potential interventions to reduce hospitalizations

Slide 17

So you want to adapt the PQI?

• Selecting indicators:
  • Stability of denominator group improves validity for long-term complications.
• Defining the numerator:
  • One admission per patient per year.
  • Using related principal dx with target secondary dx.
  • Including first hospitalization before chronic condition dxed.
• Defining the denominator:
  • Identifying patients with chronic diseases (multiple dx, population rates, pharmaceutical data).
  • Requiring minimum tenure with payor or provider.

Slide 18

Risk adjustment

• Demographics:
  • Age and gender highly rated as important.
  • Race depending on indicator.
• Disease severity:
  • Historical vs. current data.
• Comorbidity:
  • Highly rated as important.
• Lifestyle associated risk and compliance:
  • Smoking, obesity.
  • Pharmacy records.
  • Can interventions help reduce impact of these factors?
• Socioeconomic status:
  • Highly rated as important.
  • May mask true disparities in access to care.
  • Panel felt benefits of inclusion outweighed problems.

Slide 19

Policy implications

• Ensuring true quality improvement:
  • Case mix shifting, coding
• Cost/burden of data collection.
• Does avoiding hospitalization really reflect the best:
  • Quality?
  • Value?

Slide 20

Next steps
Understanding stakeholder perspectives

• Results represent clinical perspective.
• Other stakeholders may be more attuned to public health, access to care, quality uses.
• Other important perspectives:
  • Public health
  • Long term Care
  • Policymakers
  • Quality stakeholders
• Why are there differences in perspectives?

Slide 21

Next steps

• Investigate multiple definitions.
• Investigate risk adjustment approaches.
• Continue to learn from user experience.
• Identify interventions and link usefulness of indicators with true quality improvement.