How Good is Good Enough? A Case Study with Present on Admission (Text Version)

On September 28, 2010, Jeff Geppert made this presentation at the 2010 Annual Conference. Select to access the PowerPoint® presentation (940 KB). Free PowerPoint® Viewer (Plugin Software Help).

Slide 1

How good is good enough? A case study with Present on Admission

AHRQ Annual 2010 Annual Conference
September , 2010
Session #45
Jeffrey Geppert, PMP, EdM, JD
Battelle Memorial Institute

Slide 2

Acknowledgements

• Support for Quality Indicators II (290-04-0020):
  • Mamatha Pancholi, AHRQ QI Project Officer
• HCUP State Inpatient Databases (SID). Healthcare Cost and Utilization Project (HCUP). 2004-2008. Agency for Healthcare Research and Quality, Rockville, MD. http://www.hcup-us.ahrq.gov/sidoverview.jsp

Slide 3

Why Measurement?

• What is the purpose of measurement?
  • Reduce uncertainty in decisionmaking
• What is the decision?
  • Consumers: what provider to select
  • Providers: where to allocate effort
• What is the uncertainty?
  • Consumers: would have selected a different provider
  • Providers: would have allocated effort differently

Slide 4

Measuring Uncertainty

• Expected Opportunity Loss (EOL):
  • Amount of loss X Probability of Loss
  • Loss relative to the best available alternative (opportunity)
• Expected Value of Information (EVI):
  • EOL^{After measurement} - EOL^{Before measurement}
  • EVI may be positive or negative
• Risk aversion:
  • If EVI = 0, then the decision is about risk

Slide 5

Measuring Uncertainty

• Measure Model:
  • Y = alpha + X*B
• Y - the outcome of interest
  • Numerator, denominator, exclusions
• X*B - patient characteristics
  • Moderators and mediators
• Alpha - provider quality
  • Prior ability (related outcomes, related procedures, structure, context, past performance)
  • Current effort (signal)

Slide 6

Iatrogenic Pneumothorax

A condition in which air or other gas is present in the pleural cavity and which occurs spontaneously as a result of disease or injury of lung tissue, rupture of air-filled pulmonary cysts, or puncture of the chest wall or is induced as a therapeutic measure to collapse the lung.

Source: AHRQ PSI Technical Specifications, PSI #6 (Version 4.1b).

Slide 7

POA Overview: Approach

• Two sets of algorithms needed to incorporate POA information:
  1. Develop response variables and comorbidity factor covariates in the presence of POA data.
     • Less measurement error thereby more accurate and based on fewer assumptions.
  2. Develop response variables and comorbidity factor covariates in the absence of POA data.
     • Use observed POA data to estimate probability of POA for response and comorbidity factors for patients that do not have POA data.
     • Provide hospital with risk-adjusted rate that would be "most likely" had they collected POA data.
• Present on Admission White Paper:
  • http://www.qualityindicators.ahrq.gov/downloads/Web inars/Using Present on Admission.pdf

Slide 8

Iatrogenic Pneumothorax

• Case study:
  • Select two hospitals (A and B) with POA data (median denominator and rate).
  • Randomly use POA data for X% of discharges and predict the missing
    • None (no POA data)
    • 10%—90%
    • All (full POA data)—"gold standard"
  • Estimate EOL based on events, deaths, inpatient days and charges (NIS 2000 analysis by Zhan & Miller, JAMA 2003;290:1868-74).

Slide 9

Expected Opportunity Loss

Image: Line graph displays the following information and hospital rates:

SID States:
Events: 8,306
Deaths: 581
Days: 36,547
Charges: $143,697,389

Hospital A:
Events: 3
Deaths: 0
Days: 12
Charges: $48,561

Hospital B:
Events: 4
Deaths: 0
Days: 16
Charges: $63,659

Slide 10

EVI from Measurement

Image: Line graph displays the following information and hospital rates:

Hospital A (Prior):
Events:  3
Deaths:  0
Days:  12
Charges:   $48,561

Hospital A (Post): ;
Events:  5
Deaths:  0
Days:  24
Charges:  $94,370

Slide 11

EVI from Measurement

Image: Line graph displays the following information and hospital rates:

Hospital B (Prior):
Events:  4
Deaths:  0
Days:  16
Charges:  $63,659

Hospital B (Post):
Events:  5
Deaths:  0
Days:  22
Charges:  $87,472

Slide 12

EVI from Measurement

Image: Line graph displays the following information and hospital rates:

Hospital A (Prior):
Events: 5
Deaths: 0
Days: 24
Charges: $94,370

Hospital A (Post):
Events: 6
Deaths: 0
Days: 27
Charges: $107,226
 

Slide 13

EVI from Measurement

Image: Line graph displays the following information and hospital rates:

Hospital B (Prior):
Events:  5
Deaths:  0
Days:  22
Charges:  $87,472

Hospital B (Post):
Events:  6
Deaths:  0
Days:  26
Charges:  $100,306

Slide 14

EVI from Measurement

Image: A tables displays the following information:

Slide 15

Using Imperfect Measures

• The use of any measure (Q_m) that is positively correlated with the true measure (Q_t) results in overall improvement in outcomes.
• This is not true only if:
  • Difference between Q_t and Q_m
  • Variation in Q_t - Q_m among providers
  • Negative correlation between Q_t and Q_m

Slide 16

Using Imperfect Measures

Image: A graph shows infections due to Medical Care (PSI#7) for None (No POA Data) / All (Full POA Data). Most incidences (represented by dots) appear within the range of 0.03 None / 0.03 Full.

Slide 17

Using Imperfect Measures

The table shows actual reduction in adverse events in moving patients from worse to best providers based on an imperfect measure.

Slide 18

Risk Aversion

• A consumer may rationally select a provider with worse measured quality:
  • If the consumer is risk averse and prefers a more certain outcome over the greater risk of a worse outcome.
  • If the consumer has external information about the provider that reduces uncertainty.

Slide 19

Risk Aversion

• A consumer may rationally select a provider with worse measured quality:
  • If the consumer is risk averse and prefers a more certain outcome over the greater risk of a worse outcome.
  • If the consumer has other information about the provider that reduces uncertainty.

Slide 20

Risk Aversion

Image: Line graph shows Iatrogenic Pneumothorax (PSI #06) rates for Hospital B EOL vs. Hospital A EOL. Hospital A has a sharp spike, while Hospital B is an even bell curve.

Slide 21

Conclusions

• Use expected value of information (EVI) to quantify the benefit of adding enhanced data, collecting data over time.
• Largest marginal gain in EVI may come from:
  • Initial measurement
  • Systemic relationships
• Smallest marginal gain in EVI may come
  • Subsequent measurement
  • Provider specific measurement

Slide 22

Conclusions

• Target data collection to the measures, consumers and providers with highest marginal EVI.
• Give consumers information about uncertainty to allow them to incorporate preferences about risk and external information.

Slide 23

Questions?

Image: An illustration of a microphone is shown.