Slide Presentation for Mortality Measurement Meeting
On November 3, 2008, Gabriel Escobar, M.D., and Marta Render, M.D., made this presentation at the 2008 Annual Conference.
Select to access the PowerPoint® presentation (94 KB).
Slide presentations can be accessed using a Free PowerPoint® Viewer (Plugin Software Help).
Mortality Measurement in the Electronic Medical Record (EMR) Era: What Real Time Lab and Clinical Data Can Contribute to Precision and Prediction
Gabriel J. Escobar, MD
Marta Render, MD
John D. Greene, MA
Marla N. Gardner, BA
Patricia Kipnis, PhD
Topics to be Covered
- Value of readily available enhancements:
- Present-on-Admission (POA) coding
- Laboratory data
- Other physiologic data
- Work in progress: use of bed history data combined with laboratory data
Why should we settle for the lowest common denominator?
- Billing data are driven by need for reimbursement
- High propensity to being "gamed"
- "Coding creep" occurs
- Many diagnoses strongly associated with outcome can be present on admission, e.g., stroke, deep vein thrombosis (DVT), pressure ulcers
Glance et al.: Impact of Present on Admission Coding on Performance Ranking
- POA coding modifier showed that diagnoses associated with complications were present on admission only 10—22% of the time.
- Absence of POA coding led to 33—40% of low performing hospitals not being detected for these conditions.
- Coronary artery bypass grafting (CABG)
- Coronary angioplasty
- Hip replacement
- Acute myocardial infarction (AMI)
- "Treating complications as pre-existing conditions gives poor-performing hospitals 'credit' for their complications and may cause some hospitals that are delivering low-quality care to be misclassified as average- or high-performing hospitals."
Value of Incorporating Physiologic Data
- Increasingly available, particularly in hospital chains
- Much less expensive than manual chart abstraction
- Have tremendous face validity with clinicians
- Relatively easy to combine with other data
- Can be used either for disease-specific models (e.g., Fine Patient Safety Indicators (PSI) for community acquired pneumonia) or for global risk adjustment (e.g., Department of Veterans' Affairs (VA) or Kaiser Permanente risk adjustment methodologies)
Relative Contribution of Physiologic Data to Overall Model Performance
- Operational use of laboratory data first occurred in the intensive care unit (ICU).
- Render et al.—29,377 consecutive first ICU admissions in 17 VA hospitals:
- Laboratory data accounted for 74% of model predictive ability
- Diagnosis accounted for 13%
- Zimmerman et al.—110,558 ICU admissions in 45 U.S. hospitals:
- Laboratory data accounted for 65% of model predictive ability
- Diagnosis accounted for 16%
Pine et al.: Use of Laboratory Data for Non-ICU Populations
- Quantified effect of adding POA coding, laboratory data, and vital signs data for 5 conditions and 3 procedures.
- Not restricted to ICU; N ranged from 5309 for abdominal aortic aneurysms (AAA) to 200,506 for congestive heart failure (CHF).
- Average effect of adding predictors, as evidenced by change in c statistic:
|No risk adjustment
|POA + labs
|POA + labs + V S S
Tabak et al.: Definitive Quantification of Value of Laboratory Data
- Evaluated 6 disease-specific mortality predictive models for pay-for-performance (ischemic and hemorrhagic stroke, pneumonia, CHF, and sepsis).
- 194,903 admissions in 2000-2003 across 71 hospitals that imported laboratory data.
- Quantified relative contribution with omega statistic.
- Laboratory data were between 2 and 67 times more important in predicting mortality than ICD-9 variables.
- Only models where laboratory data were less important were those for stroke, where altered mental status recordings were more important.
Research in Progress: Combining Laboratory Data with Another EMR Marker, Bed History
- 207,922 hospitalizations at 19 Kaiser Permanente hospitals, 11/1/06-1/31/08
- All severity scored using laboratory acute physiology score (LAPS) and pre-admission comorbidity point score (COPS)
- Employs time stamps for patient arrival at different units (ward, OR/PAR, TCU, ICU)
- Examines mortality of intra-hospital transfers combined with laboratory testing patterns
Categorization of Intra-hospital Transfers to a Higher Level of Care
- Post-surgical (OR/PAR to TCU, OR/PAR to ICU)
- Unplanned (ward to TCU, ward to ICU, TCU to ICU)
- Laboratory testing patterns (blood gases, lactate, troponin I, blood culture)
- Not tested during 16 hour (12-4) time window surrounding transfer
- Tested during time window but not before
- Tested before and during time window
- If index test obtained in 16 hour window (12 hrs before to 4 hours after transfer to higher level of care) patient is considered to have been tested, otherwise patient is considered "not tested."
- If another test is located within 24-48 hours (depending on test type) preceding index test, the patient is considered "previously tested."
- If another test is not located within 24-48 hours preceding index test, the patient is considered "newly tested."
Outcomes Based on First Hospital Unit
Outcomes Based on Intra-hospital Transfer Type—I
|Never in TCU or ICU
|Direct admit to ICU (all)
|Experienced unplanned transfer
|Direct admit to TCU (all)
|Experienced unplanned transfer
Outcomes Based on Intra-hospital Transfer Type—II
|Post-surgical transfers to TCU or ICU
Outcomes Based on Intra-hospital Transfer Type—III
|Unplanned transfer to TCU or ICU
OEMR For Unplanned Transfers in 17 Kaiser Permanente Hospitals
Chart with facility on the x axis and observed to expected mortality ratio on the y axis. The ratios range from 2 to approximately 3.8.
Elapsed Hospital Length of Stay (ELOS) at Time of Death
||ELOS (days) @ time of death (median, mean plus or minus SD)
||3.8, 6.0 plus or minus 8.9
|TCU/ICU Direct Admits
||4.7, 10.1 plus or minus 19.6
||12.9, 21.8 plus or minus 32.2
||10.0, 17.9 plus or minus 26.0
Next EMR Challenge: Identification of Do-Not-Resuscitate (DNR)/Comfort Care Patients
Holloway and Quill:
- Mortality is a good quality measure for individuals with acute illness who are not supposed to die but is a poor quality measure for the majority of patients with multiple chronic diseases who are near the end of their life.
- "[T]aken alone, short-term mortality measures essentially treat death as a medical failure and reinforce avoiding death at all costs."
- Careful use of EMRs could permit excluding these patients from analyses (or separating them for different types of analyses).
- Evidence base for use of physiologic data in risk adjustment is overwhelming.
- Two entities, the VA system and Kaiser Permanente, are already employing physiologic-based risk adjustment operationally.
- Other uses of physiologic data (e.g., multivariable template matching for targeted case-control studies; VA's use of change in serum creatinine) make the use of these data even more compelling
- The important question is not "Should one employ automated physiology-based risk-adjustment?" but, rather, "Is it possible to convince different institutions to standardize data definitions so as to permit larger collaborative studies?"
- Entities such as IHI and AHRQ should be creating incentives for use of these data, rather than simply reinforcing the use of administrative data
- Not all hospitals have the capability to employ these data, but for those that do, IHI/AHRQ should encourage creation of networks that collaborate to employ data from EMRs
- Future research using EMRs should emphasize capture of D N R/comfort care orders
Veterans Administration Risk Adjusted Mortality in the ICU
- Risk model
- Diagnosis, comorbid disease, source of admission, worst of 11 lab values (sodium, glucose, BUN, Creatinine, albumin, Bilirubin, white blood count, hematocrit, pH/PaCO2, PaO2
- Separate models predict death at 30 days and at hospital discharge
- Advantage in addition of physiologic data
- Face validity/improved discrimination calibration to fairly portray risk and outcomes
- Unexpected advantage of using lab values in model
- Ability to create metrics related to labs:
- Mean hyperglycemia.
- Hypoglycemic rate/ patient days on hypoglycemic agent.
- Rate of acute kidney injury (AKI).
- Ability to link relationships:
- Troponin with mortality in medical patients and/or beta blocker use
- CHF readmission rates with ACE AKI
The VA Inpatient Evaluation Center
Aggregate VA ICU Standardized Mortality Rate
Chart showing year on the x axis and SMR on the y axis. Years range from 2002 to 2008. SMR ranges from 0 to 1.4. Values are shown for SMR 30 and SMR Hosp, as follows:
SMR 30: 2002, 1.099; 2003, 1.063; 2004, 1.065; 2005, 1.008; 2006, 0.996; 2007, 0.997; 2008, 0.940
SMR Hosp: 2002, 1.194; 2003, 1.152; 2004, 1.144; 2005, 1.059; 2006, 1.012; 2007, 0.977; 2008, 0.923
Mortality data and VA ICUs
- Multiple mortality measures tell you different things:
- Unadjusted mortality at hospital discharge or at 30-days for ICU and acute care patients (Are the admission criteria similar; access).
- Risk adjusted mortality at hospital discharge or at 30-days for ICU and Acute care patients (The difference may address issues of alternative access for placement for chronic acute illness/ futile care).
- Unadjusted mortality of patients transferred from the ward to the ICU (ability to detect and rescue deteriorating patients).
- Unadjusted mortality graphed against SMR may tell you about risk of underperforming hospital (vulnerability of the healthcare organization and patients).
Use of the VA ICU mortality measures
- Reported every quarter; web based; regional access with national benchmarks stratified by level and type of ICU.
- "Boots on the grounds," Targeted site visits, identification of unmeasured variables, Review of low predicted mortality patients who die, use of evidenced based practices, recommendations, follow-up in 6 months.
- Case mix index (ICU predicted mortality/all ICU predicted mortality).
- Adjusted bed turns, Track severity at ICUs with more limited services
- Targeting groups of patients for ICU LOS reduction (<2.5% predicted mortality)
- Evaluate effectiveness of initiatives:
The association of hyperglycemia with mortality varies by diagnosis
Three charts show odds ratio for mortality for three conditions by mean glucose. Odds ratio is on the x axis and mean glucose (milligrams per deciliter) is on the y axis. For unstable angina, the approximate odds ratios are: mean glucose 111 to 145, 1.7; mean glucose 146 to 199, 2.3; mean glucose 200 to 300, 3; mean glucose greater than 300, 6. Number of deaths is 23, 274. For pneumonia, the approximate odds are: mean glucose 111 to 145, 1.2; mean glucose 146 to 199, 1.9; mean glucose 200 to 300, 1.8; mean glucose greater than 300, 1.8. Number of deaths is 8,449. For sepsis, the approximate odds are: mean glucose 111 to 145, 1.2; mean glucose 146 to 199, 1.5; mean glucose 200 to 300, 2.3; mean glucose greater than 300, 3.4. Number of deaths is 4,179.
Even mild renal injury (Creatinine greater than 0.3 milligrams per deciliter) increases mortality risk and LOS
|Overall (n, 310,323)
|No AKI (n, 244,550)
||- 0.44 (3.3)
|Stage I (n, 52,773)
|Stage II (n, 7,744)
|Stage III (n, 3,271)
|Stage III-D (n, 1,985)
(CI—confidence limits; AKI—acute kidney injury; OMELOS—observed minus expected length of stay; SD—standard deviation)
Zimmerman JE, Kramer AA, McNair DS, et al. Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today's critically ill patients. Critical Care Medicine 2006;34:1297-1310.
Render ML, Kim HM, Welsh DE, et al. Automated intensive care unit risk adjustment: results from a National Veterans Affairs study. Critical Care Medicine 2003;31:1638-46.
Glance LG, Dick AW, Osler TM, et al. Accuracy of hospital report cards based on administrative data. Health Services Research 2006;41:1413-37.
Holloway RG, Quill TE. Mortality as a measure of quality. Implications for palliative and end-of-life care. JAMA 2007; 298:802-4.
Tabak Y P, Johannes R S, Silber J H. Using automated clinical data for risk adjustment. Development and validation of six disease-specific mortality predictive models for pay-for performance. Medical Care 2007; 45:789-805.
Pine M, Jordan H S, Elixhauser A, et al. Enhancement of claims data to improve risk adjustment of hospital mortality. JAMA 2007;297:71-6.
Escobar G J, Greene J D, Scheirer P, Gardner M, Draper D, Kipnis P. Risk Adjusting Hospital Inpatient Mortality Using Automated Inpatient, Outpatient, and Laboratory Databases. Medical Care 2008; 46(3):232-9.
Return to Contents
Proceed to Next Section
Current as of February 2009
Mortality Measurement in the EMR Era: What Real Time Lab and Clinical Data Can Contribute to Precision and Prediction. Slide Presentation for Mortality Measurement Meeting (Text Version). February 2009. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/qual/mortality/Escobar.htm