Slide Presentation from the AHRQ 2008 Annual Conference
On September 10, 2008, Kathryn McDonald made this presentation at the 2008 Annual Conference. Select to access the PowerPoint® presentation (45 KB; Plugin Software Help).
Future Validation and Improvement of the Agency for Healthcare Research and Quality (AHRQ) Quality Indicator (QI)
AHRQ Annual Conference
September 10, 2008
Presented by Kathryn McDonald
Validation: Where we have been
- AHRQ QI team activities:
- Face validity of indicators through panel review.
- Coding validity through chart review of selected Patient Safety Indicators (PSIs).
- Outside research:
- Present on Admission (POA).
- Coding validity.
- Resource use.
Validation: Where we are headed next
- Continued chart reviews:
- More PSIs, estimating false positives.
- Examination of under-reporting of PSIs.
- Creation of chart review tools.
- Prevention Quality Indicators validation:
- Extending the face validity of the indicators through panel review.
- New tools for using the PQIs.
- Additional work on Patient Safety Indicators:
- Assessment of new coding and POA.
- Additional work on Neonatal Indicators/Pediatric Indicators (PDIs):
- Assessment of coding validity for bloodstream infection (BSI).
- Risk adjustment development and validation.
Our Progress Depends on Collaboration
- Continuous feedback loop from validation work and user feedback creates better indicators.
- Spans all types of research projects:
- Peer reviewed to individual hospital investigations.
- Informs potential feedback on coding, guidance on indicator use, and in some cases indicator reassignment.
- Motivates additional validation studies.
Research and Use Cycle
- Graphic depicts continuous cycle:
- QI User Feedback and Validity Research.
- Assess the Generalizability of the Experience or Findings.
- Design and Test Indicator Improvements.
Examples of Cycle Benefit: Indicator Reassignment
Complications of Anesthesia:
- Several user reports noted that minor reactions such as pruritis were coded.
- Investigation of coding guidelines found that E-codes used in this indicator allowed for coding of these minor reactions.
- No way to "fix" indicator found.
- Indicator will be reassigned as an "experimental indicator."
Examples of Cycle Benefit: Modifying Guidance
Present on Admission Research:
- Several recent studies have highlighted POA rates in the Patient Safety Indicators.
- Revealed indicators for which POA is an important data element.
- Guidance to use these indicators only with POA, software modified to require POA in some cases, and National Quality Forum (NQF) endorsement conditional on POA.
Examples of Cycle Benefit: Modifying Coding Structure
- Intricacies in coding aren't always obvious in ICD-9-CM coding.
- National Association of Children's Hospitals and Related Institutions (NACHRI) supported research of the PDIs identified cases of transfusion reaction from minor antigens instead of ABO.
- Minor antigens indexed to ABO code.
- Proposal to separate transfusion reactions due to ABO from those due to minor antigens to improve the specificity of this indicator.
Examples of Cycle Benefit: Improving the Indicator
- Initial definition relied on diagnosis code for identifying numerator cases.
- VA based study found low sensitivity for dx code.
- Further investigation by Department of Veterans Affairs (VA) team identified procedure codes for delayed extubation and post-operative re-intubation improved sensitivity without significantly decreasing specificity.
- AHRQ team investigated generalizability of these findings using the Healthcare Cost & Utilization Project (HCUP) data and...
- Added the new procedure codes to the definition.
- But the story doesn't end there...
Examples of Cycle Benefit: Improving the Indicator (continued)
Respiratory Failure (continued):
- Procedure codes were imported into new pediatric indicator.
- NACHRI directed study identified children with expected extended intubations not related to respiratory failure.
- Consulting with experts and through data analysis AHRQ team identified specific operations for which intubation is extended (e.g. tracheal procedures).
- Some cases generalizable to the adult indicator.
- Continued work with NACHRI team to ensure that the solution truly improves specificity of indicator.
The Path for Future Research
- Many user experiences, small and large validation studies happening outside of AHRQ.
- These studies allow us to improve indicators and ultimately hospital quality.
- Tell us about your research on:
- Validation (sensitivity and specificity).
- Quality Improvement Programs.
- Special Populations.
Current as of February 2009
Future Validation and Improvement of the AHRQ QI. Slide Presentation from the AHRQ 2008 Annual Conference (Text Version). February 2009. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/about/annualmtg08/091008slides/McDonald.htm