Slide Presentation from the Webinar on AHRQ Quality Indicators Toolkit for Hospitals
The Agency for Healthcare Research and Quality (AHRQ) developed a toolkit to help hospitals understand the Quality Indicators (QIs). To orient users to the toolkit, AHRQ held a Web seminar on February 15, 2012. The following is a text version of the slide presentation made by Donna O. Farley and Peter Hussey.
Select to access the PowerPoint® presentation (725 KB). Plugin Software Help.
Development and Testing of the AHRQ QI Toolkit for Hospitals
Donna O. Farley, PhD
Peter Hussey, PhD RAND Corporation
What Is the Toolkit?
- Set of tools that hospitals can use to help improve performance in quality and patient safety.
- The AHRQ Quality Indicators (QIs):
- Inpatient Quality Indicators (IQIs).
- Patient Safety Indicators (PSIs).
- Targeted to wide range of hospitals:
- Independent or system-affiliated.
- Varying quality improvement experience.
- Developed through a Task Order in the AHRQ ACTION program.
- RAND partnered with UHC to develop and test the toolkit.
- AHRQ will continue toolkit support.
How Hospitals Can Use the Toolkit
- Applicable for hospitals with differing knowledge, skills, and needs.
- Serves as a "resource inventory" from which hospitals can select tools.
- Different audiences for each tool
(e.g., quality officer, finance officer, programmer).
What Are the Quality Indicators?
- Inpatient Quality Indicators—28 indicators of quality in four sets:
- Volume, counts (6).
- Mortality for conditions, rates (7).
- Mortality for procedures, rates (8).
- Utilization, rates (7).
- Patient Safety Indicators—17 indicators and a composite indicator:
- Screen for adverse events for inpatients.
- Expressed as rates.
The Development Process
Image: Flowchart shows the following steps:
- Develop Alpha Toolkit:
- Identify tools to include.
- Develop draft tools.
- Field Test:
- Improvement experiences.
- Usability of toolkit.
- Effects on QI values.
- Revise and finalize Toolkit for Dissemination.
Tool Development Steps
- Established principles to guide toolkit development.
- Reviewed literature to guide design.
- Developed outline of toolkit based on steps of a quality improvement process.
- Identified and developed specific tools for each step.
Technical Advisory Panel
- Six-member panel.
- Brought various skills and perspectives:
- Hospital experience.
- Quality improvement.
- Relevant research skills.
- Provided guidance throughout toolkit development:
- Toolkit design principles.
- Content of the tools.
- Parsimony in tool choice and design.
- Target the most important factors for implementation.
- Provide tools that offer most value for a range of hospitals.
- Readily accessible contents.
- Enable hospitals to assess effectiveness of their actions.
Structure of the Toolkit
Introduction and Roadmap
- Readiness to Change.
- Applying QIs to the Hospital Data.
- Identifying Priorities for Quality Improvement.
- Implementation Methods.
- Monitoring Progress and Sustainability
- Return-on-Investment Analysis.
- Existing Quality Improvement Resources.
- A navigational guide through the toolkit.
- For each tool, it summarizes:
- Action step being taken.
- Brief description of the tool.
- Key audience(s) to use the tool.
- Position with lead role responsibility.
Field Test Design
- Quality improvement collaborative.
- Conducted by UHC.
- 11 hospitals participated.
- Structured implementation process for improvements on the QIs.
- Evaluation performed by RAND.
What We Learned
- Six hospitals participated in evaluation.
- Designed to learn:
- Hospital implementation strategies.
- Experiences in Improvement effort.
- Usefulness and usability of the tools.
- Data collection:
- Pre/post interviews.
- Regular update calls during study period.
- Three post-interviews in site visits.
Overall, Positive Feedback
- The tools were judged by the hospitals to be usable and useful.
- Hospitals varied widely in how many, and which, tools they chose to apply.
- Toolkit was useful for achieving staff consensus on the extent of quality gaps and on evidence-based practices.
All the Hospitals Chose to Address PSIs
- PSI 4: Death among surgical inpatients
w/ serious complications.
- PSI 7: Central venous catheter-related
- PSI 12: Postoperative pulmonary
embolism or deep vein thrombosis.
- PSI 13: Postoperative sepsis.
- PSI 15: Accidental puncture/laceration.
- PSI 19: Obstetric trauma-vaginal delivery
The decision of all the participating hospitals to work with one or more of the PSIs rather than the IQIs reflects the emphasis that currently is being placed on the PSIs by external incentive programs, such as Medicare's value-based purchasing initiative. Some hospitals struggled with competing measurement priorities, reflecting the diversity of the sometimes competing measures hospitals are asked to address by external parties. For this reason, several used other quality indicators which overlapped with the AHRQ QIs in the field test.
Three Key Learnings
- Hospitals need to trust their data.
- Priority-setting is challenging.
- Keep the tools short and simple.
Need to Trust Your Data
The IQI or PSI rates have to be credible:
"If we're running reports over coding information, we have to be mindful of coding issues before engaging medical staff. Need to be sure that we're not wasting their time."
Priority-Setting is Challenging
Many hospitals commented on prioritization:
"It's a great benefit to look at data and explore it to see if it's an issue... I don't know if [hospitals] have the time to do that, unless it's driven by corporate leadership or pay structure.
[There are] so many other things that we're mandated to report and improve, it's hard to look for something else."
Keep Tools Simple
Users should be able to easily find the tools they need:
"People have so much going on that it's hard..."
"I think we have to come up with simpler versions..."
To Address These Issues
- Added a documentation and coding tool to improve PSI validity.
- Made prioritization matrix tools flexible so a hospital can tailor it with factors it considers in priority-setting.
- Simplified tools and instructions to increase usability.
Current as of March 2012
Development and Testing of the AHRQ QI Toolkit for Hospitals. Text version of slide presentation from the Webinar on AHRQ Quality Indicators Toolkit for Hospitals, held on February 15, 2012. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/qual/qitoolkit/webinar0215/toolkitdevt.htm