Development and Testing of the AHRQ QI Toolkit for Hospitals

Slide Presentation from the Webinar on AHRQ Quality Indicators Toolkit

Select to access the  Microsoft PowerPoint version - 720.32 KB .

Slide 1

Development and Testing of the AHRQ QI Toolkit for Hospitals

Donna O. Farley, PhD
Peter Hussey, PhD
RAND Corporation

Slide 2

What Is the Toolkit?

• Set of tools that hospitals can use to help improve performance in quality and patient safety.
• The AHRQ Quality Indicators (QIs):
  • Inpatient Quality Indicators (IQIs).
  • Patient Safety Indicators (PSIs).
• Targeted to wide range of hospitals:
  • Independent or system-affiliated.
  • Varying quality improvement experience.

Slide 3

Toolkit Development

• Developed through a Task Order in the AHRQ ACTION program.
• RAND partnered with UHC to develop and test the toolkit.
• AHRQ will continue toolkit support.

Slide 4

How Hospitals Can Use the Toolkit

• Applicable for hospitals with differing knowledge, skills, and needs.
• Serves as a "resource inventory" from which hospitals can select tools.
• Different audiences for each tool (e.g., quality officer, finance officer, programmer).

Slide 5

What Are the Quality Indicators?

• Inpatient Quality Indicators—28 indicators of quality in four sets:
  • Volume, counts (6).
  • Mortality for conditions, rates (7).
  • Mortality for procedures, rates (8).
  • Utilization, rates (7).
• Patient Safety Indicators—17 indicators and a composite indicator:
  • Screen for adverse events for inpatients.
  • Expressed as rates.

Slide 6

The Development Process

Image: Flowchart shows the following steps:

• Develop Alpha Toolkit:
  • Identify tools to include.
  • Develop draft tools.
• Field Test:
  • Alpha Toolkit.
• Perform Evaluation:
  • Improvement experiences.
  • Usability of toolkit.
  • Effects on QI values.
• Revise and finalize Toolkit for Dissemination.

Slide 7

Tool Development Steps

• Established principles to guide toolkit development.
• Reviewed literature to guide design.
• Developed outline of toolkit based on steps of a quality improvement process.
• Identified and developed specific tools for each step.

Slide 8

Technical Advisory Panel

• Six-member panel.
• Brought various skills and perspectives:
  • Hospital experience.
  • Quality improvement.
  • Relevant research skills.
• Provided guidance throughout toolkit development:
  • Toolkit design principles.
  • Content of the tools.

Slide 9

Principles Guiding Toolkit Development

• Parsimony in tool choice and design.
• Target the most important factors for implementation.
• Provide tools that offer most value for a range of hospitals.
• Readily accessible contents.
• Enable hospitals to assess effectiveness of their actions.

Slide 10

Structure of the Toolkit

Introduction and Roadmap

1. Readiness to Change.
2. Applying QIs to the Hospital Data.
3. Identifying Priorities for Quality Improvement.
4. Implementation Methods.
5. Monitoring Progress and Sustainability of Improvements.
6. Return-on-Investment Analysis.
7. Existing Quality Improvement Resources.

Slide 11

The Roadmap

• A navigational guide through the toolkit.
• For each tool, it summarizes:
  • Action step being taken.
  • Brief description of the tool.
  • Key audience(s) to use the tool.
  • Position with lead role responsibility.

Slide 12

Field Test Design

• Quality improvement collaborative.
• Conducted by UHC.
• 11 hospitals participated.
• Structured implementation process for improvements on the QIs.
• Evaluation performed by RAND.

Slide 13

What We Learned

Slide 14

Evaluation Design

• Six hospitals participated in evaluation.
• Designed to learn:
  • Hospital implementation strategies.
  • Experiences in Improvement effort.
  • Usefulness and usability of the tools.
• Data collection:
  • Pre/post interviews.
  • Regular update calls during study period.
  • Three post-interviews in site visits.

Slide 15

Overall, Positive Feedback

• The tools were judged by the hospitals to be usable and useful.
• Hospitals varied widely in how many, and which, tools they chose to apply.
• Toolkit was useful for achieving staff consensus on the extent of quality gaps and on evidence-based practices.

Slide 16

All the Hospitals Chose to Address PSIs

• PSI 4: Death among surgical inpatients w/ serious complications.
• PSI 7: Central venous catheter-related bloodstream infection.
• PSI 12: Postoperative pulmonary embolism or deep vein thrombosis.
• PSI 13: Postoperative sepsis.
• PSI 15: Accidental puncture/laceration.
• PSI 19: Obstetric trauma-vaginal delivery w/o instrument.

The decision of all the participating hospitals to work with one or more of the PSIs rather than the IQIs reflects the emphasis that currently is being placed on the PSIs by external incentive programs, such as Medicare's value-based purchasing initiative. Some hospitals struggled with competing measurement priorities, reflecting the diversity of the sometimes competing measures hospitals are asked to address by external parties. For this reason, several used other quality indicators which overlapped with the AHRQ QIs in the field test.

Slide 17

Three Key Learnings

• Hospitals need to trust their data.
• Priority-setting is challenging.
• Keep the tools short and simple.

Slide 18

Need to Trust Your Data

The IQI or PSI rates have to be credible:

"If we're running reports over coding information, we have to be mindful of coding issues before engaging medical staff. Need to be sure that we're not wasting their time."

Slide 19

Priority-Setting is Challenging

Many hospitals commented on prioritization:

"It's a great benefit to look at data and explore it to see if it's an issue... I don't know if [hospitals] have the time to do that, unless it's driven by corporate leadership or pay structure.

[There are] so many other things that we're mandated to report and improve, it's hard to look for something else."

Slide 20

Keep Tools Simple

Users should be able to easily find the tools they need:

"People have so much going on that it's hard..."

"I think we have to come up with simpler versions..."

Slide 21

Revised Toolkit To Address These Issues

• Added a documentation and coding tool to improve PSI validity.
• Made prioritization matrix tools flexible so a hospital can tailor it with factors it considers in priority-setting.
• Simplified tools and instructions to increase usability.

Slide 22

Questions?