Development and Use of Ambulatory Adverse Event Trigger Tools (Text Version)

On September 29, 2010, Amy Rosen made this presentation at the 2010 Annual Conference. Select to access the PowerPoint® presentation (190 KB). Free PowerPoint® Viewer (Plugin Software Help).

Slide 1

Development and Use of Ambulatory Adverse Event Trigger Tools

Amy K. Rosen, PhD

AHRQ Conference
Sept. 29, 2010

Boston University School of Public Health, Boston, MA,
VA Center for Organization, Leadership and Management Research (COLMR), Boston MA
akrosen@bu.edu

Slide 2

Acknowledgements

Sponsored by AHRQ Contract No. HHSA290200600012, Task Order Officer: Amy Helwig, MD.

• PI Amy Rosen, PhD
• Co-PI Jonathan Nebeker, MD, MS
• Co-Investigators:
  • Stephan Gaehde, MD
  • Haytham Kaafarani, MD, MPH
  • Brenna Long, MA
  • Hillary Mull, MPP
  • Brian Nordberg, BS
  • Steve Pickard, MS
  • Peter Rivard, PhD
  • Lucy Savitz, PhD, MBA
  • Chris Shanahan, MD, MPH
  • Stephanie Shimada, PhD

Slide 3

Project Goal and Settings

• Goal: Develop adverse event (AE) triggers for the outpatient setting:
  • Outpatient surgery AEs.
  • Outpatient diagnostic testing and loss to follow-up AEs.
  • Outpatient adverse drug events (ADEs)
• Three sites for patient data:
  • Boston Medical Center (BMC)
  • Intermountain Healthcare
  • Western region of the Veterans Health Administration (VA)

Slide 4

Background

• Trigger tool: A screen applied to healthcare data that triggers review of a patient record for a potential adverse event.
• Trigger: An algorithm that uses electronic medical record data to identify patterns consistent with a possible adverse event.
  Example: combination of a lab value threshold and an active prescription.
• AE specific trigger: A trigger intended to identify a specific adverse event.

Slide 5

Methods

Image: There is a flowchart next to the above table. The steps of the flowchart are: List of triggers → Program trigger algorithms → Trigger-flagged cases → Chart review. In addition, the Project database has arrows to both Program trigger algorithms and Chart review.

Slide 6

Methods: Outpatient Surgery Triggers

• Sample:
  • N=17,492 ambulatory surgeries from three institutions in CY05.
  • Sampling for chart review: 17 trigger-flagged cases from each site to the extent the data were available.
• Definition of AE:
  • NSQIP AEs: Surgeries with an event that meets the National Surgical Quality Improvement Program criteria for an AE (e.g., postoperative urinary tract infection).
  • Other AE: Surgery with an AE determined by nurse clinical judgment (e.g., postoperative iatrogenic injury).
• Analysis:
  • PPV = number of patients with AE/number of positive triggers

Slide 7

Methods: Loss to Follow-up Trigger

• Definition of AE:
  • Failure to follow-up on abnormal fecal occult blood test (FOBT) results, which could potentially lead to a failure to detect colorectal cancer or pre-cancerous growths in a timely manner.
    • In the event a patient has a positive FOBT, the standard of care is a colonoscopy within 2 months.
    • We designed the trigger to exclude bleeding that might potentially be due to a recently diagnosed ulcer or GI surgery.
• Analysis:
  • PPV was calculated for cases without a colonoscopy and cases missing an appropriate colonoscopy.

Slide 8

Methods: Focus Groups

• We conducted 90-minute focus group and multiple 30-45 minute interviews at each of the sites between October and December 2009.
• Participants were frontline staff, middle managers and executives including:
  • Physicians (medicine and surgery)
  • Nurses
  • Informatics and information systems experts
  • Pharmacists
  • Quality and patient safety experts

Slide 9

Results: Outpatient Surgery Triggers

Slide 10

Results: Outpatient Surgery Triggers

Slide 11

Results: Loss to Follow-up Trigger

• Sample:
  • N=995 cases with positive FOBT tests in CY05 from one institution

Slide 12

Results: Focus Group/Interviews

Triggers ranked from highest to lowest likelihood of adoption.

Image: A bar chart displays the following information:

Slide 13

Results: Focus Group/Interviews

Triggers ranked from highest to lowest perceived ease of implementation.

Image: A bar chart displays the following information:

Slide 14

Summary

• The set of surgical triggers (excluding 'Hematocrit') required review of 454 cases and detected at least one AE in 204 ambulatory surgeries (PPV for any AE=45%).
  • Many surgeries had more than one AE detected.
• Triggers best suited for adoption:
  • Admission: flag rate=22%; PPV for detection of any AE=41%; popular in focus groups.
  • PE/DVT: flag rate=1%; PPV for detection of any AE=65%; very popular in focus groups.
  • FOBT: flag rate=8%; PPV for detection of any cases without colonoscopy after 6 months=66%; very popular in focus groups

Slide 15

Implications

• Triggers may have potential to screen for outpatient AEs using a focused sample of cases.
• Triggers could complement existing screening programs used to detect AEs.
• Potential for real-time AE detection, particularly with electronic medical records.

Slide 16

Dissemination to Date

• Rosen AK & Nebeker JR, Use of Trigger Tools to Identify Risks and Hazards to Patient Safety, Sept. 2010. Speakers, AHRQ 2010 Annual Conference, Bethesda, MD.
• Long BL, Pickard S, Mull HJ, Hoffman JM, Rosen AK & Nebeker JN, Reliability of AHRQ Harm Scale Used with Explicit Criteria in Retrospective ADE Classification, Sept. 2010. Poster, AHRQ 2010 Annual Conference, Bethesda, MD.
• Shimada S, Mull HJ, Kaafarani H, Rosen AK, Nebeker JR, Nordberg B, Pickard S & Singh H. Development and Assessment of a Trigger Tool to Detect Patients with Positive Fecal Occult Blood Test (FOBT) Results Who Are Lost to Follow-Up. Accepted poster, AMIA 2010 National Symposium, Nov 2010, Washington, DC.
• Kaafarani H, Rosen AK, Nebeker JR, Shimada SL, Mull HJ, Rivard PE, Savitz L, Helwig A, Shin MH & Itani KMF. Development of Trigger Tools for Surveillance of Adverse Events in Ambulatory Surgery, 2010. Quality and Safety in Health Care. [E-pub ahead of print] http://www.ncbi.nlm.nih.gov/pubmed/20513790
• Mull H & Nebeker JR, Informatics Tools for the Development of Action-Oriented Adverse Drug Event Triggers, 2008. AMIA Annual Symposium Proceedings. Nov 6:505-9. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2655939/
• Mull HJ, Nordberg B, Nebeker JR, Using Electronic Medical Records Data for Health Services Research, Case Study: Development and Use of Ambulatory Adverse Event Trigger Tools. Speaker, AcademyHealth Annual Research Meeting, June 2010, Boston, MA.

Slide 17

Dissemination to Date (cont'd)

• Shimada S, Rivard P, Nebeker JR, Savitz L, Shanahan C, Gaehde S & Rosen AK. Priorities & Preferences of Potential Ambulatory Trigger Tool Users. Speaker, AcademyHealth Annual Research Meeting, June 2009, Chicago, IL.
• Kaafarani H, Rosen AK, Nebeker JR, Shimada SL, Rivard PE, Mull HJ, Long B, Shin MH, Savitz L & Itani KMF. Developing Trigger Tools for Surveillance of Adverse Events in Same-Day Surgery: A Literature-Based, End-User Inspired and Expert-Evaluated Methodology, Sept 2009. Poster, AHRQ 2009 Annual Meeting, Bethesda, MD.
• Mull HJ, Shimada S, Nebeker JR & Rosen AK, A Review of the Trigger Literature: Adverse Events Targeted and Gaps in Detection, June 2008. Proceedings of the Trigger and TIDS Expert Meeting, Agency for Healthcare Research and Quality, Bethesda, MD. http://www.ahrq.gov/qual/triggers/triggers1.htm
• Nebeker JR, Stoddard GJ & Rosen AK, Considering Sensitivity and Positive Predictive Value in Comparing the Performance of Triggers Systems for Iatrogenic Adverse Events, Proceedings of the Trigger and TIDS Expert Meeting, Agency for Healthcare Research and Quality, June 2008. Bethesda, MD. http://www.ahrq.gov/qual/triggers/triggers2.htm
• Shimada S, Rivard PE, Mull HJ, Nebeker JR & Rosen AK, Triggers and Targeted Injury Detection Systems: Aiming for the Right Target With the Appropriate Tool, June 2008. Proceedings of the Trigger and TIDS Expert Meeting, Agency for Healthcare Research and Quality, Bethesda, MD. http://www.ahrq.gov/qual/triggers/triggers3.htm