Quality and Safety Review System (QSRS)

Retrospectively Reviewing Inpatient Health Records To Identify Adverse Events

Medical errors are an ongoing challenge to the healthcare system in the United States. The extent of medical errors in U.S. hospitals was revealed in 2000 when the Institute of Medicine (now the National Academy of Medicine) published To Err Is Human, estimating that up to 98,000 deaths occurred each year due to medical errors. Tracking adverse events can be difficult, leading to the development of the Medicare Patient Safety Monitoring System (MPSMS) in 2001, which was replaced by the Quality and Safety Review System (QSRS) in 2020.

National Adverse Event Rates Available From QSRS

This report reflects the evolution of combined efforts of AHRQ and the Centers for Medicare & Medicaid Services (CMS) since 2001 to monitor inpatient hospital AEs through retrospective chart review. Analyzing data from the QSRS, this report presents weighted national estimates of AEs experienced by Medicare patients hospitalized from 2021–2023. 

• Adverse Events Among Medicare Hospitalizations in 2021–2023 (PDF, 806 KB)  
• QSRS 2020–2024 Sampling Design and Weighting Methodology.

QSRS Features

The QSRS was designed to identify the occurrence of specified adverse events to gain a better understanding of patient safety in the hospital setting. QSRS data are collected through retrospective manual abstraction of inpatient records. Human abstractors answer questions to identify whether an adverse event occurred during a particular hospital stay.

QSRS uses standardized definitions and algorithms consistent with the AHRQ Common Formats for Surveillance. These standardized specifications ensure adverse events are reliably identified across all hospitals.

QSRS covers a wide range of inpatient adverse events, grouped into 11 modules:

1. Birth-Maternal.
2. Birth-Neonatal.
3. Blood or Blood Product.
4. Device.
5. Fall.
6. Hospital-Acquired Infection.
7. Medication.
8. Pressure Injury.
9. Surgery/Anesthesia.
10. Venous Thromboembolism.
11. Other.

Development of Adverse Event Monitoring Systems

The Centers for Medicare & Medicaid Services (CMS) created MPSMS in coordination with the:

• Agency for Healthcare Research and Quality (AHRQ).
• Centers for Disease Control and Prevention.
• Food and Drug Administration.
• Department of Veterans Affairs).

Originally designed to monitor and track in-hospital adverse events among Medicare patients through retrospective chart review, MPSMS established a baseline for assessing the impact of national patient safety initiatives.

In 2009, primary coordination for MPSMS was transferred from CMS to AHRQ. For almost 20 years, MPSMS was used to determine national rates for specific adverse events. MPSMS data were last collected for calendar year 2019. Because of the COVID-19 pandemic, CMS waived inpatient data collection from January through August 2020. That same year, MPSMS reached the end of its service life and was replaced by QSRS, an improved and enhanced system.

AHRQ used MPSMS data to produce the AHRQ National Scorecard on Hospital-Acquired Conditions. Using MPSMS data, AHRQ determined that national efforts to reduce hospital-acquired conditions such as adverse drug events and injuries from falls helped prevent 20,700 deaths and saved $7.7 billion between 2014 and 2017.

In addition, MPSMS data from 2010 to 2019 showed decreases from 31 to 39 percent in the number of in-hospital adverse events for major surgery patients and those with a principal diagnosis of acute myocardial infarction, congestive heart failure, and pneumonia.

Future Vision for QSRS

QSRS currently relies on human abstractors to manually review inpatient records. Automation may improve the efficiency and quality of the abstraction process. In a 2018 report (PDF, 3.2 KB), AHRQ assessed the feasibility of automating the abstraction of electronic health records (EHRs). A followup publication in 2021 noted that although EHRs contain a lot of information, abstracting information from them is still quite challenging, particularly for complex questions, such as those concerning patient AEs.

In 2023, building on the foundation of its previous activities, AHRQ began work with the General Services Administration (GSA)’s Centers of Excellence to prepare a QSRS modernization roadmap and develop an Artificial Intelligence/Machine Learning (AI/ML) solution to automate the abstraction of medical records. The first step to modernization was development of a prototype (PDF, 4.3 MB) that integrated Optical Character Recognition (OCR), Natural Language Processing (NLP), and Large Language Models (LLM) to convert as many as thousands of pages of unstructured patient data into analyzable content. The prototype was developed using Amazon Web Services-native technologies and supported unattended batch processing that is fully scalable. Strategic benefits demonstrated by the prototype included a significant reduction in abstraction time and costs, when compared with manual abstraction.