Ambulatory Safety & Quality Initiative: Enabling Quality Measurement

Select to access the PowerPoint® presentation (91 KB).

Slide 1

Ambulatory Safety & Quality Initiative: Enabling Quality Measurement

Rebecca Roper
Session 91
September 10, 2012

Slide 2

Quality

• Agency for Healthcare Research and Quality:
  • Long-term and system-wide improvement of health care quality:
    • Only federal agency with a focus on health services research.
    • With an expanding focus on implementation and system change.
  • Not a policy-making or regulatory agency.

Images: A telescope in the lower left corner of the slide is aimed to look up at the word QUALITY spelled out in stars in the upper right corner.

Slide 3

Ambulatory, Safety, and Quality (ASQ) Initiative

• Scope of ambulatory care, increasing, as volume and complexity of care are expanding.
• Institute of Medicine Report, Patient Safety: Achieving a New Standard for Care (IOM, 2004). Priority Areas.
• Cornerstone of ASQ is to explore and demonstrate how health information technology (health IT) can improve quality of care provided in ambulatory care setting and for transitions in care settings.

See: http://healthit.ahrq.gov/ASQ

Image: A photograph shows the corner of a stone wall.

Slide 4

ASQ Grant Initiatives

• Includes four health IT-focused request for applications (RFAs):
  1. Enabling Quality Measurement through Health IT (EQM), HS-07-002.
  2. Improving Quality through Clinician Use of Health IT (IQHIT), HS-07-006.
  3. Enabling Patient-Centered Care through Health IT (PCC), HS-07-007.
  4. Improving Management of Individuals with Complex healthcare Needs through Health IT (MCP), HS-08-002.

Slide 5

RFA-Specific Summary Products

• RFA-specific Report:
  • Links to final grant reports.
  • Cross link to stories and Webinar.
• RFA-specific Exemplary stories:
  • Image: Text icon (Separate story available). Written exemplary stories.
  • Image: Camera icon (Video available). Video exemplary stories.
  • Image: Computer icon (Webinar available). RFA-Specific National Webinar.

Image: The cover of a report, AHRQ Health Information Technology Ambulatory Safety and Quality, is shown.

Slide 6

Goals of RFA-Specific Report

1. Summarizes the extent to which these projects addressed the research foci of RFA.
2. Identifies practical insight.
3. Presents illustrative initial findings to:
   • Inform research discussion.
   • Provide guidance to other entities implementing health IT systems for quality measurement and improvement.

Slide 7

Health IT Ambulatory, Safety & Quality: Enabling Quality Measurement (EQM)

• Key findings and lessons from the 17 grants of the EQM grant initiative:
  • Helps researchers understand the realities and complexities in quality measurement through health IT.

http://healthit.ahrq.gov/ASQEQMRPT2012.pdf.

Image: The cover of the report, AHRQ Health Information Technology Ambulatory Safety and Quality, is shown again.

Slide 8

EQM Investigators and Projects

Slide 9

EQM Investigators and Projects

Slide 10

Enabling Quality Measurement (EQM) Initiative

• Strategies for the development, deployment and export of quality measures from electronic health record systems:
  1. Development of retooled quality measures via health IT.
  2. Development of de novo quality measures via health IT.
• Issues addressed include:
  • Measure development across episodes of care.
  • Clinical data needs for quality measurement export and reporting.
  • Reporting of quality data for improvement.

http://grants.nih.gov/grants/guide/rfa-files/RFA-HS-07-002.html

Slide 11

EQM Foci and Associated Projects

• Total of 17 Projects with a variety of foci:

^* Some projects had multiple focus areas

• Two EQM Foci were not explicitly addressed:

Slide 12

Counts of Type of Health IT for EQM Grantees

Image: A bar graph shows the following data:

• Clinical Information Systems: 5.
• Natural Language Processing: 3.
• Automated Surveillance System: 2.
• Electronic Health Records: 15.
• Data Warehouse/Data Repository: 1.
• Clinical/Medication Reminders: 2.
• Results Reporting: 1.
• Disease Registry: 1.
• Interface: 1.
• Quality of Care Decision Support: 3.
• Interactive Voice Response/Telephony: 1.
• HIE/Regional Health Information: 1.
• Standards: 1.
• Clinical Decision Support: 1.

Slide 13

Counts of EQM Grantees by IOM Priority Area

Image: A bar graph shows the following data:

• Asthma: 3.
• Care Coordination: 3.
• Children With Special Needs: 1.
• Diabetes: 7.
• Cancer Screening: 3.
• Hypertension: 2.
• Immunization: 2.
• Heart Disease: 4.
• Major Depression: 1.
• Obesity: 1.
• Tobacco Treatment: 1.

Slide 14

Counts of EQM Grantees by Type of Ambulatory Care Setting

Image: A bar graph shows the following data:

• FQHC or CHC: 9.
• Primary Care: 8.
• Outpatient Clinic: 6.
• Specialty Practice: 4.
• Other: 5.

Slide 15

Developing New Measures

• Projects focusing on developing new measures:
  • Berner.
  • Kausha [Image: Text icon and Camera icon (Separate story and video available)].
  • Thomas.
  • Turchin [Image: Computer icon (Webinar available)].
  • Vogt and Williams.
• For a summary of findings from all projects that addressed "Developing New Measures", see "Findings and Lessons from the Enabling Quality Measurement Through Health IT Grant Initiative."

http://healthit.ahrq.gov/portal/server.pt/document/958394/6_1_3_d_final_508-compliant_eqm_report_6_21_12-to_client_pdf?qid=17450977&rank=1.

Slide 16

Developing New Measures, Selected Findings

• Berner: new measures were calculated through patient feedback collected via telephone and interactive voice response:
  • The topics of measures included patient-reported problem resolution, medication adherence, and followup activity.
• Kaushal: expert panel reviewed measures related to the quality of ambulatory care:
  • 18 existing measures were prioritized to be generated by EHRs, and 14 new measures were identified in underrepresented measurement areas.
• Vogt and Williams: developed EHR-based indices for the quality of cardiovascular disease management services in primary care.

Slide 17

Accuracy of Measurement

• Projects focusing on accuracy of measurement:
  • Bailey and Kilbridge.
  • Hazlehurst [Image: Computer icon (Webinar available)].
  • Kaushal [Image: Text icon and Camera icon (Separate story and video available)].
  • Kmetik.
  • Lehmann.
  • McColm [Image: Text icon (Separate story available)].
  • Thomas.
  • Turchin [Image: Computer icon (Webinar available)].
  • Weiner.
  • Wu and Mostashari.
• For a summary of findings from all projects that addressed "Accuracy of Measurement", see "Findings and Lessons from the Enabling Quality Measurement Through Health IT Grant Initiative."

http://healthit.ahrq.gov/portal/server.pt/document/958394/6_1_3_d_final_508-compliant_eqm_report_6_21_12-to_client_pdf?qid=17450977&rank=1.

Slide 18

Accuracy of Measurement, Selected Findings

• Kmetik tested the accuracy of patient, medical, and system-related reasons for excluding patients from measure denominators:
  • Frequency of exclusions was relatively low and the health IT systems identified them accurately compared to manual chart review.
• Bailey and Kilbridge used NLP to search clinical, demographic, encounter, laboratory, and pharmacy data to identify ADEs in children with cystic fibrosis, sickle cell disease, and cancer:
  • The system did not perform as well as chart review.
  • The system, however, identified 4 to 20 times more ADEs than the typical voluntary reporting system.

Slide 19

Accuracy of Measurement, Selected Findings

• Hazlehurst tested an NLP approach to the measurement of 18 measures related to the quality of outpatient asthma care:
  • Most health IT-enabled measures gave results comparable to manual chart review.
  • Sensitivity rates above 60 percent for 16 of the 18 measures.
• Kaushal tested the reliability of electronic generation of 11 established measures at a local FQHC:
  • Sensitivity of 88 percent and specificity of 89 percent compared to manual chart review.
  • Reliability varied considerably across measures, with measures relying on data from both structured fields and unstructured notes tending to be less reliable.

Slide 20

Accuracy of Measurement, Selected Findings

• Lehmann implemented flags in the EHR to identify patients in need of medication monitoring according to measures developed by the National Committee for Quality Assurance:
  • This was significantly more accurate than manual chart review with higher PPV, sensitivity, and specificity.
• McColm compared manual coder performance with electronic extraction and coding of data from the EHR:
  • Electronic extraction and coding was highly accurate for case identification for blood pressure, hemoglobin A1c, and low-density lipoprotein data elements.

Slide 21

Capturing and Integrating Data

• Projects focusing on accuracy of measurement:
  • Bailey and Kilbridge.
  • Davidson.
  • Hazlehurst [Image: Computer icon (Webinar available)].
  • Lazarus.
  • Lehmann.
  • Logan [Image: Text icon and Camera icon (Separate story and video available)].
  • McColm [Image: Text icon (Separate story available)].
  • Schneider.
  • Turchin [Image: Computer icon (Webinar available)].
  • Vogt and Williams.
  • Weiner.
  • Wu and Mostashari.
• For a summary of findings from all projects that addressed "Capturing and Integrating Data", see "Findings and Lessons from the Enabling Quality Measurement Through Health IT Grant Initiative."

http://healthit.ahrq.gov/portal/server.pt/document/958394/6_1_3_d_final_508-compliant_eqm_report_6_21_12-to_client_pdf?qid=17450977&rank=1.

Slide 22

Capturing and Integrating Data, Selected Findings

• Logan implemented and evaluated a set of 15 measures of the quality of colonoscopy procedures:
  • This confirmed the feasibility of the generating measures using data captured at the point of care through custom data entry screens in an EMR.
• Turchin researchers developed NLP software to extract information on insulin dosing to identify patients for whom medication therapy was intensified.
• Davidson worked with nine local CHCs to collaboratively define requirements for a shared quality information system
  • Team developed business requirements for templates for capturing data related to diabetes and smoking cessation.

Slide 23

Capturing and Integrating Data, Selected Findings

• Vogt and Williams developed EHR-based quality indices for 11 cardiovascular primary care services:
  • Even though the indices were implemented in Kaiser Permanente sites that had substantial experience using the same EHR, investigators had to create an extensive process for extracting, cleaning, and coding the data.
• Weiner integrated EHR data from two institutions that cared for some of the same patients to test a method of risk adjusting physician-level diabetes quality of care rankings:
  • Using a linkage between database tables of demographics and patient identifiers from the two systems, the researchers were able to find patients with visit activity in both locations and conduct a descriptive analysis of their patterns of care.

Slide 24

Capturing and Integrating Data, Selected Findings

• Wu and Mostashari created health IT tools to assist primary care physicians in small practices in measuring the quality of care.
  • The software "hard-coded" 34 existing measures into the EHR, making them easily accessible to the provider.
• Schneider attempted to develop a measurement approach that integrated data from primary care practices participating in three community-wide, multi-payer HIE efforts.
  • Several barriers prevented a successful evaluation of the adequacy of these data sources for performance measurement

Slide 25

Projects focusing on feedback to clinicians

• Projects focusing on feedback to clinicians:
  • Berner.
  • Davidson.
  • Lehmann.
  • Logan [Image: Text icon and Camera icon (Separate story and video available)].
  • Selby.
  • Wu and Mostashari.
• For a summary of findings from all projects that addressed "Feedback to Clinicians", see "Findings and Lessons from the Enabling Quality Measurement Through Health IT Grant Initiative".

http://healthit.ahrq.gov/portal/server.pt/document/958394/6_1_3_d_final_508-compliant_eqm_report_6_21_12-to_client_pdf?qid=17450977&rank=1.

Slide 26

Feedback to Clinicians, Selected Findings

• Selby offered feedback to staff responsible for population management on the need to intensify medication treatment for those patients with out-of-range values:
  • Modest impact on treatment intensification rates for patients with elevated systolic blood pressure and low-density lipoprotein levels.
  • No observed impact on proportions of patients with levels in the target range.
• Logan posted monthly, physician-specific performance reports:
  • Based on feedback from participating physicians, the researchers stream­lined the reports and integrated them into the EHR (analysis of the impact of the reports is in progress).

Slide 27

Feedback to Clinicians, Selected Findings

• Wu and Mostashari provided patient-specific clinician reminders and decision support at the point of care and real-time reports on a provider's overall performance on the quality measures:
  • Provider performance on nearly all measures exhibited statistically significant improvements, ranging from 5 to 20 percentage points per measure.
• Davidson found that use of built-in templates to support clinicians providing and documenting care led to improvements in measures related to smoking cessation at some sites.
• Lehmann gave primary care providers an EHR-generated paper bulletin listing patients due for therapeutic monitoring tests related to one or more medications.
• Patients appearing on the bulletins were somewhat more likely to receive monitoring within 2 months.

Slide 28

Efficiency of Measurement

• Projects focusing on efficiency of measurement:
  • Bailey and Kilbridge.
  • Lazarus.
  • Thomas.
• For a summary of findings from all projects that addressed "Feedback to Clinicians", see "Findings and Lessons from the Enabling Quality Measurement Through Health IT Grant Initiative".

http://healthit.ahrq.gov/portal/server.pt/document/958394/6_1_3_d_final_508-compliant_eqm_report_6_21_12-to_client_pdf?qid=17450977&rank=1.

Slide 29

Efficiency of Measurement, Selected Findings

• Bailey and Kilbridge used NLP to search clinical, demographic, encounter, laboratory, and pharmacy data to identify pediatric ADEs:
  • Time-savings enabled the researchers to identify a greater number of serious ADEs.
• Lazarus created a method for prospectively integrating multiple types of EHR data with the goal of identifying potential adverse events related to vaccinations:
  • Data showed that 2.6 percent of vaccinations resulted in possible reactions.
• Thomas applied electronic triggers that might be associated with a diagnostic error:
  • This methodology was more efficient than conducting random record reviews and identified errors that were more consequential than many routine errors.

Slide 30

Using Electronic Health Records To Measure and Improve Quality for Colonoscopy Procedures

• Judith Logan, OHSU.
• Effectiveness of colonoscopy screening procedures, typically done in an ambulatory setting, depends on providing high quality examinations that result in accurate diagnoses and few complications.
• Story and video highlight how the investigators were able to use data from electronic medical records for quality measurement for colonoscopy procedures. Investigators discuss the lessons learned as they formulated, implemented, presented measures the clinicians.

Image: Text icon (Separate story available). http://healthit.ahrq.gov/EQMStoryLogan2012.pdf.
Image: Camera icon (Video available). http://healthit.ahrq.gov/EQMLoganVideo

Image: A computer monitor screen shows the text "Interactive Web-based Quality Report Card," with an outline of a lower intestine.

Slide 31

LOGAN (continued)

• Logan and her team created and evaluated an electronic quality measurement and feedback program—known as excellence report-- for colonoscopies.
• Findings:
  • Physicians receptive to feedback as a way to improve effectiveness and safety of their procedures.
  • Point-of-care data entry was not seen as overly burdensome.
  • Physicians wished to have feedback shared broadly.
• Continued Use:
  • Excellence report is now delivered in the EHR.

Image: Text icon (Separate story available). http://healthit.ahrq.gov/EQMStoryLogan2012.pdf.
Image: Camera icon (Video available). http://healthit.ahrq.gov/EQMLoganVideo

Image: A computer monitor screen shows the text "Interactive Web-based Quality Report Card," with an outline of a lower intestine.

Slide 32

Developing and Testing Quality Measures for Interoperable Electronic Health Records

• Rainu Kaushal, Weil Cornell.
• Demonstrate effective use of both electronic health records (EHRs) and health information exchange (HIE) to electronically measure quality of care delivered in ambulatory settings.
• Story and video highlight the methods used to identify quality measures that could be supported and impacted by EHRs and HIE; results of reliability testing of the quality measures; and subsequent the impact of the investigators' work on national health IT policy and "Meaningful Use".

Image: Text icon (Separate story available). http://healthit.ahrq.gov/EQMStoryKaushal2012.pdf.
Image: Camera icon (Video available). http://healthit.ahrq.gov/EQMKaushalVideo

Image: A computer monitor screen shows an image of a man's hands at a computer keyboard.

Slide 33

Kaushal (continued)

• Dr. Rainu Kaushal and team pursued the identification, prioritization, development, and reliability testing of quality measures using an interoperable EHR in a primary care setting.
• Process:
  • With the assistance of an expert panel, they applied a four-part conceptual framework to identify 18 prioritized measures of chronic disease management and preventive services.
• Findings:
  • Electronic reporting correctly identified 88 percent of the patients who received recommended care and 89 percent of the patients who did not receive recommended care compared to manual chart review.
• Sustainability:
  • Fourteen new HIE-enabled measures were developed in five important categories: test ordering, medication management, referrals, followup after discharge, and revisits.

Image: Text icon (Separate story available). http://healthit.ahrq.gov/EQMStoryKaushal2012.pdf.
Image: Camera icon (Video available). http://healthit.ahrq.gov/EQMKaushalVideo

Image: A computer monitor screen shows an image of a man's hands at a computer keyboard.

Slide 34

Standardization and Automatic Extraction of Quality Measures in an Ambulatory EHR, McColm Exemplary Story—Written

• The lack of standards for clinical documentation in an EHR is a major barrier to automated quality measurement
  • Denni McColm's team established standards for clinical documentation and demonstrated the efficiency and accuracy of using data extraction and reporting to perform quality measurement in the ambulatory care setting.
• Story highlights the methods used to establish standards and findings from the implementation of an automated system for data extraction of quality measures in the ambulatory setting, including valid, reliable reports that provide actionable insight for the measurement and analysis of care.

Image: Text icon (Separate story available). http://healthit.ahrq.gov/EQMStoryMcColm2009.pdf.

Image: A computer monitor screen shows an image of a man's hands at a computer keyboard.

Slide 35

Use of Natural Language Processing to Improve Quality Measurement, a National Web Conference

• Purpose:
  • To address the existing gap between a health care and a public health practitioner's competencies as it relates to the health IT environment.
  • This specific webinar illustrated how new methods of analyzing free text data stored in electronic health records can impact quality measurement.
• Learning Objectives:
  • Discuss the principles of NLP design and implementation.
  • Describe how NLP is used to operationalize the assessment of quality measurement in asthma care.
  • Explain how NLP is used in monitoring intensification of treatment for patients with diabetes.

Image: Computer icon (Webinar available). http://healthit.ahrq.gov/nlp-eqmwebinar

Slide 36

Additional EQM Products and Links

ASQ Web site: http://healthit.ahrq.gov/ASQ
Image: Text icon (Separate story available). EQM Report: http://healthit.ahrq.gov/portal/server.pt/document/958394/6_1_3_d_final_508-compliant_eqm_report_6_21_12-to_client_pdf?qid=17450977&rank=1.
Image: Computer icon (Webinar available). NLP Webinar: http://healthit.ahrq.gov/nlp-eqmwebinar
Image: Camera icon (Video available). Kaushal Video: http://healthit.ahrq.gov/EQMKaushalVideo
Image: Text icon (Separate story available). Kaushal Story: http://healthit.ahrq.gov/EQMStoryKaushal2012.pdf.
Image: Camera icon (Video available). Logan Video: http://healthit.ahrq.gov/EQMLoganVideo
Image: Text icon (Separate story available). Logan Story: http://healthit.ahrq.gov/EQMStoryLogan2012.pdf.
Image: Text icon (Separate story available). McColm Story: http://healthit.ahrq.gov/EQMStoryMcColm2009.pdf.