Research Activities, July 2013

Accuracy of electronic quality reporting varies across quality measures

Health Information Technology

Beginning in 2014, the Federal Electronic Health Record Incentive Program will require electronic reporting of quality from electronic health records (EHRs). The goal is to avoid the limitations of measuring quality from administrative claims (which lack clinical detail) or from manual review of paper-based clinical records (which is time-consuming and yields small sample sizes). EHRs, however, were designed for clinical care, not for automated reporting of quality measures. Thus, it is important to measure the accuracy of automated reporting and understand opportunities to improve quality measurement. 

After examining the electronic records of 1,154 patients of a federally qualified health center, Rainu Kaushal, M.D., and Lisa M. Kern, M.D., M.P.H. of Weill Cornell Medical College, and colleagues found wide measure-by-measure variation in the accuracy of 12 quality measures, when they compared automated electronic reporting to manual review of the EHR.

The researchers found significant differences between electronic reporting and manual review for 3 of 12 care quality measures. For example, compared to manual review, electronic reporting significantly underestimated rates of appropriate asthma medication (38 percent vs. 77 percent) and pneumococcal vaccination (27 percent vs. 48 percent) and overestimated rates of cholesterol control in patients with diabetes (57 percent vs. 37 percent). 

The researchers suggest several possible explanations for these findings of wide measure-by-measure variation. Electronic reporting could have underestimated rates of asthma medication and pneumococcal vaccination if care was recorded in free-text notes or scanned documents rather than in structured fields. Rates of cholesterol control could have been overestimated if the electronic report and manual reviewers considered a different test to be the most recent cholesterol value. The researchers suggest that national programs that link financial incentives to quality reporting should require electronic health record vendors to demonstrate the accuracy of their automated reports. This study was supported by AHRQ (HS17067). 

See "Accuracy of electronically reported "meaningful use" clinical quality measures," by Dr. Kern, Dr. Kaushal, Sameer Malhotra, M.D., and others in the January 2013 Annals of Internal Medicine 158, pp. 77-83. See also a video about this work, which can be viewed at


Page last reviewed July 2013
Page originally created July 2013
Internet Citation: Accuracy of electronic quality reporting varies across quality measures. Content last reviewed July 2013. Agency for Healthcare Research and Quality, Rockville, MD.