Making Adjustments to Health Care Quality Scores

One of the most thorny topics in quality measurement is the adjustment of scores across different plans or providers to account for differences in the characteristics of their patients or themselves. This page reviews key issues related to adjustments to scores.

Adjusting for Risk or Severity

The most common kind of adjustment addresses variations in the characteristics of the patient population, specifically whether—as a group—patients in one facility are at greater risk for poor outcomes. Risk adjustment is a complex statistical enterprise. The extent to which adjustments can be made depends on whether the sponsor has access to other information about the patients or respondents in question, and this in turn typically depends on the source of the data. [1][2]

Why adjust for risks or severity?

Sometimes a provider’s patients are more severely ill; sometimes they have other serious co-morbidities. When presenting mortality and other health outcome measures, such adjustments are considered critical for creating a “level playing field” so that a provider is not penalized for taking care of sicker patients.

What do you adjust for?

Most common risk adjustments—for age, prior medical history, or comorbidities—are considered a good idea. It is not considered appropriate, in most circumstances, to adjust for other sociodemographic characteristics such as race, ethnicity, income, education, and/or insurance status. Such adjustments would essentially bury information that could reveal what some would term unacceptable disparities in care.

One alternative is to present information stratified by these categories, if there are enough respondents to generate a reliable score. This has been done when reporting at a national level to policymakers, although some sponsors of reports to the public are also interested in presenting information this way.


EXAMPLE: Presenting Information Stratified by Socio-Demographic Characteristics
 

Title: Providing Equitable Care
Sponsor: Massachusetts General Hospital: Quality & Safety
URL: http://qualityandsafety.massgeneral.org/measures/index.aspx
A report by Massachusetts General Hospital shows how they perform on several clinical measures for white and non-white patients.

Source: Massachusetts General Hospital: Quality & Safety, 2009. Available at
http://qualityandsafety.massgeneral.org/measures/equitable.aspx?id=4.


EXAMPLE: Presenting Information Stratified by Socio-Demographic Characteristics
 

Title: National Healthcare Disparities Report
Sponsor: Agency for Healthcare Research and Quality
URL: http://www.ahrq.gov/research/findings/nhqrdr/index.html
The Agency for Healthcare Research and Quality’s National Healthcare Disparities Report shows how quality varies across racial/ethnic groups.
Source: Agency for Healthcare Research and Quality. Available at http://www.ahrq.gov/research/findings/nhqrdr/index.html.

Adjusting for Provider Size

Sometimes the providers you are comparing differ widely in size. This can lead to a situation in which the number of observations for a given measure can also vary, with a large “n” for a larger organization and a smaller “n” for a smaller organization. In this case, normal comparisons of scores can be unreliable; there is a danger that the performance of smaller organizations will look artificially lower than that of larger organizations.

Many statisticians believe that under these circumstances, it is appropriate to adjust scores using a method called hierarchical modeling or smoothing. Since the statistics involved are quite complex, it is critical to find an expert who can both conduct the analysis and explain what was done in reasonably nontechnical terms to the individuals and organizations with whom you are working.[3]

 

[1] Kuhlthau K, Ferris TG, Iezzoni, LT. Risk Adjustment for pediatric quality indicators. Pediatrics 2004 Jan;113(1 Pt 2):210-6.
[2] O’Malley AJ, Zaslavsky AM, Elliott MN, et al. Case-mix adjustment of the CAHPS Hospital Survey. Health Serv Res 2005 Dec;40(6 Pt 2):2162-81.
[3] Arling, G, Lewis T, Kane RL, et al. Improving quality assessment through multilevel modeling: the case of nursing home compare. Health Serv Res. 2007 Jun;42(3 Pt 1):1177-99.

 


Also in "Generating Scores that Show Differences in Performance"

Page last reviewed March 2016
Page originally created February 2015
Internet Citation: Making Adjustments to Health Care Quality Scores. Content last reviewed March 2016. Agency for Healthcare Research and Quality, Rockville, MD. https://www.ahrq.gov/talkingquality/translate/scores/adjustment-scoring.html
Back To Top