Chapter 1. Introduction and Methods (Continued)

Measure Set for the 2010 NHQR and NHDR

Retired Measures

Previous reports have demonstrated that some measures of health care quality have improved. Since the first NHQR and NHDR, significant improvements in a number of measures of quality of care have occurred, with U.S. health care providers achieving overall performance levels exceeding 95%. The success of these measures limits their utility for tracking improvement over time. Because these measures cannot improve further to a significant degree, including them in the measure set creates a ceiling effect that may distort quantification of rate of change over time. Data on retired measures will continue to be collected and these measures will be added back to the reports if their performance falls below 95%. For the 2010 report, no measures have been retired or added back.

The measures that were retired in 2009 and therefore not presented in the 2010 report are:

• Adults with diabetes who had their blood cholesterol checked.
• Hospital patients with heart attack who received aspirin within 24 hours of admission.
• Hospital patients with heart attack who were prescribed aspirin at discharge.
• Hospital patients with heart attack who were prescribed a beta blocker at discharge.
• Smokers with heart attack who received smoking cessation counseling while hospitalized.

Core Measures

The NHQR and NHDR track a broad array of health care measures and have added measures each year. The 2010 reports continue to focus on a consistent subset of measures, the "core" measures, which includes the most important and scientifically supported measures in the full measure set. In 2005, the Interagency Work Group selected the core measures from the full measure set. For most core measures, findings are presented each year. A subset of the core measure group is presented on an alternating basis, typically rotating across odd or even years of the report. All alternating core measures are included in trend analyses. "Noncore" measures are included in summary statistics and may be presented to complement core measures in key areas.

Examples of alternating measures include the set of measures focusing on breast cancer and colorectal cancer. While measures are annually tracked, breast cancer measures are presented in odd calendar years; these measures were contained in the 2009 reports. Colorectal cancer measures are also tracked annually, but results are presented in even calendar years, such as in the 2010 quality and disparities reports.

Composite Measures

Policymakers and others have voiced their support for composite measures of quality because they can be used to facilitate understanding of information from many different measures. A composite measure summarizes care that is represented by individual measures that are often related in some way, such as components of care for a particular disease or illness. Composite measures are composed of two or more process^vi or outcome^vii measures that have been recommended or identified as a "best practice" in the treatment or prevention of complications associated with specific conditions, such as diabetes. Since measures used to construct composites represent various dimensions or processes of care, they provide a more complete understanding of the quality of the U.S. health care system. To ensure that actionable information is available, estimates of performance on the individual measures that make up a composite measure are available in an appendix to these reports.

Decisions concerning the appropriateness of pooling data to generate a composite measure were discussed with data sources. Several of the composite measures included in the reports were developed, tested, and estimated by the data source or other public or private organizations for use in quality assessment, monitoring, and improvement activities.

Composite measures in the NHQR and NHDR are created in a variety of different ways. The appropriateness model is sometimes referred to as the "all-or-none" approach because it is calculated based on the number of patients who received all of the services they needed. One example of this model is the diabetes composite, in which a patient who receives only one or two of the three recommended services would not be counted as having received all recommended care.

The opportunities model assumes that each patient needs and has the opportunity to receive one or more processes of care, but not all patients need the same care. Composite measures that use this model summarize the proportion of appropriate care that is delivered. The denominator for an opportunities model composite is the sum of opportunities to receive appropriate care across a panel of process measures. The numerator is the sum of the components of appropriate care that are actually delivered.

The composite measure of recommended hospital care for heart failure is an example of the use of the opportunities model. The total number of patients who receive treatments represented by individual components of the composite measure (e.g., evaluation of left ventricular ejection fraction and use of angiotensin-converting enzyme (ACE) inhibitor or angiotensin receptor blocker (ARB) among patients with left ventricular systolic dysfunction) is divided by the sum of all of the opportunities to receive appropriate care.

The CAHPS® (Consumer Assessment of Healthcare Providers and Systems) surveys have their own method for computing composite measures that has been in use for many years. These composite measures average individual components of patient experiences of care and are presented as the proportion of respondents who indicate that providers sometimes or never, usually, or always performed well.

Composite measures that relate to rates of complications of hospital care are postoperative complications and complications of central venous catheters. For these complication rate composites, an additive model is used that sums individual complication rates. Thus, for these composites, the numerator is the sum of individual complications and the denominator is the number of patients at risk for these complications. The composite rates are presented as the overall rate of complications. The postoperative complications composite is a good example of this type of composite measure: if 50 patients had a total of 15 complications among them (regardless of their distribution), the composite score would be 30%.

Other Measure Characteristics

Core and noncore measures may be characterized as "process" or "outcome" measures. Process measures track receipt of medical services and whether providers rendered care according to accepted standards. Outcome measures are indicative of the result or impact of medical care. Many factors other than the care received affect health outcomes; these include lifestyle, social and physical environment, and genetic predisposition to disease. Therefore, outcome measures are typically adjusted for risk or patient characteristics that may influence outcomes.

Both process and outcome measures are included in the 2010 NHQR and NHDR; both types of measures are not reported for all conditions due to data limitations. For example, data on HIV care are suboptimal, so no HIV process measures are included as core measures. In addition, not all core measures are included in trending analysis, because 3 or more years of data are not always available. Ideally, process measures and related outcome measures would be tracked in tandem. In reality, data are typically unavailable to examine the relationship between structural, process, and outcome measures.

Process measures reported in the NHQR and NHDR, particularly in the chapter on effectiveness, are grouped into categories related to the type of care: prevention, acute treatment, and chronic disease management. There is a sizable overlap among the care types and some measures may be considered to belong in more than one type of care category.

• Prevention. Caring for healthy people is an important component of health care. Educating people about healthy behaviors and lifestyle modification can help to postpone and avoid illness and disease. In addition, detecting health problems at an early stage increases the chances of effectively treating them, often reducing suffering and costs. Many of the preventive measures tracked in the reports come from the U.S. Preventive Services Task Force and the Centers for Disease Control and Prevention Advisory Committee on Immunization Practices.
• Treatment. Even when preventive care is ideally implemented, it cannot entirely avert the need for acute care. Delivering optimal treatment for acute illness can help reduce the effects of illness and promote the best recovery possible.
• Management. Some diseases, such as diabetes and ESRD, are chronic, which means they cannot simply be treated once; they must be managed across a lifetime. Management of chronic disease often involves lifestyle changes and regular contact with a provider to monitor the status of the disease. For patients, effective management of chronic disease can mean the difference between healthy living and frequent medical problems.

A list of core measures included in the 2010 NHQR and NHDR is shown in Table 1.3 at the end of this chapter. Measures are identified according to the priorities addressed, dimension or type of care, and focus on structure (access), process (prevention, acute care, chronic care), or outcome of care. The table also notes whether a measure is a composite measure.

Analyses

In the NHQR, measures are tracked for different groups, such as age, gender, and geographic location. In the NHDR, comparisons are made across groups defined by race, ethnicity, income, education, activity limitations, and geographic location. In general, either the largest subgroup or the best performing subgroup is used as the reference; unless specified, this would typically be individuals ages 18-44 for age contrasts, individuals with private health insurance for insurance contrasts, and non-Hispanic Whites for racial contrasts.

Two criteria are applied to determine whether the difference between two groups is meaningful:

1. First, the difference between the two groups must be statistically significant with p=0.05 on a two-tailed test.
2. Second, the relative difference between the comparison group and the reference group must be at least 10% when the measure is framed positively as a favorable outcome or negatively as an adverse outcome.

To further address the interrelationships among measures, group demographic characteristics, and socioeconomic factors, multivariate regression analyses were conducted for a small number of measures that had data available to examine the relationship between the measure, race/ethnicity, and socioeconomic factors. These analyses, which are shown for selected measures in the NHDR chapter on priority populations, generated adjusted percentages that quantify the magnitude of disparities after controlling for a number of confounding factors. For example, results of multivariate analyses are shown for an effectiveness measure"the percentage of people with diabetes who received recommended care for diabetes. Values of these measures are compared for different racial and ethnic groups after adjusting for differences in the distributions of income, education, insurance, age, gender, and geographic location.

Annual Rates of Change and Trend Analyses

For all measures for which reliable trend data are available, analyses are conducted to assess the annual rate of change. The 2010 reports use regression analysis to estimate average annual rate of change. Regression models were specified as follows:

• ln(M) = β₀ + β₁(Y), where

  ln(M) = natural logarithm of the measure value (M)

  β ₀ = intercept or constant
  β ₁(Y) = coefficient corresponding to year (Y)

Using regression results, the average annual rate of change was calculated as 100 x (exp(β₁) - 1).

Data in the NHQR and NHDR are unavailable at the person level, and aggregated estimates are used throughout analyses. The regression-estimated annual rate of change was reported only when at least three data points—or 3 years of aggregated data—were available for a measure. For inclusion as either improving or getting worse, the average annual rate of change must be at least 1% per year when the measure is framed positively as a favorable outcome or negatively as an adverse outcome.

Progress on individual measures is reported as follows:

• Progress on a measure is deemed to be improving or getting better if the annual rate of change is 1% or greater, in the desirable direction.
• Progress on a measure is deemed to be getting worse when the annual rate of change is 1% or greater, in the undesirable direction.
• Progress has remained the same if the annual rate of change is ≤1% in either the desirable or undesirable direction.

Across subpopulation groups, average annual change was estimated to ascertain the extent to which disparities in quality and access measures were increasing, decreasing, or remaining the same over time. Calculation of change in disparities was conducted in a manner similar to that described above, with the exception that a linear regression (as opposed to a log-linear regression) was used to estimate annual change for population subgroups. Change in disparities was estimated as the difference in the average annual change between the comparison and reference groups.

Measures for which the difference between groups was >1 indicate that the disparity is getting larger whereas differences < -1 indicate that the size of the disparity is getting smaller. Values between -1 and 1 suggest that group differences have not changed over time. Due to methodological changes over time, changes in data used to construct measures across years, and changes to the measure set, it is not appropriate to compare the annual change or rates of change for measure groups discussed in this year's report with those from prior years.

Summary Measures in the Highlights

The Highlights chapter reports findings across broad panels of measures. Contained in the highlights are summary data detailing:

• Measures or groups of measures for which selected population groups (e.g., race/ethnicity, income, metropolitan and nonmetropolitan groups) performed better, worse, or the same as the reference group;
• Distribution of change over time in quality and access measures for population subgroups; and
• Change in quality and access over time, by type of service (preventive care, acute care, chronic disease management) and outcomes.

This process is more complicated because data on all measures are not collected each year. In the summary trend analyses, we obtain all available data points between the year 2000 and the current data year for each measure. Composite measures are included in the core measure category. To avoid duplication of estimates within the other categories, composite measures are not included in other categories where estimates from their component measures are used. For example, the diabetes composite measure (which includes hemoglobin A1c measurement, eye exam, and foot exam) contributes to the overall rate for the core measures group but not to the diabetes group rate, which uses the estimates from the three noncore component measures. For most measures, trends include data points from 2001-2002 to 2007-2008.

Using the analytic approach previously described, we calculated the sum of measures that were identified as better, worse, or the same (when considering subgroup differences) or that were improving, worsening, or remaining the same over time (when considering trend data). The distribution of measures by subpopulation, type of service, or type of measure (i.e., quality or access) are presented as a way to summarize the status of health care quality and disparities in the United States.

Standardization of language to describe differences in the value of a measure across time and subgroups is part of the effort by HHS and AHRQ to provide information on where the Nation is"and is not"making progress in reducing disparities in health care. Furthermore, as detailed below, the use of benchmarks, or "best known level of attained performance," provides an additional way to monitor progress. It also offers an approach to measuring disparities by projecting the amount of time that would be needed for selected groups to achieve the benchmarks.

Benchmarks

Pursuant to the IOM's recommendations, when data were available, the 2010 NHQR and NHDR include measure-specific benchmarks that reflect the highest level of performance documented for individual measures. Benchmarks enable readers to assess national and State performance relative to that of the highest performing States, organizations, and other entities. They also aid in establishing reasonable performance improvement goals. From an equity perspective, standards of performance should not differ across population groups. As such, benchmarks corresponding to measures included in both the NHQR and NHDR were identical.

For measures for which they are reported, benchmarks were estimated as the measure average for the 10% of States that had the best performance on the measure of interest. For benchmarking purposes, the District of Columbia is treated as a State. Benchmarks were estimated only if data were available for a minimum of 30 States. Before settling on the approaches used, we considered alternative methods for designating benchmarks. One method would have limited "best attained performance" to the one State with the highest performance on a measure. This approach was rejected because of concerns about the reliability of data from one State, especially if the State is unique in terms of the characteristics of the population, health care infrastructure, or practice patterns. The top-performing State may simply be an outlier.

State-level estimates used in constructing benchmarks were primarily calculated from the same data source as the measure. In some cases, such as when the number of individuals sampled from a specific State was too small, data did not support estimation at a subnational level and benchmarks were not identified. We made exceptions for three measures derived from the Medical Expenditure Panel Survey (MEPS) and the National Health Interview Survey (NHIS).

For these measures of colorectal cancer screening, diabetes care, and pneumococcal vaccination, almost identical data were available from Behavioral Risk Factor Surveillance System (BRFSS) State data. However, BRFSS sampling and mode of administration differ from MEPS and NHIS. Hence, to calculate a benchmark for these measures, we first calculated the ratio of the top 10% achievable benchmark to the overall national estimate from BRFSS. We then applied this ratio to the overall national estimates from MEPS or NHIS. For example, if the BRFSS benchmark to national estimate for a measure was 1.5, we would multiply the national estimate for that measure from MEPS by 1.5 to obtain a corresponding benchmark.

Time To Achieve Benchmark

Also new to the 2010 reports are projections of the time expected for population subgroups to achieve the designated benchmark, based on past performance. Using standard linear regression of the actual values over time and extrapolating to future years, we calculated the time required for the population, or population subgroup, to perform at the level of the top-performing States.

Since projections of future performance were based on past performance data, it was necessary to ensure reliability by limiting estimates to those cases in which at least three data points were available.

An important caveat to consider in using information on time to achieve benchmarks is that the linear estimation approach used to derive these estimates assumes that characteristics of the population, technology, and health care infrastructure remain constant. Changes in the characteristics of the population or health care system may be expected to alter achievement of benchmarks. Advancements in medical science, changes in the organization of health services, or reductions in the uninsured population following implementation of the Patient Protection and Affordable Care Act (P.L. 11-148) would be expected to alter the performance trajectory. In some cases, the time to achieve the benchmark will drop, while in other cases it may increase.

Time to achieve the benchmark was not estimated for all measures in the NHQR and NHDR. Time to benchmark is not reported if:

• The average annual rate of change in a measure is less than 1%.
• The time to benchmark is estimated at 25 or more years.
• Trends over time show movement away from the benchmark (these occurrences are mentioned in the reports).
• The direction or trend changes over time; operationally, these were identified as cases in which there are at least 4 years of data showing "upward" movement and at least 4 years of data showing "downward" movement.

Quantifying Disparities

In the Highlights and Priority Populations chapters of the NHDR, the extent of disparities across the core measures is summarized for Black, Hispanic, Asian, Native Hawaiian and Other Pacific Islander (NHOPI), American Indian and Alaska Native (AI/AN), and poor populations. Racial, ethnic, and socioeconomic groups are compared with a designated reference group for each core measure. Each group could receive care that is worse than, about the same as, or better than the reference group. For each group, the percentages of measures for which the group received worse care, similar care, or better care were calculated.

In the Priority Populations chapter of the NHDR, which presents information on each population separately, all core measures with available data are used when summarizing trends in disparities across groups. For example, much less information is available for income groups than for racial and ethnic groups. Rates relative to standard reference groups are used to quantify the magnitude of disparities and to identify the largest disparities specific groups face. For each group, the group rate was divided by the reference group rate to calculate the relative rate for each core measure, with each core measure framed negatively (e.g., for immunization, the likelihood of not receiving the vaccine). Relative rates of selected core measures are presented in the Highlights chapter of the reports.

Presentation of Reports

As in past reports, the NHQR and its companion NHDR continue to be formatted as chartbooks. Each chapter begins with a description of the importance of the topic. After introductory text, charts and accompanying findings highlight a small number of measures relevant to the topic. Where applicable, key findings from the NHDR are included in the NHQR, and NHQR findings are reported in the text of the NHDR. Readers should refer to the report from which results have been drawn to gather additional details on the data presented.

Unless otherwise stated, only those findings that meet the "meaningfulness criteria," as previously described, are presented in the bullets that accompany each figure. When these data are available and relevant, the NHQR charts show contrasts by:

• Age.^viii
• Gender.
• Insurance status.
• Geographic location (rural versus urban).

To the extent that data are available, charts in the NHDR typically show contrasts by:

• Race: Whites, Blacks, Asians,^ix NHOPIs, AI/ANs, and people of more than one race.
• Ethnicity: Hispanics and non-Hispanics.^x
• Income: Poor, near poor, middle income, and high income.^xi
• Education: People with less than a high school education, high school graduates, and people with at least some college education.

When data support stratified analyses, a figure showing racial and ethnic differences stratified by socioeconomic factors is included. These data are summarized in bullet format. Figures include a note about the reference group for population-based measures and the unit of analysis for measures based on services or events.

Many of the core and composite measures have multiple years of data, so figures typically illustrate trends over time. Figures include a notation about the denominator, which is either the reference population for population-based measures or the unit of analysis for measures based on services or events from provider- or establishment-based data collection efforts.

To systematically identify the relationship between geographic location and quality of care, when possible, findings in the NHQR and NHDR show measures of quality of care for individuals residing along the urban-rural continuum described above.

Defining Individuals With Disabilities

For the purpose of the NHDR, people with disabilities are those with physical, sensory, and/or mental health conditions who also have an associated decrease in functioning in such day-to-day activities as bathing, walking, doing everyday chores, and/or engaging in work or social activities. This is consistent with the Americans With Disabilities Act (ADA) of 1990,^xii which defines disability to include "a physical or mental impairment that substantially limits one or more major life activities of such individual" as well as with Federal program definitions based on the ADA.

In displaying the data on disability, paired measures are shown to preserve the qualitative aspects of the data:

• Limitations in basic activities represent problems with mobility and other basic functioning at the person level.
• Limitations in complex activities represent constraints encountered when people, in interaction with their environment, attempt to participate in community life.

The use of paired measures of basic and complex activity limitations is conceptually similar to the way others have defined "disability." It is also consistent with the International Classification of Functioning, Disability, and Health separation of activities and participation domains. These two categories are not mutually exclusive; people may have limitations both in basic activities and complex activities. Further information can be found in the Individuals With Disabilities or Special Health Care Needs section of the NHDR, in the chapter on Priority Populations.

Table 1.3. Core measures included in the 2010 NHQR and NHDR, by measure characteristic

* No new data.

^vi "Receipt of three recommended diabetes services" is one example of a composite process measure. This composite was formed by combining information on adults with diabetes who received the following clinical preventive services: hemoglobin A1c measurement, a dilated eye exam, and a foot exam.
^vii "Adults with ambulatory visits who reported poor communication with health providers" is an example of a composite outcome measure. This composite was formed by combining information on patient perceptions of their providers, including the extent to which the provider listened to the patient, respect shown by the provider, time spent with the patient, and explanations offered by providers.
^viii Unless otherwise specified, the NHQR and NHDR define children as individuals under the age of 18; adults include people age 18 and over.
^ix Asian includes the former category of Asian or Pacific Islander prior to Office of Management and Budget guidelines, when information was not collected separately by group.
^x Not all data sources used in the NHDR collect information by race and ethnicity separately. In such cases, comparisons are made by combining racial/ethnic group categories (e.g., comparing non-Hispanic Blacks and Hispanics with non-Hispanic Whites.)
^xi Unless otherwise indicated, throughout this report, poor is defined as having family income less than 100% of the Federal poverty level (FPL); near poor refers to incomes between 100% and 200% of the FPL; middle income refers to incomes between 200% and 400% of the FPL, and high income includes incomes 400% or more of the FPL. These are based on U.S. Census poverty thresholds for each data year, which are used for statistical purposes.
^xii 42 U.S.C. 12102
^xiii Cancer measures alternate by year. Data on colorectal cancer are presented in "even year" reports, and data on breast cancer are presented in "odd year" reports.

Proceed to Next Section