Chapter 1. Introduction and Methods

In 1999, Congress directed the Agency for Healthcare Research and Quality (AHRQ) to produce an annual report, starting in 2003, on health care quality in the United States (42 U.S.C. 299b-2(b)(2)). AHRQ, with support from the Department of Health and Human Services and private-sector partners, has designed and produced the National Healthcare Quality Report (NHQR) to respond to this legislative mandate.

This is the seventh annual report on the state of health care quality nationally. Similar to the previous six reports, it is designed to summarize data across a wide range of patient needs, including staying healthy, getting better, living with chronic illness and disability, and coping with the end of life. The main purpose of the report is to show readers the extent to which care in the United States is delivered in an effective, safe, timely, patient-centered, and efficient manner.

The first NHQR was a comprehensive national overview of the quality of health care that the general U.S. population received. The 2004 NHQR initiated a second critical goal of the report series: tracking the Nation's quality improvement progress. The 2005 NHQR introduced a set of core measures and a variety of new composite measures. The 2006 NHQR continued to improve data, measures, and methods, adding databases and measures and refining methods for quantifying and tracking changes in health care. The 2007 NHQR launched a new chapter on health care efficiency. The 2008 NHQR included an expanded chapter on patient safety.

This chapter summarizes the methodological approaches AHRQ has taken in producing the 2009 NHQR. Material that is new in this year's report includes sections on:

• Health care-associated infections.
• Patient safety culture in hospitals.
• Pain management among home health care and hospice patients.
• Care coordination.
• Potentially avoidable hospitalizations among home health care and nursing home patients.
• Efficiency of care for congestive heart failure.
• Reduction of unnecessary costs.

As in previous years, the 2009 NHQR was written by AHRQ staff, with the support and guidance of AHRQ's National Advisory Council and the Interagency Work Group for the NHQR. 

How This Report Is Organized

The basic structure of the report consists of the following:

• Highlights summarizes key themes and highlights from the 2008 report.
• Chapter 1: Introduction and Methods documents the organization, data sources, and methods used in the 2009 report and describes major changes from previous reports.
• Chapter 2: Effectiveness examines the quality of health care in the general U.S. population. This year, this chapter has been reorganized around eight clinical areas (cancer, diabetes, end stage renal disease, heart disease, HIV and AIDS, maternal and child health, mental health and substance abuse, and respiratory diseases) and three types of health care services that typically cut across clinical conditions (lifestyle modification, functional status preservation and rehabilitation, and supportive and palliative care). Measures of the quality of health care used in this chapter are identical to measures used in the National Healthcare Disparities Report (NHDR) except when data to examine disparities are unavailable for inclusion in the NHDR.
• Chapter 3: Patient Safety tracks measures of patient safety, including health care-associated infections, postoperative complications, other complications of hospital care, and complications of medications. A new section this year focuses on patient safety culture in hospitals and indicates the extent to which hospital staff incorporate patient safety principles and practices into their work.
• Chapter 4: Timeliness examines the delivery of time-sensitive clinical care and patient perceptions of the timeliness and accessibility of their care.
• Chapter 5: Patient Centeredness tracks patients' experiences with care in an office or clinic and satisfaction with communication during a hospital stay in order to incorporate the patient's experience and perspective into the report. This year's chapter also includes a new section on care coordination.
• Chapter 6: Efficiency presents a conceptual view and an expanded set of analyses of this dimension of health care performance. Efficiency was first introduced in the 2007 NHQR.

Appendixes include the following:

• Data Sources provides information about each database analyzed for the NHQR, including data type, sample design, and primary content.
• Measure Specifications provides information about how to generate each measure analyzed for the NHQR. Measures highlighted in the report are described, as well as other measures that were examined but not included in the text of the report.
• Data Tables provides detailed tables for most measures analyzed for the NHQR, including measures highlighted in the report text and measures examined but not included in the text. A few measures cannot support detailed tables and are not included in the appendix.ⁱ 

Measure Set for the NHQR and NHDR

The NHQR and NHDR track a broad array of health care measures and have added measures each year. In this report, we begin the process of trimming the measure set by retiring some measures that have attained high levels of overall performance. As in previous years, the 2009 reports focus on a subset of core report measures. In addition, composite measures are included to provide readers with a summary picture of some aspect of health care by combining information from multiple component measures.

Retired Measures

Previous NHQRs have demonstrated that most measures of health care quality tend to improve. Since the first NHQR, significant improvements in a number of measures of quality of care have occurred. U.S. health care providers have achieved overall performance levels exceeding 95% on five measures tracked in past NHQRs.

The measures that have achieved 95% performance are:

• Adults with diabetes who had their blood cholesterol checked.
• Hospital patients with heart attack who received aspirin within 24 hours of admission.
• Hospital patients with heart attack who were prescribed aspirin at discharge.
• Hospital patients with heart attack who were prescribed a beta blocker at discharge.
• Smokers with heart attack who received smoking cessation counseling while hospitalized.

The success of these measures limits their utility for tracking improvement over time. Because these measures cannot improve further to a significant degree, including them in the measure set creates a ceiling effect that may distort quantification of rate of change over time. Hence, we have retired these measures from our measure set for tracking purposes. However, we will continue to gather data on them and will add them back if their performance falls below 95%.

Core Measures

This 2009 NHQR continues to focus on a consistent subset of measures, the "core" measures, which includes the most important and scientifically supported measures in the full NHQR measure set. In 2005, the Interagency Work Group selected the core measures from the full measure sets and findings based on the measures presented each year. In addition, "noncore" measures are included in summary statistics and may be presented to complement core measures in key areas. For some topics, the NHQR uses alternating sets of core measures. These measures are listed in Table 1.1. All alternating core measures are included in analyses of rate change.

Table 1.1. Alternating core measures

^* The measures listed in this column will be presented again in the 2010 reports.

All core measures fall into two categories: process measures, which track receipt of medical services; and outcome measures, which in part reflect the results of medical care. Both types of measures are not reported for all conditions due to data limitations. For example, data on HIV care are suboptimal; hence, no HIV process measures are included as core measures. In addition, not all core measures are included in trending analysis, because 2 or more years of data are not always available. Ideally, process measures and the outcome measures they could affect would be tracked in tandem. However, data to support such processoutcome measurement pairs are not typically available. Related process and outcome core measures in the 2009 NHQR measure set are listed in Table 1.2. Other core measures are listed in Table 1.3.

Table 1.2. Related core process and outcome measures (measures without new data in italics)

Composite Measures

In some cases, measures can be combined to form a single composite measure of health care quality. A composite measure summarizes care that is represented by individual measures that are often related in some way, such as components of care for a particular disease or illness. Policymakers and others have voiced their support for composite measures because they can be used to facilitate understanding of information from many individual measures. The effort to develop new composites is ongoing and, in 2006, a number of composite measures were added.ⁱⁱ The complete list of composite measures is shown in Table 1.4.

Composite measures in the NHQR are created in a variety of different ways. The appropriateness model is sometimes referred to as the "all-or-none" approach because it is calculated based on the number of patients who received all of the services they needed. One example of this model is the diabetes composite, in which a patient who receives only one or two of the three services would not be counted as having received all recommended care.

The opportunities model assumes that each patient needs and has the opportunity to receive one or more processes of care, but not all patients need the same care. Composite measures that use this model summarize the proportion of appropriate care that is delivered. The denominator for an opportunities model composite is the sum of opportunities to receive appropriate care across a panel of process measures. The numerator is the sum of the components of appropriate care that are actually delivered. The composite measure of recommended hospital care for heart failure is an example where this model is applied. The total number of patients who receive treatments represented by individual components of the composite measure (e.g., evaluation of left ventricular ejection fraction and ACE inhibitor or ARB among patients with left ventricular systolic dysfunction) is divided by the sum of all of these opportunities to receive appropriate care.

Measures from the CAHPS® (Consumer Assessment of Healthcare Providers and Systems) surveys have their own method for computing composite measures that has been in use for many years. These composite measures average individual components of patient experiences of care. They are typically presented as the proportion of respondents who reported that providers sometimes or never, usually, or always performed well.

Composite measures that relate to rates of complications of hospital care are postoperative complications and complications of central venous catheters. For these complication rate composites, an additive model is used that sums individual complication rates. Thus, for these composites, the numerator is the sum of individual complications and the denominator is the number of patients at risk for these complications. The composite rates are presented as the overall rate of complications. The postoperative complications composite is a good example of this type of composite measure: If 50 patients had a total of 15 complications among them (regardless of their distribution), the composite score would be 30%.

 
Table 1.4. Composite measures in the 2009 NHQR and NHDR

Presentation

As in past reports, the NHQR and its companion NHDR continue to be formatted as chartbooks. Each section in the 2009 report begins with a description of the importance of the section's topic in a standardized format. After introductory text, charts and accompanying findings highlight a small number of measures relevant to the topic. Sometimes these charts show contrasts by age, when age data are available and relevant.

Almost all core measures and composite measures have multiple years of data, so figures typically illustrate trends over time. Figures include a notation about the denominator, either the reference population for population-based measures or the unit of analysis for measures based on services or events from provider- or establishment-based data collection efforts.

New themes emphasized in this report include exploring the effects of health insurance on quality of care and examining State variation in processes and outcomes together. The NHDR has identified insurance as an important determinant of disparities in health care. To systematically identify the relationship between insurance and quality of care, when possible, findings in the NHQR show measures of quality of care for individuals with different types of insurance. For those under age 65, individuals with any private insurance, public insurance only, and no insurance are typically compared. For those age 65 and over, individuals with Medicare and private insurance, Medicare and other public insurance, and Medicare only are typically compared.

Previous NHQRs have shown that wide variation exists across States in processes and outcomes of health care. This year, the NHQR introduces a new type of State variation map. Rather than focus on a single process or outcome measure, these maps identify States that perform poorly on both a process measure and a related outcome measure. These maps do not imply causality; improvements in processes of care typically affect outcomes some time in the future. Rather, these maps are intended to help identify those States that may have the greatest motivation to improve performance in this area. These maps generally focus on States in the worst performing quartile of process and outcome measures.

In addition, to place findings in the context of other Federal reporting initiatives, this report indicates where NHQR measures are also included in Healthy People 2010. Note that the Healthy People 2010 targets represented in the report figures, where applicable, reflect target values that were current when the reports were being prepared. Targets may be revised as new information becomes available. Therefore, the targets shown on the figures may differ from those in past reports or subsequent revisions. Also, Healthy People 2010 targets are only referenced in relation to the total population, not particular age groups. In addition, the data source for estimates reported here must be the same as the Healthy People 2010 data source in order for comparisons to be made. 

Databases

Table 1.5 lists the databases used in the 2009 reports.

Table 1.5 lists the databases used in the 2009 reports.

^a Not all States participate in HCUP. For details, see HCUP entry in Appendix A, Data Sources.
Note: Measures from the American Community Survey are used only in the NHDR. For details on these surveys, refer to Chapter 1, Introduction and Methods, in the 2009 NHDR. 

Analyses

This section summarizes the methodologies behind the analyses shown in the report. Unless otherwise specified, only findings that meet our criteria for significance are discussed in the text of this report.

For all measures, analyses are conducted to assess change over time. For individual measures, the average annual rate of change is calculated between the earliest and the most recent estimates within a specified date range. Consistent with Health, United States, a formula that produces the geometric rate of change is used for this calculation for each measure.^iv This geometric rate of change assumes the same rate of change each year between the two time periods. For each measure discussed in the reports, two criteria are then applied to determine whether a meaningful trend exists:

• First, the difference between the oldest and most recent estimates must be statistically significant with alpha ≤0.05 on a two-tailed test.
• Second, the average annual rate of change must be at least 1% per year when the measure is framed either positively as a favorable outcome or negatively as an adverse outcome.

For example, assume a favorable outcome, receipt of a needed service, increased from 90% to 94% over a 5-year period. To assess meaningfulness, statistical significance of this difference would first be assessed using a two-tailed test. If significant with alpha ≤0.05, the average annual rate of change would then be calculated using the geometric rate of change formula. In this case, [(94/90)^1/5-1] x100 yields an average annual rate of change of 0.9% per year, less than our threshold of 1% per year. The measure is then converted into an adverse outcome, lack of receipt of a needed service, which decreased from 10% to 6% over the time period. Applying the geometric rate of change formula, [(6/10)^1/5-1] x100, yields an average annual rate of change of -9.7% per year. Because this rate of change with the measure framed negatively as an adverse outcome exceeds our 1% threshold, we consider this change to be meaningful.

In addition, many measures are tracked for different groups defined by age or insurance status, and comparisons among groups are made. In general, the largest groups are used as the standard reference groups; unless specified, this would typically be individuals ages 18-44 for age contrasts and individuals with private health insurance for insurance contrasts. Two criteria are applied to determine whether the difference between two groups is meaningful:

• First, the difference between the two groups must be statistically significant with alpha≤0.05 on a two-tailed test.
• Second, the relative difference between the comparison group and the reference group must be at least 10% when the measure is framed either positively as a favorable outcome or negatively as an adverse outcome.

In the Highlights chapter, we also summarize rates of change over time across broad panels of measures. This process is more complicated because data on all measures are not collected each year. Therefore, specifying the optimal time period for analysis without excluding large numbers of measures has been a challenge. Specific issues include:

• Changes in the measure set over time.
• Changes in the data source over time.
• Lack of availability of data for particular data years.
• Recalculation of prior years' data.

Changes in the measure set may result from the deletion of measures due to lack of availability of new data or a determination by the NHQR's Interagency Work Group that a measure no longer meets its criteria for inclusion. Changes also result from the addition of a measure. For example, this report includes a core measure for daily use of preventive medicine for current asthma. This measure was added in 2008 and uses data from AHRQ's Medical Expenditure Panel Survey (MEPS). Data for this measure were first collected in 2003. The latest MEPS data year available for this NHQR was 2006. Therefore, for this measure data were only available for 4 years: 2003, 2004, 2005, and 2006. A 5-year or longer period might be available for other measures.

For this and other reasons (e.g., variability of collection schedules among the different data sources used by the NHQR), if a strict time-interval criterion for trend analysis were used (e.g., only the 2000 and 2006 data years), a large number of measures would be excluded. The approach taken for this year's report favors inclusion of as many measures as possible over a strict application of a minimum number of data points or time interval.

For this year's summary trend analyses, for each measure, we obtain all available data points between the year 2000 and the current data year. Linear regression is applied to these data and the slope of the regression line is used as the estimate of change over time.^v Across different panels of measures, these estimates are then summarized as means and medians.

One other methodological issue should be noted. Composite measures are included in the core measure category. To avoid duplication of estimates within the other categories, composite measures are not included in other categories where estimates from their component measures are used. For example, the diabetes composite measure (which includes HbA1c measurement, eye exam, and foot exam) contributes to the overall rate for the core measures group but not to the diabetes group rate, which uses the estimates from the three noncore component measures.

Various words and phrases might be used to refer to a change, depending on the specific measure being discussed. For example,"more likely to," "significantly below," "decreased," "had the highest rate," "change," "improvement," "statistically higher," and "less likely to" all refer to changes that meet the two criteria listed above. Although the explicit use of the term "statistically significant" is warranted in some cases, imposing its use in every sentence where a change is discussed would be overly cumbersome. Also, not every significant change among data years or populations is noted. Therefore, no conclusions should be drawn if a numeric difference in a figure is not referenced in the corresponding text or bullet.

Due to the methodological changes discussed here, changes to estimates for data from prior years, and changes to the measure set, it is not appropriate to compare the rates of change for measure groups discussed in this year's report with those from prior years.

Finally, this report conforms to the Government Printing Office Style Manual. In some cases, terms or spelling may vary to reflect an original data source or an agency or program name. For example, "health care" usually appears as two words but may appear as one word in an agency name, such as the Agency for Healthcare Research and Quality. These minor variations in spelling and usage do not alter the meaning or intent of the data and are purely cosmetic in nature.

ⁱ NHQR data can be accessed through NHQRDRnet, an online tool that provides Internet users with an opportunity to specify dimensions of analysis and produce data tables. NHQRDRnet is available through the AHRQ Web site at http://nhqrnet.ahrq.gov.
ⁱⁱ Go to Chapter 1, Introduction and Methods, in the 2006 Qualities Report.
ⁱⁱⁱ The vaccines included in this measure are based on the corresponding Healthy People 2010 objective and do not include all vaccines currently recommended for children. Specifically, varicella vaccine and vaccines added after 1998 to the recommended schedule for children up to 35 months of age are not included in this composite measure.
^iv [(Pn/Po)^1/N-1]x100 where Pn=variable at later time period, Po=variable at earlier time period, and N=number of years in interval. From the entry for Average Annual Rate of Change in Appendix II, Definitions and Methods, Health, United States, 2008. Available at: http://www.cdc.gov/nchs/hus.htm.
^v More precisely, to calculate annual rate of change, the natural log of each year's data point is entered into a linear regression model.

Proceed to Next Section