Page 1 of 1

Chapter 5: Enhancing Data Resources

Future Directions for the National Healthcare Quality and Disparities

As the nation moves forward with enhanced health information technology ( Health IT ) and building a health care data infrastructure, AHRQ can leverage its position as the producer of the national healthcare reports to identify health care quality measurement and data needs. Subnational data, for example, can inform trends on emerging measures and serve as a model for the development of more widespread data collection on measures that show promise for quality improvement. Race, ethnicity, and language need, among other sociodemographic variables, continue to influence the quality of care individuals receive. For that reason, standardized information regarding these variables is a necessary component of the national health care data infrastructure.

Collecting and reporting accurate, comparative data that are useful to measuring health care quality is a “time-consuming” process (NPP, 2008). There is movement among quality improvement stakeholders to harmonize performance measures to reduce the data burden on organizations and health care providers. At the same time, there is extensive development and testing of new measures to fill shortcomings in measurement areas or improve existing measures. The Future Directions committee believes AHRQ, by leveraging its position as the producer of the NHQR and NHDR can identify health care quality measurement and data needs for development, and utilize subnational data sources when national data do not yet exist. This chapter underscores the importance of the evolving national health care data infrastructure as an emerging source of information for the NHQR and NHDR. The chapter also outlines the pros and cons of using subnational data to fill needs for measurement areas in the NHQR and NHDR and proposes criteria for the use of such data.

In addition, the chapter summarizes the independent consensus study of a subcommittee to the Future Directions committee, which culminated in the report Race, Ethnicity, and Language Data: Standardization for Health Care Quality Improvement.1 The subcommittee's recommendations in that report (go to Appendix G) emphasized the need to increase the availability of standardized race, ethnicity, and language need data across the health care system. This chapter addresses the relationship of the subcommittee's findings to improving the content and analyses in the NHDR and discusses the utilization of socioeconomic and insurance status data in analyses for the NHDR and NHQR.

Building a National Data Infrastructure

Information on quality and disparities can promote understanding of where health care needs and quality gaps exist. In the mid-1960s, the National Halothane Study first indicated how data on variation in performance can advance our understanding of health care and provide opportunities for improvement. The results from the Halothane Study, which principally evaluated mortality rates in the use of anesthesia, revealed unexpected variation in surgical outcomes across hospitals. After adjusting for differences in procedure, age, and physical status, differences in death rates between institutions remained "very much larger" than the differences among anesthetics used (Subcommittee on the National Halothane Study of the Committee on Anesthesia, 1966, p. 128). Looking beyond anesthesia, health care variation by both institution and geographic region remains very much an issue in 2010, including how variation in quality affects the cost of health care (Fisher et al., 2009; Skinner et al., 2010; Weinstein and Skinner, 2010). We now understand that "unwarranted variation" occurs and must be identified in order to be addressed in a "logical and manageable fashion" (Wennberg and Wennberg, 2003, p. 614). Once health care organizations have evaluated and identified the factors contributing to undesirable variation, they are better positioned to develop and implement quality improvement interventions to reduce or eliminate it.

The absence of a national health care data infrastructure hinders the potential for national measurement and reporting to actually improve quality (James, 2003). The development of such an infrastructure has been labeled an "awesome task" (Mechanic, 2007, p. 46) that requires national coordination of performance measures, data aggregation, methodology, and technology (Roski, 2009). Yet AHRQ can play a role in defining the content for such a national health care data infrastructure by identifying and fostering measures and data sources, even if the measures and data are not yet national in scope, and by specifying measurement areas with the greatest potential to improve population health as quality and equity gaps are closed.

Data directly related to care processes and outcomes are needed to comprehensively describe the quality and quantity of care provided by individuals and institutions. Accordingly, data illuminating how care is delivered, who is delivering care, and where care is delivered are necessary to identify opportunities for system change. Electronic health records (EHRs), patient-based registries, and all-payer claims data (APCD) offer long-term potential for comprehensive patient data that can be used to measure the quality of care being provided across settings and time. These data sources have the potential to link use of services, intermediate outcomes, and demographics, and may be large enough to address questions about the quality of care provided to specific subpopulations.

The American Recovery and Reinvestment Act of 20092 authorizes and provides resources for the Office of the National Coordinator for Health Information Technology (ONC) within HHS to guide the "development of a nationwide health information technology infrastructure that allows for the electronic use and exchange of information." Proposed rules on standards to receive Medicare and Medicaid reimbursement incentives for the implementation of EHRs were issued in December 2009 and describe ways in which EHR systems should be used for purposes that include quality improvement and the elimination of disparities in health and health care (CMS, 2010).

In addition, there is potential for data linkages between health information exchanges (HIEs) and APCD databases (Rogers, 2009). An APCD database would ideally contain information on all covered services, regardless of the setting or the location of the provider, and would include eligibility information and medical, pharmacy, and dental claims. APCD databases may be able to provide data by payers and plans, and could provide the sample size necessary to report on populations and measurement areas where statistical power currently limits quality reporting. Ideally, APCD could be used to define episodes of care and to handle issues of risk and severity adjustment without the need for medical records data. In reality, putting together the requisite data and addressing patient confidentiality concerns require significant investment of time and resources. For instance, Maine, New Hampshire, and Vermont, among others, have APCD databases, but these databases do not always capture care for residents who have out-of-state plans and none of these databases have integrated Medicare data to allow long-term follow-up. Kansas' APCD database, which is called the Kansas Health Insurance Information System (KHIIS), is a repository for data from group insurers, Medicaid, the Children's Health Insurance Program (CHIP), and the state employee health plan. It does not include Medicare data and faces budgetary, political, and data quality hurdles (Allison, 2009). In December 2009, HHS announced its intent to build a universal claims database for health research.3

In the near term, multi-site clinical registries may provide data that allow the NHQR and NHDR to illustrate the potential of a health care data infrastructure for national performance measurement. The Northern New England Cardiovascular Disease Study Group, National Surgical Quality Improvement Program, and National Quality Program of the Cystic Fibrosis Foundation are examples of registries with an explicit focus on provider-specific performance, sharing data, exploring the causes of variations in outcomes, and applying established quality improvement techniques (e.g., benchmarking and site visits to high-performing providers) (American College of Surgeons, 2009; Cystic Fibrosis Foundation, 2009; Leavitt et al., 2009; Likosky et al., 2006). These collaboratives may provide insight into what levels of performance are possible.

As EHR and other Health IT provisions are implemented, and as national registries, health information exchanges, and APCD become more comprehensive and available, the potential to build the NHQR and NHDR on a solid foundation of provider- and community-specific performance measurement will become even greater. These data sources have the potential to complement or replace some of the data sources AHRQ currently uses to monitor specific conditions; however, AHRQ may face resource challenges to analyze and use new data sources without additional funding.

In the near term, AHRQ should continue to work with various stakeholders, such as states, the National Quality Forum (NQF), and other HHS agencies to stimulate data development when data do not exist to support desirable measures. Such data development could be accomplished by adding pertinent questions to existing surveys, or data elements to EHR systems and existing registries. AHRQ could work with the Centers for Medicare and Medicaid Services (CMS), for instance, to further develop datasets on a widening array of clinical services. CMS is already beginning to publicly report on risk-adjusted 30-day outcomes for acute myocardial infarction (AMI) across almost all U.S. hospitals (CMS, 2009); the reported measure tracks outcomes in addition to mortality and could supplement AHRQ's current measure on AMI mortality rates. Furthermore, AHRQ could capitalize on other opportunities for partnership in measure and data development, particularly given the contract awarded by HHS to the NQF to identify the most important quality and efficiency measures for individuals cared for under Medicare (NQF, 2009).

Additionally, in AHRQ's portfolio of research, including the burgeoning field of comparative effectiveness, there are opportunities to promote the generation of measures that may be of high impact for quality improvement. Previous AHRQ-funded research projects have yielded performance measures. For example, a project focused on aggregating utilization data on psychopharmacology use among children enrolled in Medicaid resulted in several potentially useful quality and safety measures, even though the project was not specifically aimed to develop measures (Crystal et al., 2009). Additionally, AHRQ could fund measure development activities, as it has done in the past. For example, from 1996 through 1999, AHRQ funded the Expansion of Quality of Care Measures (Q-SPAN) project to develop and test clinical performance measures focused on specific conditions, patient populations, or health care settings. AHRQ may need additional resources to support measure development in areas identified in its measurement agenda (go to Chapters 4 and 7).

The preceding discussion indicates that analysis of quality and disparities can be informed by multiple data sources—nationally representative provider-based and household surveys, administrative databases such as the Medicare and Medicaid programs and hospital discharge data, and clinical data obtained from sources such as EHRs and disease registries (IOM, 2002). Comprehensive quality and disparities reporting currently requires utilizing data available from all of these types of sources.

Filling Measurement and Data Needs

The NHQR and NHDR are a "mosaic of existent data sources" (IOM, 2001, p. 128). To compile the 2008 NHQR and NHDR, AHRQ used 35 diverse data sources, including population surveys, vital statistics databases, administrative data, and clinical data (Table 5-1). Despite the use of these data sources, the committee finds important areas of measurement for which data are not included in the NHQR and NHDR (go to Chapter 3). In many of these measurement areas (e.g., Health IT adoption and care coordination), national data sources do not support such measures. In some cases, though, the Future Directions committee believes that data sources beyond those currently included in the NHDR and NHQR have the potential to provide important insight into certain aspects of quality and disparities measurement.

Incorporating information from additional data sources into the NHQR and NHDR could help to ensure that the reports tell a more complete story of the nation's progress in improving the quality of health care. These additional data sources may be nationally representative or national in scope (e.g., the National Surgical Quality Improvement Program, the Cystic Fibrosis Patient Registry) and may provide clinical information, data on alternate payment streams, and information on populations of interest (e.g., children with special health care needs) that are not represented in large enough numbers in existing datasets used by AHRQ. For example, Healthcare Effectiveness Data and Information Set (HEDIS) data often include ambulatory clinical care measures that expand beyond information available in administrative data to provide details on actual treatment, not just testing.

Using Subnational Data in the Absence of National Data

As David Lansky of the Pacific Business Group on Health told the Future Directions committee, "a snapshot of national performance is instructive to establish a national vocabulary on quality for trending and benchmarking, but there is a risk of 'looking under the lamppost' and failing to focus on the right (and evolving) problems" (Lansky, 2009). The committee believes that looking "under the lamppost" and potentially missing important areas of quality measurement is an apt metaphor of caution for the selection of national measures for inclusion in the NHQR, NHDR, and related products.4 If the reports measure only areas for which national data are currently available, the measure selection process becomes circular, precluding development of new measures in national priority areas for health care quality improvement. For that reason, it is important for AHRQ to identify novel quality measurement possibilities and to look beyond existing data sources.

Defining Subnational Datasets

Although it is preferable that the national healthcare reports rely on nationally representative data or data that are national in scope, there are instances, whether due to insufficient sample sizes at the national level (e.g., ethnic populations in some surveys) or underdeveloped areas for measure development and reporting (e.g., end-of-life care, adoption of HIT), when subnational data may be informative for additional or otherwise overlooked measures of quality and disparities. The IOM's 2002 Guidance for the National Healthcare Disparities Report described subnational datasets as "surveys produced by single states" or surveys of "all or multiple" states or localities. Subnational data also includes, for instance, state-based APCDs.

Subnational datasets can represent health care entities (e.g., hospitals, payers) in certain areas of the country or contain data on specific population groups. Currently, AHRQ uses several subnational datasets to fill gaps in data on specific population groups and on specific measures. State-based data from states with a high proportion of specific racial or ethnic groups can help portray the health care issues specific to populations not well represented in national datasets (e.g., data for Native Hawaiians in Hawaii or on individuals of specific Asian ethnicities in California)5. The California Health Interview Survey (CHIS), for instance, provides estimates of insurance coverage and barriers to care for many of the sizable population groups present in California, such as recent immigrants, however, for which national data are lacking. AHRQ uses CHIS to supplement some information in the NHDR that is principally provided by the Medical Expenditure Panel Survey (MEPS). Other state-based surveys (e.g., the Rhode Island Health Interview Survey, the Hawaii Health Survey, and the Massachusetts State Health Survey) may also provide useful data for AHRQ; these surveys tend to have smaller samples sizes than CHIS.

Rationale for Using Subnational Datasets

For certain areas of quality and disparity reporting, national databases provide insufficient or no data. As an example, quality data for all major population groups—as defined by the Office of Management and Budget (OMB) categories of White, Black or African American, Asian, Hispanic, American Indian or Alaska Native, and Native Hawaiian or Other Pacific Islander—are often unavailable because national survey samples often contain insufficient data to stratify all measures for each population group. Due to small sample sizes, this problem particularly arises for AHRQ in the case of American Indian or Alaska Natives, and Native Hawaiian or Other Pacific Islanders.

Although oversampling has the potential to resolve information gaps, costs and logistical issues constrain the use of oversampling techniques (Madans, 2009). In an effort to provide information in the 2008 NHDR on quality measures that were otherwise limited by sample size, AHRQ included data from the Indian Health Service for several measures (AHRQ, 2009b).

As discussed in Chapter 3 and Appendix D, information gaps exist for measurement areas such as the adoption of Health IT, end-of-life care, efficiency, and care coordination. Several subnational datasets provide information that could be used to fill measurement gap areas (as examples, go to Box 5-1 and Box 5-2). A principal rationale for using subnational data is that these data would inform a priority area identified by this Future Directions committee or by the Secretary as a result of health reform6 that is not sufficiently addressed with current national data. Using subnational data could not only fill gaps where important national measures do not currently exist, but also could spur development of nationally representative data for measurement areas.

Box 5-1. Using Subnational Data to Provide Insight into Potential Health Information Technology Measures

While the adoption of Health IT is no guarantee of quality, Health IT is a stepping stone to quality improvement as it facilitates interoperability, data sharing, and streamlined work processes. Currently, data at the national level are available to report on the adoption and use of Health IT in some, but not all, health care settings. This measurement area is therefore considered developmental.

While there are not reliable estimates of the rates of Health IT use in all health care settings, national data on the adoption of Health IT in hospitals have been collected via survey by the American Hospital Association (Jha et al., 2009). Additionally, proprietary data on the uptake of computerized physician order entry (CPOE) and its impact on length of stay and costs are collected by The Leapfrog Group (The Leapfrog Group, 2010). Furthermore, the Healthcare Information and Management Systems Society (HIMSS) Analytics collects and analyzes proprietary data related to the Health IT market in hospitals and integrated health care delivery systems (HIMSS Analytics, 2010).

Regional quality improvement initiatives such as Minnesota Community Measurement, the Integrated Healthcare Association, and the Wisconsin Collaborative for Healthcare Quality measure Health IT use within their respective states (Minnesota, California, and Wisconsin, respectively) and report on measures of electronic prescribing, use of electronic lab or diagnostic results, and use of electronic clinical reminders (IHA, 2009a; Mayberry and Hunkins, 2008; Minnesota Community Measurement, 2009b; Wisconsin Collaborative for Healthcare Quality, 2009). AHRQ might feature (in a sidebar, for example) some of the measures used by these initiatives to examine the use of Health IT and its impact on quality improvement.


Box 5-2. Measuring Medical Home in Large, Population-Based Surveys

An important indicator of quality is whether individuals, especially those with chronic conditions, receive their care through a medical home, that is, a source of care that provides comprehensive, ongoing, coordinated, patient-centered care. Most questionnaires that measure whether a person has a medical home were developed for studying care coordination, communication, and doctor-patient relationships in clinical settings.

The UCLA Center for Health Policy Research included medical home measures in the 2009 California Health Interview Survey (CHIS), a large, comprehensive population health survey that the state's policy makers and researchers use to assess the prevalence and care of chronic conditions in California's ethnically and racially diverse population. CHIS developed a survey module that collects information from respondents on (1) whether they report having a medical home (i.e., a usual source of care and specific health care professional) (RAND, 2000), (2) whether in the last year they contacted their provider's office with a question about their condition and received a timely answer (AHRQ, 2006), (3) whether their provider worked with them to develop a care management plan (RAND, 2000), (4) whether the patient is confident about managing their own condition (Beal et al., 2007), and (5) whether their provider helps coordinate their medical care. These indicators are considered important elements of a medical home. CHIS's comprehensive questionnaire and large, diverse sample will permit analyses of the extent to which California residents with differing characteristics have a medical home and, of particular interest to AHRQ, the existence of disparities.

Beal and colleagues analyzed data from the 2005 Household Component of the Medical Expenditure Panel Survey (MEPS) to identify Latino subgroup variation in having a medical home, the impact of having a medical home on disparities, and the factors associated with Latinos having a medical home. The researchers used MEPS data to determine whether patients had a medical home based on (1) having a regular provider, (2) the role of the provider in total care for the patient (i.e., preventive care, ongoing health problems, referrals), (3) patient engagement in care (e.g., provider asked patient about medications), and (4) patient access to care (e.g., ability to contact provider during business hours, on nights or weekends). Because the MEPS survey oversamples Black and Latino households, the data had enough statistical power to provide unbiased national estimates (Beal et al., 2009).

Criteria for the Use of Subnational Data

As the previous discussion indicates, subnational datasets have the potentialin both the interim and long-term—to supplement information presented in the NHQR and NHDR. The committee deliberated on the degree to which AHRQ should rely on these data in the national healthcare reports. On one hand, utilizing these datasets in the NHQR and NHDR may provide insight into important opportunities for quality improvement or reduction of disparities. On the other hand, these datasets are, by definition, not nationally representative as they represent only specific populations or geographic regions. The presentation of subnational data has the potential to mislead readers; therefore, AHRQ should clearly underscore the limitations of such data. The committee suggests that AHRQ only use subnational data when national data are not available and that AHRQ should clearly present caveats to ensure that readers of the NHQR and NHDR understand what population the data represent (i.e., subnational data should not be advertised as being nationally representative). AHRQ may, for example, explicitly note: We do not currently have national data for this specific measure; these data represent a region, a particular population, or a sector. Presenting the information in either textboxes or sidebars would help clarify that subnational data are examples of areas for future measure or data development.

Recommendation 4: AHRQ should use subnational data for domains that do not yet have national data in order to illustrate the types of national data that need to be developed to satisfy measurement and data gaps. Subnational data should meet the following minimum requirements for reporting:
  • The data source allows the calculation of a measure of interest, ideally one identified as a national priority.
  • The data source uses reliable and well-validated data collection mechanisms and tested measures.
  • The sample used in the data source is representative of the population intended to be reported on (e.g., a region, state, population group) or is drawn from the entire population group even if it is not necessarily generalizable to the nation.

To further the development of strong subnational datasets and encourage the generation of needed national data, AHRQ could collaborate with sponsors of datasets such as the type identified in Table 5-2. This list is meant to illustrate the kinds of subnational datasets that may be useful but is not comprehensive in scope. These datasets share several key characteristicsthey are used to generate measures that are robust in their accuracy and actionability; they have an established infrastructure, and a process for measure development and reporting that has gained credibility and trust among key stakeholders; and, the tools and methods used are not idiosyncratic and are thus replicable in other parts of the country. AHRQ might partner with the Quality Alliance Steering Committee (QASC), the National Committee for Quality Assurance (NCQA), the Institute for Healthcare Improvement (IHI), the Robert Wood Johnson Foundation's Aligning Forces for Quality initiative, the National Association of State Medicaid Directors, the National Association of State Offices of Minority Health, the Association of State and Territorial Health Officials, health information exchanges, and other regional quality collaboratives. Although some of these organizations are national in scope, they often sponsor regional or state-based initiatives that may provide population- or measure-specific data.

The committee did not investigate whether costs or confidentiality agreements would interfere with utilization of datasets such as those included in Table 5-2 but encourages AHRQ to explore the feasibility of incorporating additional data sources and enhancing those currently used. The committee understands that AHRQ currently spends about half of its reports-related budget on data acquisition and analysis even though much of the data incorporated in the reports is provided by AHRQ's federal partners. AHRQ will need additional funding to support and expand its data acquisition to additional external sources (go to Chapter 7).

1 The full text of Race, Ethnicity, and Language Data: Standardization for Health Care Quality Improvement is available at Exit Disclaimer
2 American Recovery and Reinvestment Act of 2009, Public Law 111-5 §3002(b)(2)(B)(vii), 111th Cong., 1st sess. (February 17, 2009).
3 For more information, see the Federal Business Opportunities Web site: (accessed December 20, 2009). In the interim, state-based claims databases may provide comparative data.
4 A man is on his knees under a lamppost crawling around looking for something. A passerby asks him what is he doing. "Looking for lost keys," he replies. "Is this where you lost them?" "No, but there is light here" (Rogers and Wright, 1998; Salinger, 2006).
5 Numerous organizations including Papa Ola Lokahi, the Asian and Pacific Islander American Health Forum, and the National Indian Health Board encourage and foster the development of subnational datasets specific to racial and ethnic groups that are underrepresented in national surveys.
6 Patient Protection and Affordable Care Act, Public Law 111-148 § 3013, 3014, 111th Cong., 2d sess. (March 23, 2010).

Return to Contents
Proceed to Next Section

Current as of December 2010
Internet Citation: Chapter 5: Enhancing Data Resources: Future Directions for the National Healthcare Quality and Disparities . December 2010. Agency for Healthcare Research and Quality, Rockville, MD.