Future Directions for the National Healthcare Quality and Disparities Reports
Chapter 4: Adopting a More Quantitative and Transparent Measure Select (pt. 4)
Tools for Assessing Equity
A high-value health care system, by definition, requires the provision of equitable, high-value care to all individuals; therefore, metrics that assess equity in health care delivery should be considered in the prioritization process for measure selection. Measures in which the nation as a whole is performing well (i.e., for which there is little or no gap between the national average and achieving recommended care for the entire applicable population) may show performance gaps when the data are stratified by population subgroups. Therefore, the goal of achieving value in health care must be balanced by considering the needs of population groups that differ in age, race, ethnicity, gender, disability, and socioeconomic status. Chosen quality measures should promote the core quality dimension of equity in health care.
An inequity is a measurable, observable difference that can and should be closed (Carr-Hill, 2001; Whitehead and Dahlgren, 1991). For example, because the incidence of AIDS is more than 20 times higher among Black than White women, and two-thirds of new AIDS cases among women are in Black women (Kaiser Family Foundation, 2009), the CPB of interventions related to AIDS, such as use of highly active antiretroviral therapy, is much greater among Black women than among the population of all women.
As is further discussed in Chapter 5, the identification of disparities is often hampered by sample sizes and a lack of systematic, standardized collection of sociodemographic data. Yet large disparities that are statistically insignificant due to small sample sizes may still be indicative of problems with equity (Siegel et al., 2009). This section explores some of the established techniques and tools that allow for the identification of disparities. It is important to consider (1) whether the disparity is measured on a relative or absolute scale, (2) the reference point from which differences are measured, and (3) whether the disparity is weighted by population size or degree of inequity.
Relative and Absolute Difference
Absolute and relative measures of disparity can provide contradictory evidence regarding changes in a disparity over time. In the context of health care quality improvement, increasing relative but decreasing absolute inequality occurs when the rate of improvement is smaller for the group with the worst performance rate (Harper et al., 2010). In concert with one another, absolute and relative differences can provide a more comprehensive picture of a disparity than either method alone. The committee does not recommend a single approach to measuring disparities and instead emphasizes that the method of measurement can determine the size and direction of a potential disparity.
AHRQ presents information on disparities in terms of both relative and absolute differences in either adverse or favorable outcomes. In the Highlights section of the 2008 NHDR, AHRQ presents the three largest disparities in quality for different groups using relative differences (AHRQ, 2009c). The committee was not able to assess the validity of these rankings. A relative measure expresses the disparity as a ratio relative to the reference point or group, so that reference point becomes the unit of comparison. Absolute measures of disparity are simply the difference between a group rate and the reference group; most of the AHRQ graphs reflect absolute differences. Go to Table 4-2 for a list of ways to measure absolute and relative health disparity.
The following example highlights how examining relative and absolute differences can lead to different conclusions, especially when comparing over time. In 2000, the rate of a specific disease was 8 percent in the African American population and 4 percent in the White population. In absolute terms, this was a 4-point difference, whereas in relative terms, the African American rate was twice the White rate. In 2010, the African American rate is 6 percent, and the White rate is 3 percent. Both groups have better rates, and the African American rate has improved more than the White rate. In absolute terms, the gap has shrunk from 4 points to 3 points. In relative terms, the African American rate is still double the White rate. In this case, the relative rate does not reflect that the situation is better in 2010 than it was in 2000. A 2005 report released by the CDC advised that to promote a more complete understanding of the "magnitude of disparities," disparities should be measured in both absolute and relative terms, especially when making comparisons over time, geographic regions, or populations (Keppel et al., 2005). Additionally, Harper and colleagues have urged researchers against always using a single measure (e.g., a rate ratio), and instead advised researchers to "pay more attention to the normative choices inherent in measurement" (Harper et al., 2010, p. 22).
When both absolute and relative difference cannot be presented (due to space constraints, for instance), major medical journals are trending toward presenting absolute differences (Braveman, 2006; Dombrowski et al., 2004; Regidor et al., 2009; Rosvall et al., 2008). The advantage of this approach is that it is more consistent with using population health burden as a metric for prioritizing within populations. When both measures cannot be presented, the committee suggests AHRQ might include absolute rates in graphs and tables and add a comment in the text about whether the relative disparity is changing.
Calculating Disparities Using Odds Ratios
By expressing disparities in terms of odds ratios, researchers can calculate and present the risk of one group over another (similar to relative rate).12 AHRQ employs this method to calculate the "odds," for example, for uninsurance for Black and Asian adults to White adults. This method allows AHRQ to easily convey that the risk of uninsurance is 0.9 times higher for Blacks and 1.1 times higher for Asians (AHRQ, 2009c). Odds ratios should be used with caution as they can exaggerate differences and may be misleading in terms of clinical significance. For any notion of causality, notations of the absolute difference should be readily available (that is, on the probability scale).
The Reference Population
As Nerenz and Fiscella have noted, the quality measures that matter to the overall population also matter to minority populations (Fiscella, 2007; Nerenz et al., 2006). Disparities may be assessed by stratifying quality data by various population groups. Indeed, AHRQ presents data on measures for priority populations in this way in the NHDR. This method also has the benefit of being able to use the same measures to assess performance levels for both disparities and quality among populations. However, additional measures of disparity may be relevant and necessary to fully document the extent or presence of inequities.
Measuring disparities requires a comparison group. The reference group or point can be the unweighted mean of all groups, the weighted mean of the total population, the most favorable rate among population groups, or an external deliberate standard such as a Healthy People 2010 target or benchmark. Although each of these reference points can be useful, the group with the most favorable rate is often chosen as the reference point in disparities studies because it assumes that every group in the population has the potential to achieve the health of the best-off group. (In Chapter 6, the committee suggests that in the NHDR, AHRQ use benchmarks based on the best-in-class performance rate not just the highest population rate, which often is worse than the best-in-class performance rate.)
An Index of Health Care Disparities
Indices of disparities summarize average differences between groups and express the summation as a ratio of the reference rate (Harper et al., 2008). Most disparity indices measure statistically significant disparities across all populations for a given condition or disease (e.g., among all races in a given state), but do not always measure variance for a single discrete population group (Gakidou et al., 2000). Pearcy and Keppel's Index of Disparity gives equal weight to each group, even when each group represents different proportions of the population (Pearcy and Keppel, 2002). This kind of unweighted measure of disparity means that an individual in a larger population group may receive more weight than an individual in a smaller population group. To be clinically relevant to providers, a disparity index needs to measure disparities in care among discrete subpopulations and needs to give greater weight to disparities that affect greater numbers of patients (Siegel et al., 2009). Doing so captures population impact. Siegel and colleagues developed a disparities index that takes in account the quality of health care being provided to all patients, the size of the affected population, and changes over time (Siegel et al., 2009). Another benefit of using population-weighted measures is that they are able to account for changes in the distribution of the population that inevitably occur over time (Harper and Lynch, 2005).
For the purposes of the national healthcare reports, measures of equity may need to consider more than just the number of individuals affected in the entire population. For instance, a very large gap in quality of care between one relatively small subpopulation and the overall population may have significant implications for quality. A report prepared for the National Cancer Institute on measuring cancer disparities adopted a population health perspective on disparities. This perspective means that the researchers were primarily concerned with the total population burden of disparities and thus considered both absolute differences between groups and the size of the population subgroups involved (Harper and Lynch, 2005).
The methods discussed above should be considered when analyzing data relevant to assessing disparities in performance among different populations and prioritizing measure selection. Measures that reveal an equity gap, even when those same measures are equivalent in assessments of value, should be considered for prioritization as they exhibit an important attribute of the health care system where greater improvements in health care quality can be made.
The Future Directions committee has recommended improving the process for selecting performance measures for the NHQR and NHDR to make the process more transparent and quantitative. It has also recommended establishing a Technical Advisory Subcommittee for Measure Selection to advise AHRQ through the NAC. Although there are limits to applying more quantitative techniques in valuing measurement areas, they should be used whenever feasible. Their use is common in prioritization practices for resource allocation. National prioritization of measures can influence where resources are devoted to quality improvement. The potential impact of focusing quality improvement on closing the performance gaps of specific measure choices should be analyzed with care, particularly as the committee believes the national reports should be driving action rather than passively reporting on past trends.
AHRQ (Agency for Healthcare Research and Quality). 2002. Preliminary measure set for the National Healthcare Quality Report. Federal Register 67(150):53801-53802.
—. 2003a. National Healthcare Quality Report, 2003 . Rockville, MD: Agency for Healthcare Research and Quality.
—. 2003b. Preliminary measure set for home health in the National Healthcare Quality Report—request for comments. Federal Register 68(56):14240-14241.
—. 2004. National Healthcare Quality Report, 2004. Rockville, MD: Agency for Healthcare Research and Quality.
—. 2005. National Healthcare Disparities Report, 2005 . Rockville, MD: Agency for Healthcare Research and Quality.
—. 2008a. Agency for Healthcare Research and Quality (AHRQ) National Advisory Council (NAC) meeting summary, April 4, 2008.
—. 2008b. Evidence-based practice centers overview. http://www.ahrq.gov/research/findings/evidence-based-reports/overview/index.html (accessed February 10, 2016).
—. 2009a. Introductory remarks and charge to the subcommittee: Slide. Agency for Healthcare Research and Quality. Presentation to the AHRQ NAC Subcommittee on Children's Healthcare Quality Measures for Medicaid and CHIP Programs, July 22, 2009. Rockville, MD.
—. 2009b. National Advisory Council for Healthcare Research and Quality. http://www.ahrq.gov/cpi/about/organization/nac/index.html (accessed February 10, 2016).
—. 2009c. National Healthcare Disparities Report, 2009. Rockville, MD: Agency for Healthcare Research and Quality.
—. 2010. National Advisory Council for Healthcare Research and Quality. http://www.ahrq.gov/cpi/about/organization/nac/index.html (accessed February 10, 2016).
Barclay, L., and D. Lie. 2006. Most valuable clinical preventive services identified. http://cme.medscape.com/viewarticle/532983 (accessed May 13, 2010).
Barlow, G., D. Nathwani, F. Williams, S. Ogston, J. Winter, M. Jones, P. Slane, E. Myers, F. Sullivan, N. Stevens, R. Duffey, K. Lowden, and P. Davey. 2007. Reducing door-to-antibiotic time in community-acquired pneumonia: Controlled before-and-after evaluation and costeffectiveness analysis. Thorax 62(1):67-74.
Bell, C. M., D. R. Urbach, J. G. Ray, A. Bayoumi, A. B. Rosen, D. Greenberg, and P. J. Neumann. 2006. Bias in published cost-effectiveness studies: Systematic review. British Medical Journal 332(7543):699-703.
Bleichrodt, H., D. Crainich, and L. Eeckhoudt. 2008. Aversion to health inequalities and priority setting in health care. Journal of Health Economics 27(6):1594-1604.
Center for the Evaluation of Value and Risk in Health. 2009. Welcome to the CEA Registry. https://research.tufts-nemc.org/cear/Default.aspx (accessed September 25, 2009).
The Commonwealth Fund. 2002. Roadmap to reduce disparities in the quality of care for minority patients identified: National Quality Forum experts recommend 0 steps be implemented. http://www.commonwealthfund.org/Content/News/News-Releases/2002/Jun/ Roadmap-To-Reduce-Disparities-In-The-Quality-Of-Care-For-Minority-Patients-Identified.aspx (accessed February 22, 2010).
Fielding, J. E. 2009. Lessons learned in Healthy People 2010 & their application in Healthy People 2020. L.A. County Department of Public Health. Presentation at the Healthy Israel 2020 conference, April 27, 2009.
Fiscella, K. 2002. Using existing measures to monitor minority healthcare quality. In Improving healthcare quality for minority patients: Workshop proceedings, edited by the National Quality Forum. Washington, DC: National Forum for Health Care Measurement and Reporting. Pp. B1-B42.
—. 2007. Eliminating disparities in healthcare through quality improvement. In Eliminating healthcare disparities in America: Beyond the IOM report, edited by R. A. Williams. Totowa, NJ: Humana Press Inc. Pp. 141-178.
Furuno, J. P., M. L. Schweizer, J. C. McGregor, and E. N. Perencevich. 2008. Economics of infection control surveillance technology: Cost-effective or just cost? American Journal of Infection Control 36(3 Suppl 1):S12-S17.
Gakidou, E. E., C. J. L. Murray, and J. Frenk. 2000. Defining and measuring health inequality: An approach based on the distribution of health expectancy. Bulletin of the World Health Organization 78(1):42-54.
Gold, M. R., J. E. Siegel, L. B. Russell, and M. C. Weinstein. 1996. Cost-effectiveness in health and medicine. New York: Oxford University Press.
Gold, M. R., D. Stevenson, and D. G. Fryback. 2002. HALYS and QALYS and DALYS, oh my: Similarities and differences in summary measures of population health. Annual Review of Public Health 23(1):115-134.
Greenberg, D., A. B. Rosen, N. V. Olchanski, P. W. Stone, J. Nadai, and P. J. Neumann. 2004. Delays in publication of cost utility analyses conducted alongside clinical trials: Registry analysis. British Medical Journal 328(7455):1536-1537.
Guyatt, G., D. Gutterman, M. H. Baumann, D. Addrizzo-Harris, E. M. Hylek, B. Phillips, G. Raskob, S. Z. Lewis, and H. Schönemann. 2006. Grading strength of recommendations and quality of evidence in clinical guidelines: Report from an American College of Chest Physicians Task Force. Chest 129(1):174-181.
Harper, S., and J. Lynch. 2005. Methods for measuring cancer disparities: Using data relevant to Healthy People 2010 cancer-related objectives. Bethesda, MD: National Cancer Institute.
—. 2007. Selected comparisons of measures of health disparities: A review using databases relevant to Healthy People 2010 cancer related objectives. Bethesda, MD: National Cancer Institute.
Harper, S., J. Lynch, S. C. Meersman, N. Breen, W. W. Davis, and M. E. Reichman. 2008. An overview of methods for monitoring social disparities in cancer with an example using trends in lung cancer incidence by area-socioeconomic position and race-ethnicity, 1992-2004. American Journal of Epidemiology 167(8):889-899.
Harper, S., N. B. King, S. C. Meersman, M. E. Reichman, N. Breen, and J. Lynch. 2010. Implicit value judgments in the measurement of health inequalities. Milbank Quarterly 88(1):4-29.
Hauck, K., P. C. Smith, and M. Goddard. 2004. The economics of priority setting for health care: A literature review. Washington, DC: The International Bank for Reconstruction and Development/The World Bank.
HHS (U.S. Department of Health and Human Services). 2009a. Feasibility, alternatives, and cost/benefit analysis guide: Appendix B: Glossary. http://www.acf.hhs.gov/programs/cb/systems/sacwis/cbaguide/appendixb.htm (accessed January 25, 2010).
—. 2009b. Leading health indicators...touch everyone [Healthy People 2010]. http://www.healthypeople.gov/lhi/touch_fact.htm (accessed November 23, 2009).
—. 2009c. Secretarial review and publication of the annual report to Congress submitted by the contracted consensus-based entity regarding performance measurement. Federal Register 74(174):46594-46603.
Hurley, E., I. McRae, I. Bigg, L. Stackhouse, A. M. Boxall, and P. Broadhead. 2009. The Australian health care system: The potential for efficiency gains. A review of the literature. Commonwealth of Australia: National Health and Hospitals Reform Commission.
IOM (Institute of Medicine). 2001. Envisioning the National Healthcare Quality Report. Washington, DC: National Academy Press.
—. 2003. Priority areas for national action: Transforming health care quality. Washington, DC: The National Academies Press.
—. 2006. Valuing health for regulatory cost-effectiveness analysis. Washington, DC: The National Academies Press.
—. 2008. Knowing what works in health care: A roadmap for the nation. Washington, DC: The National Academies Press.
Keppel, K., E. Pamuk, J. Lynch, O. Carter-Pokras, I. Kim, V. Mays, J. Pearcy, V. Schoenbach, and J. S. Weissman. 2005. Methodological issues in measuring health disparities. Vital and Health Statistics 2(141):1-22.
Kilbourne, A. M., G. Switzer, K. Hyman, M. Crowley-Matoka, and M. J. Fine. 2006. Advancing health disparities research within the health care system: A conceptual framework. American Journal of Public Health 96(12):2113-2121.
Langley, G. J., K. M. Nolan, C. L. Norman, L. P. Provost, and T. W. Nolan. 1996. The improvement guide: A practical approach to enhancing organizational performance. New York: Jossey-Bass Business and Management Series.
Maciosek, M. V., A. B. Coffield, N. M. Edwards, T. J. Flottemesch, M. J. Goodman, and L. I. Solberg. 2006a. Priorities among effective clinical preventive services: Results of a systematic review and analysis. American Journal of Preventive Medicine 31(1):52-61.
Maciosek, M. V., N. M. Edwards, A. B. Coffield, T. J. Flottemesch, W. W. Nelson, M. J. Goodman, and L. I. Solberg. 2006b. Priorities among effective clinical preventive services: Methods. American Journal of Preventive Medicine 31(1):90-96.
Maciosek, M. V., A. B. Coffield, N. M. Edwards, T. J. Flottemesch, and L. I. Solberg. 2009. Prioritizing clinical preventive services: A review and framework with implications for community preventive services. Annual Review of Public Health 30(1):341-355.
National Quality Measures Clearinghouse™. 2009a. U.S. Department of Health & Human services (HHS) measure inventory. http://www.qualitymeasures.ahrq.gov/index.aspx (accessed November 23, 2009).
—. 2009b. Welcome! http://www.qualitymeasures.ahrq.gov/ (accessed November 23, 2009).
NCVHS (National Committee on Vital and Health Statistics). 2002. National Committee on Vital and Health Statistics (NCVHS), Subcommittee on Populations—Working Group on Quality. http://www.ncvhs.hhs.gov/020725fr.htm (accessed November 23, 2009).
Nerenz, D. R. 2002. Quality of care measures of special significance to minority populations. In Improving healthcare quality for minority patients, edited by the National Quality Forum. Washington, DC: National Quality Forum. Pp. C1-C23.
Nerenz, D. R., K. A. Hunt, and J. J. Escarce. 2006. Health care organizations' use of data on race/ethnicity to address disparities in health care. Health Services Research 41(4p1):1444-1450.
Neumann, P. J., J. A. Palmer, N. Daniels, K. Quigley, M. R. Gold, and S. Chao. 2008. A strategic plan for integrating cost-effectiveness analysis into the US healthcare system. American Journal of Managed Care 14(4):185-188.
NIHR (National Institute for Health Research) Centre for Reviews and Dissemination. 2009. CRD databases. http://www.crd.york.ac.uk/crdweb/ (accessed January 7, 2010).
NQF (National Quality Forum). 2009a. Measure evaluation criteria. http://www.qualityforum.org/uploadedFiles/Quality_Forum/Measuring_Performance/Consensus_Development_Process%E2%80%99s_Principle/EvalCriteria2008-08-28Final.pdf?n=4701 [Plugin Software Help] (accessed February 22, 2010).
—. 2009b. NQF-endorsed standards. http://www.qualityforum.org/Measures_List.aspx (accessed October 28, 2009).
Romano, P. S. 2009. Quality performance measurement in California: Findings and recommendations to the California Office of the Patient Advocate. Center for Healthcare Policy and Research, University of California, Davis. Presentation to the IOM Committee on Future Directions for the National Healthcare Quality and Disparities Reports, March 11, 2009. Newport Beach, CA. PowerPoint® Presentation.
Rosvall, M., G. Engstrom, G. Berglund, and B. Hedblad. 2008. C-reactive protein, established risk factors and social inequalities in cardiovascular disease—the significance of absolute versus relative measures of disease. BMC Public Health 8(189):1-10.
Russell, L. B., J. E. Siegel, N. Daniels, M. R. Gold, B. R. Luce, and J. S. Mandelblatt. 1996. Cost-effectiveness analysis as a guide to resource allocation in health: Roles and limitations. In Cost-effectiveness in health and medicine, edited by J. E. Siegel, L. B. Russell, and M. C. Weinstein. New York: Oxford University Press. Pp. 3-24.
Secretary's Advisory Committee on National Health Promotion and Disease Prevention Objectives for 2020. 2008a. Phase I report: Recommendations for the framework and format of Healthy People 2010. Appendix . Washington, DC: U.S. Department of Health and Human Services.
—. 2008b. Phase I report: Recommendations for the framework and format of Healthy People 2010. Appendix . Washington, DC: U.S. Department of Health and Human Services.
Siu, A. L., E. A. McGlynn, H. Morgenstern, M. H. Beers, D. M. Carlisle, E. B. Keeler, J. Beloff, K. Curtin, J. Leaning, B. C. Perry, H. P. Selker, W. Weiswasser, A. Wiesenthal, and R. H. Brook. 1992. Choosing quality of care measures based on the expected impact of improved care on health. Health Services Research 27(5):619-650.
Tengs, T. O., and J. D. Graham. 1996. The opportunity cost of haphazard social investments in life-saving. In Risks, costs, and li es sa ed: Getting better results from regulation, edited by R. W. Hahn. New York: Oxford University Press. Pp. 167-182.
Whitlock, E. P., S. A. Lopez, S. Chang, M. Helfand, M. Eder, and N. Floyd. 2010. AHRQ series paper 3: Identifying, selecting, and refining topics for comparative effectiveness systematic reviews: AHRQ and the effective health-care program. Journal of Clinical Epidemiology 63(5):491-501.
12 "Odds ratios are a common measure of the size of an effect and may be reported in case-control studies, cohort studies, or clinical trials. Increasingly, they are also used to report the findings from systematic reviews and meta-analyses. Odds ratios are hard to comprehend directly and are usually interpreted as being equivalent to the relative risk. Unfortunately, there is a recognized problem that odds ratios do not approximate well to the relative risk when the initial risk (that is, the prevalence of the outcome of interest) is high. Thus there is a danger that if odds ratios are interpreted as though they were relative risks then they may mislead" (Davies et al., 1998, p. 989).