National Healthcare Quality and Disparities Report
Introduction and Methods
This document describes the contents of the National Healthcare Quality and Disparities Report and the methods used to compile and analyze data.
Introduction and Methods
Background on the National Healthcare Quality and Disparities Report and Related Chartbooks
Changes in the 2014 Quality and Disparities Report
National Quality Strategy
Organization of the 2014 National Healthcare Quality and Disparities Report and Related Chartbooks
Methods of the National Healthcare Quality and Disparities Report and Related Chartbooks
The National Healthcare Quality and Disparities Report (QDR) is the product of collaboration among agencies across the Department of Health and Human Services (HHS). Many individuals guided and contributed to this report. Without their magnanimous support, the report would not have been possible.
Specifically, we thank:
Primary AHRQ Staff: Richard Kronick, Jeff Brady, Amy Helwig, Ernest Moy, Karen Chaves, Veronica Soileau, Vera Rosenthal, Elizabeth Bishop, Darryl Gray, Nancy Wilson, and Doreen Bonnett.
HHS Interagency Workgroup for the NHQR/NHDR: Girma Alemu (HRSA), Chisara N. Asomugha (CMS), Kirsten Beronio (ASPE), Nancy Breen (NCI), Miya Cain (ACF), Victoria Cargill (NIH), Steven Clauser (NCI), Wayne Duffus (CDC), Olinda Gonzalez (SAMHSA), Kirk Greenway (IHS), Chris Haffer (CMS-OMH) , Linda Harlan (NCI), Rebecca Hines (CDC-NCHS), Edwin Huff (CMS), Deloris Hunter (NIH), Sonja Hutchins (CDC), Ruth Katz (ASPE), Tanya Telfair LeBlanc (CDC), Shari Ling (CMS), Darlene Marcoe (ACF), Tracy Matthews (HRSA), Karen McDonnell(CMS), Curt Mueller (HRSA), Karen Nakano (CMS), Iran Naqvi (HRSA), Ann Page (ASPE), Kimberly Proctor (CMS-OMH), D.E.B. Potter (ASPE), Asel Ryskulova (CDC-NCHS), Adelle Simmons (ASPE), Alan Simon (CDC-NCHS), Marsha Smith (CMS), Caroline Taplin (ASPE), Emmanuel Taylor (NCI), Sayeedha Uddin (CDC-NCHS), Nadarajen Vydelingum (NIH), Chastity Walker (CDC), Barbara Wells (NHLBI), Valerie Welsh (OASH-OMH), and Tia Zeno (ASPE).
AHRQ QDR Team: Roxanne Andrews (CDOM), Barbara Barton (SSS), Doreen Bonnett (OCKT), Cecilia Casale (OEREP), Karen Chaves (CQuIPS), Frances Chevarley (CFACT), Beth Collins-Sharp (OEREP), Denise Dougherty (OEREP), Noel Eldridge (CQuIPS), Zhengyi Fang (SSS), Erin Grace (CQuIPS), Darryl Gray (CQuIPS), Kevin Heslin (CDOM), Anika Hines (Truven), Leif Karell (SSS), Anil Koninty (SSS), Eric Lui (CQuIPS), Atlang Mompe (SSS), Ernest Moy (CQuIPS), Janet Pagán-Sutton (SSS), Susan Raetzman (Truven), Vera Rosenthal (CQuIPS), Veronica Soileau (CQuIPS), Lily Trofimovich (SSS), Yi Wang (SSS), Nancy Wilson (CQuIPS), Sean Yin (SSS), and Chava Zibman (CFACT).
HHS Data Experts: Clarice Brown (CDC-NCHS), Anjani Chandra (CDC-NCHS), Laura Cheever (HRSA), Frances Chevarley (AHRQ), Robin Cohen (CDC-NCHS), Steven Cohen (AHRQ), Rupali Doshi (HRSA), Paul Eggers (NIH), John Fleishman (AHRQ), Elizabeth Goldstein (CMS), Beth Han (SAMHSA), Haylea Hannah (CDC), Kimberly Lochner (CMS), Marlene Matosky (HRSA), William Mosher (CDC-NCHS), Cynthia Ogden (CDC-NCHS), Robert Pratt (CDC), Asel Ryskulova (CDC-NCHS), Alek Sripipatana (HRSA), Alan Simon (CDC-NCHS), and Xiaohong (Julia) Zhu (HRSA).
Other Data Experts: Dana Auden (Oklahoma Foundation for Medical Quality [OFMQ]), Timothy Chrusciel (OFMQ), Mark Cohen (American College of Surgeons National Surgical Quality Improvement Program [ACS NSQUIP]), Sheila Eckenrode (MPSMS-Qualidigm), Beth Forrest (USRDS), Selena Gonzalez (CDC-HIV), David Grant (UCLA), Michael Halpern (American Cancer Society), Matthew Haskins (National Hospice and Palliative Care Organization), Clifford Ko (ACS NSQIP), Allen Ma (OFMQ), Richard Moser (NCI), Wato Nsa (OFMQ), Nicholas Okpokho (OFMQ), Robin Padilla (University of Michigan), Bryan Palis (American College of Surgeons, NCBD), Pennsylvania Patient Safety Authority, Royce Park (UCLA), William Ross (Fu Associates), Scott Stewart (OFMQ), VA National Center for Patient Safety, Yolanta Vucic (OFMQ), Reda Wilson (CDC-ONDIEH-NCCDPHP), Richard Wolitski (CDC-HIV), and Claudia Wright (OFMQ).
Other AHRQ Contributors: Cindy Brach, Monique Cohen, James Kirby, Biff LeVee, Iris Mabry-Hernandez, Edwin Lomotan, Gerri Michael-Dyer, Karen Migdail, Shyam Misra, Pamela Owens, Mamatha Pancholi, Larry Patton, Wendy Perry, Richard Ricciardi, Mary Rolston, and Randie Siegel.
Data Support Contractors: Booz Allen Hamilton, Fu Associates, Social & Scientific Systems, Truven Health Analytics, and Westat.
Each year since 2003, the Agency for Healthcare Research and Quality (AHRQ) has reported on progress and opportunities for improving health care quality and reducing health care disparities. As mandated by the U.S. Congress, the reports focus on "national trends in the quality of health care provided to the American people" (42 U.S.C. 299b-2(b)(2)) and "prevailing disparities in health care delivery as it relates to racial factors and socioeconomic factors in priority populations" (42 U.S.C. 299a-1(a)(6)).
This year, AHRQ combined the National Healthcare Quality Report and National Healthcare Disparities Report into a single document, the National Healthcare Quality and Disparities Report (QDR).The report is produced with the support of an HHS Interagency Work Group and guided by input from AHRQ's National Advisory Council and the Institute of Medicine.
As in previous years, the 2014 report tracks more than 250 health care process, outcome, and access measures, covering a wide variety of conditions and settings. Data years vary across measures; most trend analyses include data points from 2000-2002 to 2011-2012. It is important to note that the report provides a snapshot of health care prior to implementation of most of the health insurance expansions included in the Affordable Care Act and serves as a baseline to track progress in upcoming years. An exception is rates of uninsurance, which we are able to track through the first half of 2014.
We substantially reorganized the 2014 QDR in response to feedback from readers to make it more usable while continuing to make available the data and analyses included in previous reports. We reviewed suggestions from AHRQ leadership, the Interagency Work Group, AHRQ's National Advisory Council, National Quality Strategy (NQS) implementation staff, and the Institute of Medicine related to focus, content, format, and dissemination.
The result of this input was a number of design principles that maximize value to potential users of this 2014 QDR:
- Analyses of change over time to assess performance and to distinguish areas that are improving from those that may benefit from additional attention are emphasized. While many groups provide report cards on health and health care in the United States at a point in time, QDR data support trends across a broad array of quality measures for a wide variety of health care services and settings.
- Integration with the NQS (http://www.ahrq.gov/workingforquality/) is enhanced. The NQS identifies quality improvement priorities for the Nation and reports on promising initiatives to make health care better. The 2014 QDR is reorganized around tracking progress for each of the six NQS priorities. It also tracks access to health care, which is not part of the NQS framework but cuts across all six priorities.
- The importance of simultaneously considering both performance and disparities is reinforced. Instead of two separate reports, the 2014 QDR provides a more complete and integrated assessment of access to and quality of health care. As specified in the Healthcare Research and Quality Act of 1999, it continues to focus on disparities related to race, ethnicity, and socioeconomic status. The new QDR also incorporates analyses of other priority populations, including women, children, older adults, people with disabilities and at the end of life, and residents of rural areas and inner cities.
- Electronic dissemination is expanded, recognizing that the vast majority of users of the QDR prefer the Web site to paper documents. This allows the paper document to be shortened and to focus on summarizing information for policymakers. Analyses of individual measures will be disseminated in a series of chartbooks on the Web (http://www.ahrq.gov/research/findings/nhqrdr/2014chartbooks/). Additional information on each priority population will also be posted as chartbooks on the Web.
Mandated by the Affordable Care Act, the NQS was developed through a transparent and collaborative process with input from a range of stakeholders. More than 300 groups, organizations, and individuals, representing all sectors of the health care industry and the general public, provided comments. Based on this input, the NQS established a set of three overarching aims that builds on the Institute for Healthcare Improvement's Triple Aim®, supported by six priorities that address the most common health concerns that Americans face. To align with NQS, stakeholders can use nine levers to align their core business or organizational functions to drive improvement on the aims and priorities.
The three aims are used to guide and assess local, State, and national efforts to improve health and the quality of health care:
- Better Care: Improve overall quality, by making health care more patient centered, reliable, accessible, and safe.
- Healthy People/Healthy Communities: Improve the health of the U.S. population by supporting proven interventions to address behavioral, social, and environmental determinants of health in addition to delivering higher quality care.
- Affordable Care: Reduce the cost of quality health care for individuals, families, employers, and governments.
To advance these aims, the NQS focuses on six priorities.
- Patient Safety: Making care safer by reducing harm caused in the delivery of care.
- Person-Centered Care: Ensuring that each person and family is engaged as partners in their care.
- Care Coordination: Promoting effective communication and coordination of care.
- Effective Treatment: Promoting the most effective prevention and treatment practices for the leading causes of mortality, starting with cardiovascular disease.
- Healthy Living: Working with communities to promote wide use of best practices to enable healthy living.
- Care Affordability: Making quality care more affordable for individuals, families, employers, and governments by developing and spreading new health care delivery models.
Each of the nine NQS levers represents a core business function, resource, or action that stakeholders can use to align to the strategy:
- Measurement and Feedback.
- Public Reporting.
- Learning and Technical Assistance.
- Certification, Accreditation, and Regulation.
- Consumer Incentives and Benefit Designs.
- Health Information Technology.
- Innovation and Diffusion.
- Workforce Development.
Each year, a progress report is produced. The 2014 NQS progress report (http://www.ahrq.gov/workingforquality/reports/annual-reports/nqs2014annl...) features Priorities in Action, which highlights promising and transformative quality improvement programs and spotlights organizations that have adopted the NQS as a framework for quality improvement. To complement this activity, the 2014 QDR begins tracking progress along each of the six NQS priorities. An illustrative measure tracked by the QDR is presented for each priority. Information on trends and disparities is also shown for each priority with sufficient data to summarize. Tracking of all QDR measures is included in a series of statistical chartbooks that will be posted online after the release of the 2014 QDR.
This Introduction and Methods document provides background on the QDR and modifications that have occurred over time. This document includes an overview of the methods used to generate estimates, measure trends, and examine disparities.
The 2014 report and chartbooks are organized around the concept of access to care, quality of care, disparities in care, and NQS priorities. Summaries of the status of access, quality, disparities, and NQS priorities can be found in the report. Details for individual measures are found in the chartbooks.
The chartbooks will be organized in this manner:
- Access to Health Care presents measures that cut across several priority areas and includes measures of health insurance, usual source of care, timeliness of care, and infrastructure to provide health care to minority and low-income populations.
- Patient Safety tracks safety within a variety of health care settings, including hospitals, nursing homes, home health settings, and ambulatory care settings.
- Person- and Family-Centered Care examines individual experiences with care in an office or clinic setting, during a hospital stay, and while receiving home health care. It tracks measures of perceptions of communication with providers and satisfaction with the provider-patient relationship.
- Care Coordination presents data to assess the performance of the U.S. health care system in coordinating care across providers and services. It includes measures of transitions between health care settings and health information technologies that help to coordinate care.
- Effective Treatment is organized around care for the leading causes of mortality and morbidity in the United States: cancer, cardiovascular disease, chronic kidney disease, diabetes, HIV and AIDS, mental health and substance abuse, musculoskeletal diseases, and respiratory diseases.
- Healthy Living examines health care services that typically cut across clinical conditions: maternal and child health, lifestyle modification, clinical preventive services, functional status preservation and rehabilitation, and supportive and palliative care.
- Care Affordability discusses costs of health care and tracks measures of financial barriers to care as well as misuse of health care services.
- Priority Populations summarizes quality and disparities in care for populations at elevated risk for receiving poor health care, including racial and ethnic minorities; low-income populations; children; older adults; residents of rural areas; lesbian, gay, bisexual, and transgender populations; and individuals with disabilities, multiple chronic conditions, or special health care needs. Because of the large number of priority populations covered in the report, it is anticipated that sections of this chartbook will be developed over several report cycles.
- Appendixes consist of:
- Data Sources, which provides information about each database analyzed for the reports, including data type, sample design, and primary content.
- Measure Specifications, which provides information about how measures are generated and analyzed for the reports. Measures highlighted in the report are described, as well as other measures that were examined but not included in the text of the report.
- Detailed Methods, which provides detailed methodological and statistical information about selected databases analyzed for the report.
Access to Health Care
- Purpose. To assess access to care for the overall U.S. population and for priority populations, to track changes in access to care over time, and to identify aspects of access to care that are improving and aspects that are not improving.
- Approach. Factors that facilitate accessing health care, including having health insurance and a usual source of care, have been tracked since the first reports. With the reorganizations of the reports around NQS priorities this year, measures of timeliness of care (previously in a section on Timeliness) and infrastructure to provide health care to minority and low-income populations (previously in a section on Health System Infrastructure) were added to the Access measure set.
- Summaries of Access. At times, the reports will present summary information across a panel of access.
Quality of Health Care
- Purpose. To assess quality care for the overall U.S. population and for priority populations, to track changes in quality of care over time, and to identify aspects of quality of care that are improving and aspects that are not improving.
- Initial Approach. The selection of quality measures to include in the first reports involved several steps:
- The Institute of Medicine (IOM) provided criteria for the selection of quality measures: overall importance of the aspects of quality being measured, scientific soundness of the measures, and feasibility of the measures. It also provided criteria for the measure set as a whole: balance, comprehensiveness, and robustness.
- Calls for Measures were issued by IOM and AHRQ and yielded hundreds of measures submitted by private and governmental organizations.
- A Federal Measures Workgroup was convened to apply the IOM criteria to the measures submitted for consideration.
- A Preliminary Measure Set was published in the Federal Register for public comment; additional comments were obtained through a hearing organized by the National Committee on Vital and Health Statistics.
- This yielded an Initial Measure Set that included 147 measures from two dozen data sources.
- Types of Quality Measures. Most measures tracked in the reports reflect processes of care, outcomes of care, and patient perceptions of care. A few measures track structural elements that are important for quality health care.
- Processes of Care. These measures generally represent percentages of people receiving care that they need or percentages of people receiving care that they should not receive. Measures are specified so that everyone in the denominator needs the service and optimal care equals 100%. These measures are generally not adjusted for age and sex since need is captured in the specification of the denominator.
- Outcomes of Care. These measures generally represent rates of adverse events or deaths. These measures are generally adjusted for age and sex; adjustment is also done for comorbidities when possible. Because death rates often reflect factors other than health care, only death rates with moderate ties to processes of care are tracked. For example, colorectal cancer death rates are tracked because they are related to rates of colorectal cancer screening.
- Patient Perceptions of Care. These measures generally represent percentages of people who perceived problems with aspects of their care
- Infrastructure. These measures generally represent the availability of different health care resources. They are often difficult to interpret; are there fewer resources because needs are not met or because the resources are not needed? Hence, they are only included when measures of processes, outcomes, and patient perceptions are not available. They are generally not included in summaries of measures.
- Refinement of the Measure Set. Since the first reports, the measure set has been reviewed each year and changes made as needed. All changes are approved by the HHS Interagency Work Group that supports the reports.
- Additions have been made to the measure set as new domains of quality, data, and measures have become available. For example, Care Coordination and Care Affordability were not recognized quality domains when the reports started, and measures of these domains were identified and added after they were recognized.
- Deletions have been made when data collection for measures ceased or when new scientific information indicated that a measure did not represent high-quality care. In addition, process measures that achieve overall performance levels exceeding 95% are not tracked in the reports. The success of these measures limits their utility for tracking improvement over time. Because these measures cannot improve to a significant degree, including them in the measure set creates a ceiling effect that may dampen quantification of rates of change over time. Data on retired measures continue to be collected and these measures will be added back to the reports if their performance falls below 95%.
- Modifications have been made when clinical recommendations change. For example, clinical recommendations often set new target levels or recommended frequencies for specific services.
- Summaries of Quality. At times, the reports will present summary information across a panel of quality measures. This panel includes measures that are widely considered important for health care quality and include measures of processes, outcomes, and patient perceptions. The panel excludes measures with less clear interpretation, typically measures of infrastructure and costs.
Overview of Data
The data included in the reports were determined by the measures chosen for tracking. Dozens of data sources are used in the reports to provide a comprehensive assessment of access to health care and quality of health care in the United States. Almost all data are national in geographic scope in order to provide estimates for the Nation. Most are nationally representative or cover the entire U.S. population.
Different types of data are used to provide complementary perspectives of health care and include patient surveys, provider surveys, administrative data from facilities, medical records, registries, surveillance systems, and vital statistics. Settings of care covered include ambulatory care, health centers, emergency departments, hospitals, nursing homes, hospices, and home health.
Only data sources that are regularly reported in the reports are listed below. Not included on the list are sources that do not collect data on a regular basis; such data are presented intermittently in the reports when they address topics or populations not well covered by regular data collections.
Federal Sources of Data
Most data tracked in the reports come from Federal sources because Federal data collections are typically national in scope and annual. Databases from Federal agencies used in the reports include the following:
- Healthcare Cost and Utilization Project (HCUP):
- National (Nationwide) Inpatient Sample (NIS).
- Nationwide Emergency Department Sample (NEDS).
- State Inpatient Databases (SID).
- State Emergency Department Databases (SEDD).
- Disparities Analytic File.
- Hospital Survey on Patient Safety Culture.
- Medical Office Survey on Patient Safety Culture.
- National CAHPS® Benchmarking Database (NCBD)—Health Plan Survey Database.
- Healthcare Cost and Utilization Project (HCUP):
- Centers for Disease Control and Prevention (CDC):
- Behavioral Risk Factor Surveillance System (BRFSS).
- National Ambulatory Medical Care Survey (NAMCS).
- National Health and Nutrition Examination Survey (NHANES).
- National Health Interview Survey (NHIS).
- National HIV/AIDS Surveillance System.
- National Hospital Ambulatory Medical Care Survey (NHAMCS).
- National Hospital Discharge Survey (NHDS).
- National Immunization Survey (NIS).
- National Program of Cancer Registries (NPCR).
- National Tuberculosis Surveillance System (NTBSS).
- National Vital Statistics System—Linked Birth and Infant Death Data (NVSS-I).
- National Vital Statistics System—Mortality (NVSS-M).
- National Vital Statistics System—Natality (NVSS-N).
- Centers for Medicare & Medicaid Services (CMS):
- Home Health Outcome and Assessment Information Set (OASIS).
- Medicare Enrollment and Claims (or Medicare Data and Chronic Conditions).
- Medicare Quality Improvement Organization (QIO) Program.
- Nursing Home Minimum Data Set (MDS).
- University of Michigan Kidney Epidemiology and Cost Center.
- Health Resources and Services Administration (HRSA):
- National Survey of Children's Health (NSCH).
- Uniform Data System (UDS).
- Indian Health Service (IHS):
- IHS National Data Warehouse (NDW).
- National Institutes of Health (NIH):
- United States Renal Data System (USRDS).
- Substance Abuse and Mental Health Services Administration (SAMHSA):
- National Survey on Drug Use and Health (NSDUH).
- Substance Abuse Treatment Episode Data Set (TEDS).
- U.S. Census Bureau:
- American Community Survey (ACS).
- Multiagency Data Sources:
- HIV Research Network (HIVRN).
- Medical Expenditure Panel Survey (MEPS).
- Medicare Patient Safety Monitoring System (MPSMS)
Professional Organization Sources of Data
Federal data sources are supplemented by data from other organizations that collect national data. Nonfederal databases used in the reports include:
- American Hospital Association (AHA) Annual Survey Information Technology Supplement, which provides information on adoption of health information technologies by hospitals.
- National Hospice and Palliative Care Organization (NHPCO) Family Evaluation of Hospice Care Survey (FEHCS), which provides information on access to and quality of hospice care.
- Commission on Cancer and American Cancer Society (ACS) National Cancer Data Base (NCDB), which provides information on quality of cancer treatment.
Some aspects of health care quality and disparities are not covered well by national data, both Federal and nonfederal, but are covered by subnational data. Such unique subnational data collections are included in the report to illustrate the potential value of improving national data collection. Subnational data used in the reports include:
- California Health Interview Survey (CHIS), which allows examination of health care received by Californians who identify as:
- Lesbian, gay, bisexual, or transgender.
- Hispanic, including people with Mexican, Puerto Rican, or Cuban ancestry.
- Asian, including people with Asian Indian, Chinese, Filipino, Japanese, Korean, or Vietnamese ancestry.
- People who speak languages other than English at home.
- Hawaii State Inpatient Database and Hawaii Health Survey, which allows examination of health care received by Hawaiians who identify as:
- Native Hawaiian, Samoan, or Other Pacific Islander.
- Asian, including people with Chinese, Filipino, Japanese, or Korean ancestry.
Overall U.S. Population
- Purpose. A key function of the QDR and related chartbooks is to assess access to health care and quality of health for the overall U.S. population.
- Approach. National data are used as collected without additional exclusions. For each database, the included population is described in the Data Sources appendix. Common population limitations include the following:
- Most Federal health surveys are limited to the civilian noninstitutionalized population and do not include people on active duty in the military or who reside in nursing homes or penal or mental institutions.
- Many facility data collections do not include Federal facilities run by the Departments of Defense or Veterans Affairs or by IHS.
- Purpose. Another key function of the QDR and related chartbooks is to assess access to health care and quality of health for select populations defined by age, sex, race, ethnicity, income, education, health insurance, activity limitations, and geographic location.
- Approach. To the extent supported by data collection, definitions of priority populations are standardized across different data sources. Typical priority population definitions available in multiple databases include:
- Age: 0-17, 18-44, 45-64, and 65 and over.
- Sex: Male and female.
- Race: White, Black, Asian, Native Hawaiian or Other Pacific Islander, American Indian or Alaska Native, and more than one race.i
- Ethnicity: Hispanic and non-Hispanic.ii
- Income: Poor, low income, middle income, and high income.iii
- Education: People with less than a high school education,iv high school graduates, and people with any college.
- Health insurance, ages 0-64: Any private insurance, public insurancev only, and no insurance.
- Health insurance, age 65 and over: Medicare and any private insurance, Medicare and other public insurance, and Medicare only.
- Disabilities: Basic activity limitations include problems with mobility, self-care (activities of daily living), domestic life (instrumental activities of daily living), and activities that depend on sensory functioning (limited to people who are blind or deaf); complex activity limitations include limitations experienced in work and in community, social, and civic life. For the purpose of the QDR, adults with disabilities are those with physical, sensory, and/or mental health conditions that can be associated with a decrease in functioning in such day-to-day activities as bathing, walking, doing everyday chores, and engaging in work or social activities. The paired measure is intended to be consistent with statutory definitions of disability, such as the first criterion of the 1990 Americans With Disabilities Act (ADA) and other Federal program definitions of disability.
- Children with special health care needs (CSHCN): Children ages 0-17 with activity limitations or with the need or use of more health care or other services than is usual for most children of the same age. Question sequencesvi are asked about the following five health consequences: the need or use of medicines prescribed by a doctor; the need or use of more medical care, mental health care, or education services than is usual for most children; limitations in or inability to do things most children can do; the need or use of special therapy such as physical, occupational, or speech therapy; and the need or use of treatment or counseling for emotional, developmental, or behavioral problems. Children with responses to at least one of the five health consequences were identified as having a special health care need.
- Geographic location: Large central metropolitan, large fringe metropolitan, medium metropolitan, small metropolitan, micropolitan, and noncore areas based on the National Center for Health Statistics Urban-Rural Classification Scheme.
- Special Analyses. Other important groups have been more difficult to identify in health care data:
- Beginning in the 2011 reports, information on lesbian, gay, bisexual, and transgender people has been included, but few databases support these analyses.
- Beginning in the 2012 reports, contrasts by granular racial subgroups have been included. Information on populations identified as Mexican, Puerto Rican, Cuban, other Hispanic/Latino/Spanish origin, Asian Indian, Chinese, Filipino, Japanese, Korean, Vietnamese, other Asian, Native Hawaiian, Guamanian or Chamorro, Samoan, and other Pacific Islander have been sought, but no health care database that identifies all of these subgroups has been found.
- Beginning in the 2013 reports, analyses by number of multiple chronic conditions have been included, but databases differ in the chronic conditions that can be identified. Improving measurement and data for these groups is critical to understand the reasons they cannot access high-quality health care and to develop effective interventions to help them overcome these barriers.
- While the reports do not address social determinants directly, analyses of disparities in health care related to family income and education are at the core of the QDR and demonstrate the importance of socioeconomic status on quality of and access to health care.
Trends in Quality
- Purpose. To assess change over time considering both magnitude of change and statistical significance. Magnitude of change was considered important because large databases could have trends that were statistically significant but not large enough to be clinically meaningful.
- Approach. Weighted log-linear regression.
- Data requirement. Estimates for at least four time points between 2000 and 2014; fewer than four time points was deemed insufficient to calculate slopes of regression lines.
- Data preparation.
- Estimates were framed negatively.
- Estimates were divided by earliest estimate so that earliest indexed estimate equaled one and subsequent indexed estimates were relative to the earliest estimate.
- The natural logarithm of each indexed estimate was calculated.
- Model. ln(M) = β0 + β1Y, where ln(M) is the natural logarithm of the measure value, β0 is the intercept or constant, and β1 is the coefficient corresponding to year Y.
- Weight. w = (M2/v), where M2 is the square of the measure value and v is the variance.
- Improving = Average annual percentage change >1% per year in a favorable direction and p <0.10.vii
- Worsening = Average annual percentage change >1% per year in an unfavorable direction and p <0.10.
- No Change = Average annual percentage change ≤1% per year or p ≥0.10.
- Summaries of trends. Trends across panels of measures can be summarized in a variety of ways. The average annual percentage change of each measure is calculated and summary over the panel of measures presented as:
- The median value.
- The distribution of average annual percentage change.
- A stacked bar chart showing the percentage of measures that are Improving, Worsening, or No Change.
- Measures with extreme trends. To help identify measures that are changing the most quickly, measures are sorted by average annual percentage change.
- Improving quickly = Average annual percentage change >10% per year in a favorable direction and p <0.10.
- Purpose. To define a high level of performance that has been attained to help readers understand national and State performance and to serve as an achievable quality improvement goal.
- Approach. Average of best performing States.
- Data requirement. Estimates for at least 30 States. Note that only about half of QDR measures meet this requirement.
- Calculation. Average of estimates from top 10% of States (e.g., average of top five States if estimates available on all 50 States and DC).
- Updating. Most benchmarks are based on 2008 data, which were the most recent data available when the QDR began this approach. If overall performance on a measure reached the benchmark, a new benchmark was set.
- Figures. When available, benchmarks are shown as dashed red lines on figures.
- Time to benchmark. When data support analysis of trends (see above), time to benchmark is calculated to quantify the distance from the benchmark. The average annual percentage change is used to extrapolate forward to the time when the benchmark will be achieved. Time to benchmark is not reported if:
- Average annual percentage change is less than 1% (interpreted as no change).
- Time to benchmark of all groups is estimated at 25 or more years.
- Trends show movement away from the benchmark.
- Direction of trend changes over time.
Size of Disparities Between Two Subpopulations
- Purpose. To assess whether access or quality differs between two subpopulations. Comparisons are typically made between a priority population group and a reference group within a population characteristic (e.g., Blacks vs. Whites within the Race characteristic). The largest subgroup is typically used as the reference group.
- Approach. Two criteria are applied to determine whether the difference between two groups is meaningful:
- The difference between the two groups must be statistically significant with p <0.05 on a two-tailed test.
- The relative difference between the priority population group and the reference group must have an absolute value of at least 10% when framed positively or negatively ([p1-p2]/p2 >0.1 OR [(1-p1)-(1-p2)]/(1-p2) >0.1).
- Better = Priority population estimate more favorable than reference group estimate by at least 10% and with p <0.05.
- Worse = Priority population estimate less favorable than reference group estimate by at least 10% and with p <0.05.
- Same = Priority population and reference group estimates differ by 10% or less or p ≥0.05.
- Summaries of disparities. Disparities across panels of measures are usually summarized as stacked bar charts showing the percentage of measures that are Better, Worse, or Same for priority populations compared with a reference group.
Trends in Disparities
- Purpose. To observe whether difference in access or quality between two subpopulations has changed over time. Comparisons are typically made between a priority population group and a reference group within a population characteristic (e.g., Blacks vs. Whites within the Race characteristic).
- Approach. Weighted linear regression.
- Data requirement. Estimates for at least four time points between 2000 and 2014 for both the priority population and reference group; fewer than four time points was deemed insufficient to calculate slopes of regression lines.
- Data preparation.
- Estimates were framed negatively.
- For each group, estimates were divided by earliest estimate so that earliest indexed estimate equaled one and subsequent indexed estimates were relative to the earliest estimate.
- Model. M = β0 + β1Y, where M is the value of the measure, β0 is the intercept or constant, and β1 is the coefficient corresponding to year Y.
- Weight. w = (1/v), where v is the variance.
- Improving = Average annual change of the priority population and reference group differs by more than 1% per year in a favorable direction and p <0.10.
- Worsening = Average annual change of the priority population and reference group differs by more than 1% per year in an unfavorable direction and p <0.10.
- No Change = Average annual change of the priority population and reference group differs by 1% per year or less or p ≥0.10.
- Summaries of trends in disparities. Trends in disparities across panels of measures are usually summarized as stacked bar charts showing the percentage of measures that are Improving, Worsening, or No Change for priority populations compared with a reference group.
- Measures with extreme trends in disparities. To help identify measures with disparities that are changing the most quickly for each priority population, measures are sorted by the difference in average annual change between the priority population and reference group.
- Disparities eliminated = Disparity improving and priority population estimates reached or surpassed reference group estimate.
- Purpose. For ease of interpretation, most analyses presented in the reports focus on one characteristic at a time. However, on occasion, bivariate and multivariate analyses are presented to highlight specific characteristics or interactions of characteristics.
- Stratified analyses. Whenever supported by databases, estimates of race and ethnicity stratified by income, education, and health insurance and of income and education stratified by race and ethnicity are collected. These data are typically shown when patterns of racial or ethnic disparities differ for different socioeconomic groups.
- Regressions. Logistic or linear regression models are sometimes created for specific measures to quantify the unique contribution of specific characteristics to disparities. In examining the relationship of race and ethnicity with a measure, for example, multivariate regression analyses are sometimes performed to control for differences in the distribution of income, education, insurance, age, gender, and geographic location.
Results are typically presented as adjusted percentages, which quantify the magnitude of disparities after controlling for a number of confounding factors.
- Purpose. For ease of reporting, some shorthand is used in presenting results. Unless otherwise specified:
- Results presented in text or bullets meet our criteria for magnitude and statistical significance.
- Children are ages 0-17, adults are age 18 and over, and older adults are age 65 and over.
- "Blacks" indicates individuals who identify their race as Black or African American.
- "Hispanics" indicates individuals who identify their ethnicity as Hispanic, Latino/a, or Spanish origin and include all races.
- "Measure improved" indicates performance on the measure improved; "measure got worse" indicates performance on the measure showed worsening.
i. Asian includes the former category of Asian or Pacific Islander prior to Office of Management and Budget guidelines, when information was not collected separately by group.
ii. Not all data sources collect information by race and ethnicity separately. In such cases, comparisons are made by combining racial/ethnic group categories (e.g., comparing non-Hispanic Blacks and Hispanics with non-Hispanic Whites).
iii. Unless otherwise indicated, throughout this report, poor is defined as having family income less than 100% of the Federal poverty level (FPL); low income refers to income of 100% to 199% of the FPL; middle income refers to income of 200% to 399% of the FPL; and high income refers to income of 400% of the FPL and above. These are based on U.S. census poverty thresholds for each data year, which are used for statistical purposes.
iv. Less than a high school education refers to people who did not complete high school.
v. Public insurance includes Medicaid, Children's Health Insurance Program (CHIP), State-sponsored or other government-sponsored health plans, Medicare, and military plans.
vi. A CSHCN Screener instrument was developed through a national collaborative process as part of the Child and Adolescent Health Measurement Initiative coordinated by the Foundation for Accountability. For more information, see: Bethel CD, Read D, Stein REK, et al. Identifying children with special health care needs: development and evaluation of a short screening instrument. Ambul Pediatr 2002 Feb;2(1):38-48.
vii. A probability of 0.10 was selected as the significance level because the magnitude of the standard errors varied considerably by type of data.
Page originally created April 2015