Recommendations to Improve Children's Health Care Quality Measures

Background Report on the 2012 Process

This background report describes the process used to identify, evaluate, and select children's health care quality measures to be recommended for addition to the initial core set of 24 measures released by the Secretary, U.S. Department of Health and Human Services, in December 2009.

Contents

Structured Abstract
Introduction
Methods for Recommending Measures
Results
Discussion
Conclusion
References
Table 1. SNAC Scoring Opportunities
Table 2. SNAC-Recommended Children's Health Care Quality Measures for Additions to the Core Set in 2013 and for Two Other CHIPRA Purposes
Appendix A: Members of the 2012 AHRQ Subcommittee to the National Advisory Council on  Children's Health Quality Measures (SNAC)
Appendix B: Measure Scores by CHIPRA Category, Measure Name, and Scoring Round 

Structured Abstract

Background: In 2009, the Children's Health Insurance Program Reauthorization Act (CHIPRA) (Public Law 111-3) directed the Secretary of the U.S. Department of Health and Human Services (HHS) to identify by January 1, 2010, an initial core set of health care quality measures for voluntary use by Medicaid and the Children's Health Insurance Program (CHIP) programs. In addition, CHIPRA directed the Secretary to establish a CHIPRA Pediatric Quality Measures Program (PQMP). The purposes of the PQMP go beyond use by Medicaid and CHIP programs; the PQMP is to:

  1. Improve and strengthen the initial core set of measures of health care quality established under CHIPRA.
  2. Expand on existing pediatric quality measures used by public and private health care purchasers.
  3. Increase the portfolio of evidence-based consensus pediatric quality measures available to public and private purchasers of children's health care services, providers, and consumers.

Measures developed under the PQMP are to be used to recommend changes to the initial core set beginning January 1, 2013, and annually thereafter. In this document, measures recommended as changes to the initial core set (purpose A) are referred to as additions to the core set or the improved core set. Measures meeting purposes (B) and (C) are referred to as measures for "other uses."

Methods: The Agency for Healthcare Research and Quality (AHRQ) and the Centers for Medicare & Medicaid Services (CMS) established the PQMP by the legislative deadline and subsequently awarded seven cooperative agreements to designated CHIPRA PQMP Centers of Excellence (COEs) to develop, enhance, and improve child health quality measures, as well as a contract to provide technical support for the PQMP. In order to meet the January 1, 2013, deadline, AHRQ, CMS, and a broad array of stakeholders and experts worked throughout 2011 and 2012 to develop and/or assess children's and related health care quality measures for potential inclusion in the 2013 Improved Core Set and other uses. Key steps in the process included development of a set of desirable measure attributes, a call for public nominations of measures, and submission of measures by the COEs. A Subcommittee of the AHRQ National Advisory Council on Healthcare Research and Quality (SNAC) was also established to review a subset of submitted measures containing sufficient information. The SNAC applied the desirable measure attributes using a modified Delphi approach and other scoring processes to assess measures and to make recommendations for measures to be added to the core set and for other uses. CMS considered the SNAC recommendations and advised the Secretary of HHS on potential improvements to the initial core set.

Results: In all, 77 measures were submitted for consideration by the SNAC; 64 by the public and 13 by CHIPRA PQMP COEs supported with cooperative agreement grants. Fourteen public nominations did not have sufficient information for SNAC review; thus, 63 measures (50 publicly nominated and 13 COE-nominated) were reviewed by the SNAC. The SNAC recommended five measures as improvements to the initial core set, and two for other uses. The five measures recommended by the SNAC to CMS as improvements to the core set were: coverage in Medicaid and CHIP, duration of a newborn's first enrollment, human papillomavirus (HPV) vaccination in female adolescents, recording computerized tomography (CT) exposure in children, and medication management for children with asthma. The two recommended for other uses were: behavioral health risk assessment in prenatal care, and tobacco use and help with quitting among adolescents. After consideration, CMS is recommending to the Secretary of HHS three measures: HPV vaccination in female adolescents, medication management for children with asthma, and behavioral health risk assessment in prenatal care. The 2013 Improved Core Set will be posted by CMS in a State Health Official (SHO) letter to be released by January 1, 2013.

Conclusion: The first year of a collaborative public-private process met the CHIPRA legislative deadline of January 1, 2013. The recommended measures take into account the legislative requirements of CHIPRA, help fill gaps and complement measures in the initial core set, and begin to identify areas of need for quality improvement.

Return to Contents 

Introduction

Initial Core Set

In 2009, the Children's Health Insurance Program Reauthorization Act (CHIPRA) (Public Law 111-3) amended Section 1139 of Title XXI of the Social Security Act by adding Section 1139A on Child Health Quality Measures.1 Section 1139A(a) charged the Secretary of the U.S. Department of Health and Human Services (HHS) with identifying an initial core set of health care quality measures for voluntary use by Medicaid and Children's health Insurance Program (CHIP) programs. This initial core set of measures was to be culled from measures already in use and include certain key characteristics (e.g., evidence-based, able to identify disparities, and comprehensive enough to cover all child ages and settings and providers of care, including services to promote healthy birth and childhood [e.g., perinatal and peripartum services]). In 2009, a partnership between the Agency for Healthcare Research and Quality (AHRQ) and the Centers for Medicare & Medicaid Services (CMS) and a 2009 Subcommittee of the AHRQ National Advisory Council on Healthcare Research and Quality2 (2009 SNAC) were established to develop criteria for assessing identified measures and to recommend measures for the initial core set. The 2009 SNAC recommended 25 measures, and the Secretary of HHS posted 24 measures for public comment by the legislative deadline of January 1, 2010.The initial core set was officially released via a State Health Official letter in February 2011.3

Pediatric Quality Measures Program

In recognition that children's health care quality measurement was a nascent but growing field in 2009, and that the measures included in the initial core set were to be selected only from measures available at that time, Section 1139A(b) of CHIPRA required the establishment of a Pediatric Quality Measures Program (PQMP) by January 1, 2011. As stated in Section 1139A(b)(1), the purposes of the PQMP were to:

  1. Improve and strengthen the initial core set of measures of health care quality established under CHIPRA.
  2. Expand on existing pediatric quality measures used by public and private health care purchasers.
  3. Increase the portfolio of evidence-based consensus pediatric quality measures available to public and private purchasers of children's health care services, providers, and consumers.

In addition, Section 1139A(b)(2) called for the measures developed under PQMP to be:

  • Evidence-based and, where appropriate, risk-adjusted.
  • Designed to identify and eliminate racial and ethnic disparities in child health and the provision of health care.
  • Designed to ensure that the data required for such measures is collected and reported in a standard format that permits comparison of quality and data at the State, plan, and provider levels.
  • Updated periodically.
  • Responsive to the child health needs, services, and domains of health care quality described in clauses (i), (ii), and (iii) of subsection (a)(6)(A).

CHIPRA also delineated a broad set of stakeholders to be included in the PQMP process (Section 1139A(b)(3)) and stipulated that grants and contracts could be used to develop, validate, and test a portfolio of pediatric quality measures (Section 1139A(b)(4)). Section 1139A(b)(5) required the Secretary of HHS to "publish recommended changes to the core measures described in subsection [(Section 1139A)] (a) that shall reflect the testing, validation, and consensus process for the development of pediatric quality measures described in subsection paragraphs (1) through (4)."

Measures developed under the PQMP are to be used to recommend changes to the initial core set beginning January 1, 2013, and annually thereafter. In this document, measures recommended as changes to the initial core set are referred to as additions to the core set or the improved core set. Measures meeting purposes (B) and (C) are referred to as measures for "other uses."

In response to the legislative directive to develop, validate, and test measures (Section 1139A(b)(4), in March 2011, AHRQ and CMS awarded cooperative agreement grants to seven CHIPRA PQMP Centers of Excellence (COEs).4 Each of these programs comprises multiple entities with diverse talents and expertise, including measurement capability, clinical knowledge, and the stakeholder groups of frontline providers, State Medicaid and CHIP programs, and patient representatives. Following a kickoff meeting with the COEs in April 2011, and consultation with CMS, the COEs were assigned an initial set of topics for measure development. In early 2012, the COEs were assigned a second set of priority topics.5 AHRQ and CMS also awarded a contract to Research Triangle International, Inc., to provide technical support to the PQMP (the Coordinating and Technical Assistance Center [CCTAC]).

In response to Section 1139A(b)(3), an Expert Panel was identified in 2011 to help develop desirable measure attributes for making recommendations6; the Expert Panel, with several additional members, became the 2012 Subcommittee on Children's Healthcare Quality Measures of the AHRQ National Advisory Council on Healthcare Research and Quality (SNAC) (go to Appendix A).

In addition to the COEs, the CCTAC, and the SNAC, two CMS CHIPRA Quality Demonstration Grantee States (Illinois and Massachusetts) participate in the PQMP. As part of their State demonstration projects, these two States will experiment with and evaluate new children's health care quality measures7,8; they joined the PQMP to share lessons learned with AHRQ, CMS, CCTAC, and the COEs.

This report describes the process used to assess submitted measures for all three purposes of the CHIPRA PQMP, and it includes a section on CMS's final measure recommendations for the 2013 Improved Core Set.

Return to Contents 

Methods for Recommending Measures

In order to meet the January 1, 2013, deadline, AHRQ, CMS, and a broad array of stakeholders and experts worked throughout 2011 and 2012 to assess children's and related health care quality measures for potential use by Medicaid and CHIP programs and other public and private entities. Key steps in the process included development of a set of desirable measure attributes, a call for public nominations of measures, identification of the Expert Panel and SNAC, and SNAC's use of a modified Delphi process9 to assess measures against the set of desirable measure attributes and make recommendations for measures to be added to the core set and for other uses.

Identification of Desirable Measure Attributes

In September 2011 and early 2012, AHRQ engaged an expert panel and the COEs to determine the desirable measure attributes that should be used to assess measures for addition to the core set for voluntary use by Medicaid and CHIP programs ("core set") and other uses, beginning in 2013. The desirable measure attributes included specific domains identified in the CHIPRA legislation (e.g., child-focused, ability to identify disparities, understandability), as well as other key measure characteristics routinely used by other national leaders in health care quality measurement (e.g., importance, reliability, validity, feasibility).10,11,12 The results were codified in a password-protected online CHIPRA Pediatric Quality Measures Program Candidate Measure Submission Form (CPCF).

Submission of Measures

Public submissions. Using a Federal Register notice, AHRQ invited members of the public to nominate measures to be considered for improving the core set and other uses. Although the Federal Register notice summarized the components of the CPCF, the public could not be required to use the CPCF as a template for the 2012 submission cycle because the CPCF had not yet been approved by the U.S. Office of Management and Budget for Paperwork Reduction Act purposes. Thus, the information submitted to support consideration of measures varied. Data from the public submissions were extracted into the CPCF sections by CCTAC for public measure submissions judged to have sufficient information to assess a substantial number of desirable measure attributes codified in the CPCF.

COE submissions. The COEs were able to use the online CPCF containing the desirable measure attributes for measure submission.

SNAC 2012 Assessment of the Measures

Information from the online CPCF was provided to the SNAC using a password-protected console on the CCTAC Web site (chipra.rti.org). The SNAC was provided access to original public measure submissions. The 2012 SNAC used this information to evaluate each measure, as described below. In addition, to provide context for the SNAC, the AHRQ team created and shared a spreadsheet that organized all 2012 measure submissions by CHIPRA category and highlighted any overlap with measures in the initial core set, COE assignments, and measures in the adult core Medicaid set required by the Patient Protection and Affordable Care Act.13

Evaluation of Measures by the SNAC

For SNAC scoring purposes, each measure's online CPCF form was expanded to include a function for scoring nine of the desirable measure attributes and three opportunities for "global scoring" (Table 1). In consideration of the limited time SNAC members had for scoring substantial numbers of measures on multiple attributes, scoring opportunities 1 through 9 were optional. SNAC members could score each of these separately (using the modified Delphi scoring approach on a scale of 1-9 described below) and calculate a combined global score, or they could simply read through the responses related to desirable measure attributes and more qualitatively assign a global score. In addition, SNAC members were asked to score on a scale of 1-5 whether a measure should be included as a core set measure for voluntary use by Medicaid and CHIP programs (Scoring Opportunity 11) and/or for other uses (Scoring Opportunity 12).

Scoring opportunities 1 through 10 used standard Delphi scoring categories (1-3 = Low; 4-6 = Medium; and 7-9 = High). For scoring opportunities 11 and 12, a Likert scale of 1-5 was used (1 = Strongly disagree; 2 = Disagree; 3 = Neither Agree nor Disagree; 4 = Agree; 5 = Strongly Agree).

In addition to numerical scoring, the online mechanism for use by the SNAC provided opportunities for members to insert comments related to the desirable measure attributes.

Modified Delphi Process. A modified Delphi process using three rounds of scoring was used to determine the measures that would be considered for the improved core set and for other purposes. Delphi rounds I and II were completed by SNAC members prior to an in-person meeting held on September 12, 2012. The final (Delphi III) round to reach consensus on measures was conducted at the in- person meeting.

Delphi Round I. In Delphi Round I, the 50 publicly nominated measures that were extracted into the CPCF form were evaluated by the SNAC across a 4-week period. After reviewing a summary of the submitted scores during a webinar, the SNAC subsequently agreed on the following cutoff points for further consideration of a measure:

  • If on Question 10, a measure received a median score of 7 or greater and had an interquartile range (IQR) of 2.25 or lower, indicating a low dispersion of scores, AND on Question 11 or Question 12 received a median score of 4 or greater and had an IQR of 1.25 or lower, the measure would be considered for the improved core set and/or other uses at the in-person meeting. In summary, measures that met this cutoff were considered to have received high scores.
  • If the measure had a median score of greater than 7 for Question 10, regardless of the median score for Question 11 or 12, OR if the measure received a median score of 6 or greater and/or an IQR greater than 3 for Question 10, AND a median score of 4 or higher for Question 11 or 12, then the measure was to be voted on again in Delphi II.
  • If a measure scored below the predetermined cutoff points in Delphi I and all preceding rounds, the measure would not receive further consideration in Delphi II or III but could be brought back for discussion at the in-person meeting, time permitting.

Delphi Round II. In Delphi II, public measures that received moderately high scores and/or had a relatively high IQR were voted on again by the SNAC. In addition, COEs were encouraged to submit measures for consideration by the SNAC, and three of the COEs were able to do so and submitted a total of 13 measures. The same scoring system used in Delphi Round I was used in Delphi Round II, except that only measures that received a median score of 7 or higher on a summary rating scale and had an IQR of 2.25 or higher, indicating a greater dispersion of scores, progressed onto Delphi III scoring. Measures scoring 7 or higher on the summary rating scale with a low dispersion of scores were moved forward for voting at the in-person meeting. SNAC members participated in a webinar to discuss the results of Delphi II and plans for Delphi III in early September.

Delphi Round III. As an introduction to the in-person SNAC meeting, CMS presented a summary of the Medicaid and CHIP programs (e.g., eligibility by income, types of delivery system, extent of voluntary State reporting using the initial core set). With the initial core set only in place for about 18 months, CMS's goal is to focus on making incremental changes to the core set in order to maintain a "parsimonious" number of measures. Many States face challenges because of widely varying contractual and service delivery arrangements and resource limitations, and they will soon be asked to report on adult core Medicaid measures.13 AHRQ noted the importance of also recommending measures for use by private payers (as an example of potential "other uses"). More than 50 percent of children ages 4 and older are covered by private plans.14 The ratio of spending for privately insured children is almost twice that of publicly insured children.15 As an example of quality problems, in 2008, privately insured children ages 3-6 were less likely to have ever had their vision checked than were publicly insured children.16

In Delphi Round III, SNAC members used an electronic scoring system to score each measure that was moved forward from Delphi II. Due to characteristics of the electronic scoring system, instead of scoring measures on a scale from 1 to 9, as measures were scored in earlier Delphi rounds, SNAC members were required to use a three point scale:

  • "Low" (corresponding to Delphi scores 1–3).
  • "Medium" (scores 4–6).
  • "High" (scores 7–9).

For a measure to move forward from Delphi Round III into consideration for addition to the core set or for other uses, a measure had to be scored "high" by 70 percent or more of voting SNAC members.

Recommendations for the Core Set and/or Other Purposes. After the completion of Delphi Round III, SNAC members discussed the measures that had scored highly in Delphi Rounds I, II, or III and voted on whether to recommend the measures as additions to the core set and/or for other uses. The SNAC considered each measure in the context of other measures in the same category in the initial core set and in the 2012 measure submissions. For example, a cesarean section measure is already included in the initial core set, but two measures of cesarean section were submitted in 2012. Two asthma measures were submitted, and the SNAC discussed which one would have more validity as a quality measure. The initial core set includes a measure of health-care associated infections for newborns and children in intensive care units; this measure was compared to the submitted measure of newborn infection rates. The SNAC focused considerable attention on the feasibility of Medicaid and CHIP programs' implementation of measures.

To be recommended for consideration as an addition to the core set, at least two-thirds (66 percent) of voting SNAC members needed to support the measure in a YES (recommend for improved core set)/NO (do not recommend for improved core set) vote. High-scoring measures that were not recommended to be added to the core set were voted on to be recommended for other uses. Another YES/NO vote was held for a measure to be recommended for other uses, and again, approval for recommendation required endorsement by 66 percent of voting SNAC members.

A transcript of the entire SNAC meeting will be made available at the AHRQ CHIPRA Web.

Return to Contents 

Results

Appendix B summarizes, for each measure considered in 2012, the results of three rounds of scoring using the Modified Delphi approach applied to Scoring Opportunity 10 (the global score, taking all desirable measure attributes and information into consideration), as well as the results of votes for additions to the core set or for the two other CHIPRA PQMP purposes.

In Delphi Round I, six measures scored highly enough to be voted on at the in-person meeting, 20 met criteria to be considered again in Round II, and 24 did not move forward. In Round II, of the 20 measures moved forward from Round I, five met criteria to be voted on for the improved core set and other purposes at the in-person meeting, 10 met criteria for Delphi III, and 18 did not move forward. Of the 13 measures submitted by the three COEs for initial consideration during Delphi Round II, two met criteria for voting for the improved core set at the in-person meeting, five met criteria for Delphi Round III, and six did not move forward. Of the 10 measures considered in Delphi III, two moved forward for consideration for the improved core set and other purposes, for a total of 13 measures for such consideration. The SNAC recommended five measures as additions to the core set. Finally, of the eight high-scoring measures that were not recommended as improvements to the core set, the SNAC recommended two measures for use by other private and public programs. A summary of measures recommended for the improved core set and for other purposes is presented in Table 2; each of the recommended measures is described in more detail in the following section.

SNAC-Recommended Measures

Coverage in Medicaid and CHIP. This measure is designed to assess the continuity of enrollment of children in Medicaid and CHIP as the total percentage of time an individual is enrolled over an 18-month interval. The total percentage of enrollment may reflect a single enrollment spell or the sum of non-contiguous enrollment spells occurring within the specified time interval and represents a global picture that takes into account both lengths of enrollment and gaps in coverage. For an individual, the metric is a ratio of their total time enrolled (as a sum of days) divided by the time eligible (as a sum in days). In addition, for population reporting, two quantities are to be tracked. The first is a "ratio" measure, which is the sum of all months enrolled for all patients divided by the sum of all months eligible for all patients. The other is an "average" measure, which is the average of the individual patient ratios.

Despite the request in the CHIPRA legislation to include a duration and coverage measure in the core set, the 2009 SNAC was unable to identify a sufficiently valid indicator for this topic. Thus, development of such a measure was assigned to the Children's Hospital of Philadelphia (CHOP) COE. At the same time, CMS had to respond to the requirement in CHIPRA Section 402 for a measure to assess enrollment and retention data (including data with respect to continuity of coverage or duration of benefits).1

This measure is flexible; the cohort can be defined at any time period of interest and can be tracked as long as data are available. Moreover, the definition of new enrollee can be varied. This measure is also sensitive to gaps in coverage and somewhat sensitive to trends in eligibility rates due to changes in state or nationwide economic downturns and upturns, which are periods of time that children tend to enroll and disenroll, respectively.

The SNAC supported recommending the CHOP COE measure principally because of its feasibility—i.e., States can calculate coverage and enrollment from existing Medicaid and CHIP administrative data. Furthermore, it measures continuity of coverage by Medicaid or CHIP, rather than measuring coverage in each public program separately, thereby presenting a more complete picture of public insurance coverage. States may also find this measure valuable because the measure of coverage in Medicaid and CHIP could highlight State practices that reduce or increase administrative barriers to obtaining coverage in public programs, such as frequency of re-enrollment and documentation necessary for enrollment.

Duration of Newborn's First Enrollment. This measure is a prospective metric that quantifies the number of newborn enrollees continuously enrolled in public coverage at 6 months, 12 months, and 18 months after enrollment. It is an age stratification of the CMS Duration Measure required in Section 402 of CHIPRA. There are several variations of this measure that can be adapted to the different types of State Medicaid/CHIP programs, such as standalone or jointly administered programs. The measure assesses the total number of newborns enrolled in Medicaid and/or CHIP during a 6-, 12-, or 18-month prespecified observation period and can be calculated by specific program or by coverage by either Medicaid or CHIP, which takes into account transitions between programs.

When recommending this measure, SNAC members noted that maintaining coverage for a newborn is critical, especially in light of the number of well-child visits that are scheduled in the first year of life. In addition, the measure could be used to highlight relationships to other health outcomes, including infant mortality. The SNAC also noted that this measure was likely to be feasible for States because it can be derived from existing Medicaid and CHIP administrative data.

HPV Vaccine for Female Adolescents. This measure determines the percentage of female adolescents 13 years of age who have had three doses of the HPV vaccine by their 13th birthday. To calculate this measure, there should be documentation of three doses of the HPV vaccine administered between the 9th and the 13th birthday for female adolescents who turned 13 years of age during the measurement year. Adolescents who had a contraindication for the HPV vaccine are excluded. According to the Centers for Disease Control and Prevention (CDC), sexually transmitted diseases (STDs) are among the most common infections in American youth, and HPV infection is the most common STD in the United States. HPV is responsible for nearly 70 percent of all cases of cervical cancer and 90 percent of cases of anogenital warts.17 Morbidity and deaths associated with HPV infections could be prevented through vaccination, and vaccination is most effective before a female is exposed to HPV, i.e., before they become sexually active. Thus, HPV vaccination is recommended at 11 and 12 years of age, as the median age of first sexual experience for Americans is 17 years, and studies suggest that 13 percent of girls initiate sexual activity before the age of 15.17

This measure is part of the National Committee for Quality Assurance (NCQA) Health Effectiveness Data and Information Set (HEDIS) Health Plan measure set, which is a widely used instrument for health plan quality measurement and improvement. This measure complements the NCQA HEDIS Immunizations for Adolescents measure already in the initial core set, which assesses immunization status for meningococcal vaccine (MCV4), and one tetanus, diphtheria, and acellular pertussis vaccine (Tdap) or one tetanus, diphtheria toxoids vaccine (Td), by a child's 13th birthday, but does not include HPV vaccination. The HPV vaccine was added to the Advisory Committee on Immunization Practices (ACIP) vaccine schedule for adolescents in 2007.

The SNAC discussed how this measure reflects the current research on the importance of vaccination for HPV vaccine for the adult health outcome of preventing cervical cancer. SNAC members noted that by recommending this measure, the improved core set would reflect current, evidence-based practice guidelines.

Recording Radiation Exposure from Diagnostic Computerized Tomography (CT) Exams. This measure assesses the number of diagnostic computed tomography (CT) exams for which metrics of radiation exposure are documented in the electronic health record (EHR) as a proportion of CT scans performed at a facility. The measure requires including at least one metric of radiation exposure in the radiology report that describes a CT study's findings and interpretation. Suitable metrics include any of the following: Computed Tomography Dose Index Volume (CTDIvol), Dose Length Product (DLP), Size-Specific Dose Estimate (SSDE), or any other metric recommended by the American Association of Physicists in Medicine (AAPM).

The increasing numbers of CT scans performed in the pediatric population have raised concerns about the long-term risks of medical imaging to children.18 SNAC members noted that this measure could encourage providers to measure and record the level of radiation exposure, providing a basis for documenting cumulative exposure, and argued that including this measure in the core set would encourage greater transparency about the radiation exposure children receive. However, they acknowledged this as an aspirational measure with potential challenges to implementation, especially since more validity and reliability testing needs to be done. In addition, SNAC members and other participants in the SNAC meeting were aware that a COE had been assigned the task of developing a measure related to imaging.

Medication Management for People with Asthma. This measure assesses whether children 5 to 18 years of age with persistent asthma remain on their medications. A first rate examines the percentage of children who remained on an asthma controller medication prescription for at least 50 percent of their treatment period. The second rate examines those who remained on the medication for at least 75 percent of their treatment period. These rates are reported as the proportion of days covered by at least one prescription for an asthma controller medication, and the medication dispensed, during the measurement year.  

This measure can be paired with the initial core set measure that calculates the annual percentage of asthma patients with one or more asthma-related emergency room visits. Over 7 million children under the age of 18 have asthma, placing it among the leading causes of hospitalization for children.19 Appropriate medication management potentially could prevent a significant proportion of asthma-related costs (hospitalizations, emergency room visits, and missed work and school days).20 In 2009, child hospitalizations for asthma as a principal diagnosis cost the nation $1.5 billion.21

The SNAC rationale for recommending this measure is that asthma is a common chronic pediatric health condition, and this measure focuses on disease management. Moreover, the SNAC considered that it is feasible to implement this measure in Medicaid and CHIP because it can be derived from existing administrative data.

Behavioral Risk Assessment During Prenatal Care. This measure assesses the percentage of patients, regardless of age, who gave birth during a 12-month period, were seen at least once for prenatal care, and received a behavioral health screening risk assessment. Risk assessments include the following screenings at the first prenatal visit: screening for depression, alcohol use, tobacco use, drug use, and intimate partner violence. This measure provides a mechanism to help identify pregnant women with drug, alcohol, or smoking problems, as well as depression and abuse, which may help prevent adverse neonatal outcomes. The adverse effects of alcohol, tobacco, and drugs on the fetus, including fetal alcohol syndrome and intrauterine growth restriction, are widely known.22 In addition, women abused during pregnancy are more likely to be depressed, suicidal, and experience pregnancy complications and poor outcomes, including maternal and fetal death.23

SNAC members commented that the measure accounts for the mental and social factors involved in perinatal care. However, SNAC members were concerned that the measure relies on EHRs as its data source, and the use of EHRs is not yet universal among physicians. The measure would not be immediately feasible for use by all Medicaid and CHIP programs, according to the SNAC, which recommended the measure for other uses.

Tobacco Use and Help with Quitting Among Adolescents.This measure identifies the percentage of adolescents 12 to 20 years of age during the measurement year for whom tobacco use status was documented who received help with quitting if identified as a tobacco user. "Receiving help" is determined as documentation of any of the following: patient given advice to quit smoking or tobacco use, counseling on the benefits of quitting smoking or tobacco use, assistance with or referral to external smoking or tobacco cessation support programs, or current enrollment in smoking or tobacco use cessation program. The measure focuses on a clinical process (documentation of tobacco use and appropriate followup) that, if followed, has the potential to achieve a desirable clinical outcome (cessation of tobacco use, which can reduce the risk for a wide range of conditions known to be associated with tobacco use, such as asthma and lung cancer).

Smoking is a relevant issue in pediatric health care because over 2.6 million adolescents 18 years of age and younger are current tobacco users, with nearly one-fifth of all adolescents becoming current smokers before finishing high school.24,25 Furthermore, of adults who smoke on a daily basis, 82 percent reported trying their first cigarette before the age of 18, and 53 percent reported becoming daily smokers before the age of 18. The financial burden incurred from tobacco use is significant. From 2000 to 2004, annual expenditures (public and private) related to smoking were estimated to be $96 billion, and another $97 billion can be attributed to lost productivity each year.26

This measure was developed by the National Collaborative for Innovation in Quality Measurement (NCINQ) COE as part of its assignment to develop measures of the content of adolescent well-care. The COE submission provided, and the SNAC reviewed, results from measure testing that indicated that tobacco use status can be found in paper and electronic records, and that where found, documentation of followup care (including advice to quit smoking) existed for only 30 percent of patients, suggesting that considerable room for improvement exists. However, SNAC members noted that, because the data sources for this measure are the paper record and EHR, it would be challenging for some Medicaid and CHIP programs to calculate this measure. Other challenges for this measure identified by the SNAC include the lack of a definitive evidence base to support the effectiveness of provider counseling to quit smoking in adolescents, although such counseling in adults is evidence-based and recommended.27 In addition, none of the prescription drugs that can help adults quit smoking are approved for use in adolescents.

Finalizing Recommendations

As part of its review of the SNAC-recommended measures, CMS took into account noted concerns about several of the SNAC-recommended measures for the core set and considered other programmatic issues (e.g. feasibility, alignment with other national programs, and burden of reporting). CMS identified a total of three measures to recommend to the Secretary of HHS: HPV vaccination, medication management for asthma, and behavioral counseling in prenatal care. A State Health Official etter outlining the 2013 Improved Core Set will be released by January 1, 2013.28

Return to Contents 

Discussion

The SNAC process in 2012 was the first opportunity to recommend measures for consideration to improve the initial core set of measures. As with any new process, it also revealed opportunities for improvement in coming years.

One major opportunity is to identify additional opportunities for the adoption of rigorously developed, evidence-based, consensus measures of children's health care quality that have not been recommended for the core set for use by Medicaid and CHIP, as well as for opportunities for other public and private programs, plans, providers, and patients to use the Medicaid and CHIP core set.

Additionally, the SNAC has asked to be kept informed of the wider world of measure development and use in children's and perinatal health care so it can consider how the content of the core set of quality measures should evolve over time, in the context of the comprehensive set of CHIPRA domains (e.g., across all child ages, health service types, health care settings, and providers) and other sets of priorities (e.g., the National Quality Strategy,29 the Institute of medicine [IOM] domains30). The SNAC also suggested refinements to the SNAC review process for subsequent years. For instance, to make the review process less cumbersome, the SNAC suggested greater triaging of measures before assigning them to the SNAC, rather than having each member review every measure. The SNAC also asked for more time to conduct their reviews and Delphi rounds. Some SNAC members recommended the use of consistent scoring scales for all questions (rather than switching to a 3-point or yes/no scale for final Delphi scoring and recommendations) and use of a measure of dispersal other than the IQR (which assumes a normal distribution of scores) for the Delphi cutoff points.

Return to Contents 

Conclusion

This was the first year of a process to identify recommendations for additions to the initial core set of children's health care quality measures for Medicaid and CHIP and to recommend measures for the two other CHIPRA PQMP uses beyond Medicaid and CHIP. The SNAC recommendations take into account the legislative requirements of CHIPRA, and the proposed measures help fill gaps and complement measures in the initial core set. The core set is meant to include measures that are evidence-based and feasible and cover a wide range of pediatric health services. As the PQMP work continues, the program focuses on advancing the science of quality measurement, aligning its work with the work of other entities, and, ultimately, adding value to the health care delivered to children in the United States and improving children's outcomes, especially the outcomes of the vulnerable children covered by Medicaid and CHIP programs.

Return to Contents 

References

  1. U.S. Congress and U.S. President. Public Law 111-3, Children's Health Insurance Program Reauthorization Act 2009. Accessed Dec. 7, 2012.
  2. Agency for Healthcare Research and Quality. National Advisory Council on Healthcare Research and Quality. Accessed Dec. 5, 2012.
  3. Centers for Medicare & Medicaid Services. Initial Core Set of Children's Health Care Quality Measures: Technical Specifications and Resource Manual for Federal Fiscal Year 2011 Reporting—Updated December 2011. 2011. Accessed Dec. 7, 2012.
  4. Agency for Healthcare Research and Quality. Pediatric Quality Measures Program (PQMP) Centers of Excellence Grant Awards 2012. Accessed Dec. 7, 2012.
  5. Agency for Healthcare Research and Quality. CHIPRA Measures by CHIPRA Categories: Initial Core Set and Pediatric Quality Measures Program Centers of Excellence Measure Assignments; CHIPRA Undated. Accessed Dec. 7, 2012.
  6. Agency for Healthcare Research and Quality. 2011 Expert Panel on Assessing Healthcare Quality Measures for CHIPRA Improved Core Sets—Pediatric Quality Measures Program CHIPRA 2011. Accessed Dec. 7, 2012.
  7. Secretary of U.S. Department of Health and Human Services. 2010. Accessed Dec. 5, 2012.
  8. Agency for Healthcare Research and Quality. National Evaluation of the CHIPRA Quality Demonstration Grant Program. Accessed Dec. 5, 2012.
  9. Boulkedid R, Abdoul H, Loustau M, et al. Using and reporting the Delphi method for selecting healthcare quality indicators: A systematic review. PLOS One Jun 2011;6(6):e20476.
  10. gency for Healthcare Research and Quality. Domain Framework and Inclusion Criteria. National Measures Clearinghouse™; Undated. Accessed Nov. 26, 2012.
  11. National Committee for Quality Assurance. HEDIS and Quality Measurement: Desirable attributes of HEDIS. Undated. Accessed Nov. 26, 2012
  12. National Quality Forum. Measure Evaluation Criteria. 2011. Accessed Nov. 26, 2012.
  13. U.S. Department of Health and Human Services Centers for Medicare & Medicaid Services. Medicaid Program: Initial Core Set of Health Care Quality Measures for Medicaid-Eligible Adults. 2012; Vol. 77, No. 2. Accessed Nov. 26, 2012.
  14. Agency for Healthcare Research and Quality. Table 1. Health insurance coverage of the civilian noninstitutionalized population: Percent by type of coverage and selected population characteristics, United States, first half of 2010. Medical Expenditure Panel Survey (MEPS).  Accessed Sep. 8, 2012.
  15. Cohen J. Per capita U.S health care expenditures in dollars, children <18 with a health care expenditure, 2009. Personal communication with Dougherty D, Rockville, MD; 2012.
  16. Agency for Healthcare Research and Quality. National Healthcare Quality and Disparities Reports. Measuring Health Care Quality, multiple years; http://www.ahrq.gov/qual/measurix.htm. Accessed Dec. 7, 2012.
  17. Centers for Disease Control and Prevention. Recommended immunization schedules for persons aged 0 through 18 years, United States-2012. MMWR, 2012. Accessed Nov. 14, 2012.
  18. Brenner D, Hall E. Computed tomography—an increasing source of radiation exposure. N Engl J Med 2007;357:2277-2284.
  19. Akinbami L. The state of childhood asthma, United States, 1980–2007. Pediatrics 2009;123:S131-145.
  20. American Lung Association. Trends in asthma morbidity and mortality. 2009. 2012. Accessed Nov. 8, 2012.
  21. HCUPNet KID 2009 calculation. Accessed Dec. 7, 2012.
  22. Minnes S, Lang A, Singer L. Prenatal tobacco, marijuana, stimulant, and opiate exposure: outcomes and practice implications. Addict Sci Clin Pract Jul 2011;6(1):57-70.
  23. Nelson H, Bougatsos C, Blazina I. Screening women for intimate partner violence: a systematic review to update the U.S. Preventive Services Task Force recommendation. Ann Intern Med Jun 5 2012;156(11):796-808.
  24. University of Michigan. Monitoring the Future Study, 2011. Accessed Nov. 14, 2012.
  25. Substance Abuse and Mental Health Services Administration. National Survey on Drug Use and Health (NSDUH), 2010. Accessed Nov. 14, 2012.
  26. Campaign for Tobacco-Free Kids. Tobacco overview. 2012. Accessed Nov. 14, 2012.
  27. U.S. Preventive Services Task Force. Counseling to Prevent Tobacco Use and Tobacco-Caused Disease. 2003. Accessed Nov. 26, 2012.
  28. Centers for Medicare & Medicaid Services. SHO Letter.
  29. S. Department of Health and Human Services. National Strategy for Quality Improvement in Health Care. 2011. Accessed Dec. 5, 2012.
  30. Institute of Medicine. Future Directions for the National Healthcare Quality and Disparities Reports. 2010. Accessed Dec. 5, 2012.

Return to Contents
Proceed to Next Section

Current as of December 2012
Internet Citation: Recommendations to Improve Children's Health Care Quality Measures: Background Report on the 2012 Process. December 2012. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/policymakers/chipra/pubs/background-2012/index.html