Spotlight on North Carolina

National Evaluation of the CHIPRA Quality Demonstration Grant Program

July 2015

This brief highlights the major strategies, lessons learned, and outcomes from North Carolina's experience in the first 5 years of the quality demonstration funded by the Centers for Medicare & Medicaid Services (CMS) through the Children's Health Insurance Program Reauthorization Act of 2009 (CHIPRA). In this demonstration, CMS awarded 10 grants that supported efforts in 18 States to identify effective, replicable strategies for enhancing the quality of health care for children. With funding from CMS, the Agency for Healthcare Research and Quality is leading the evaluation of the program.

North Carolina's Goals: Improve the quality of care for children by—

  • Calculating, reporting, and using quality measures.
  • Helping practices strengthen the medical home model for children with special health care needs.
  • Testing the Children’s Electronic Health Record Format.

North Carolina used quality measures to drive quality improvement

In collaboration with Community Care of North Carolina (CCNC), a public-private partnership covering all 100 counties in the State, North Carolina expanded the scope of its collection, reporting, and use of the Core Set of Children’s Health Care Quality Measures for Medicaid and CHIP (Child Core Set).1 Using CHIPRA quality demonstration funds, North Carolina—

  • Reported to CMS on 25 of 26 Child Core Set measures in 2014, up from 2 in 2010. North Carolina drew on data from various State agencies to calculate and report the measures to CMS. It also hired an independent vendor to collect survey data on patient experience.
  • Improved existing practice-level quality reports. North Carolina incorporated additional child-focused measures, including Child Core Set measures, into quarterly reports that the State makes available to all practices serving Medicaid and CHIP beneficiaries. In 2014, 63 percent of 235 randomly sampled North Carolina pediatricians and family physicians reported that they received reports from Medicaid, and 58 percent reported that they received quality reports with selected Child Core Set measures included.2
  • Helped more than 200 practices improve care quality. North Carolina hired 14 pediatric quality improvement (QI) specialists—one for each CCNC provider network— to analyze network- and practice-level data and work with practices to set QI goals. QI specialists helped practices develop QI teams, identify QI activities, and improve targeted care processes. When the State determined that some QI specialists needed additional QI skills to be most effective, North Carolina hired a statewide QI coordinator and invested in substantial training in technical QI and clinical content areas to ensure that the specialists were prepared to support practices across a range of activities.
  • Improved performance on quality measures. North Carolina made modest but meaningful improvements on several quality measures (Figure 1) during a 15-month period. Demonstration staff believe that collaboration between the QI specialists and practices contributed to the changes.

Figure 1. Increases in measures for all children enrolled in Medicaid and CHIP in North Carolina

Figure 1 shows changes in statewide pediatric quality measures for all children enrolled in Medicaid and the Children’s Health Insurance Program (CHIP). North Carolina reported measures for December 2011, December 2012, and March 2013. Fifty-five percent of children aged 3 or older received dental varnishing in 2011 compared to 58 percent in 2012 and 2013. The percentage of children receiving autism screening increased from 42 percent in 2011 to 53 percent in 2012 and to 55 percent in 2013. The percentage of adolescents receiving behavioral and risk factor screening increased from 7 percent in 2011 to 11 percent in 2012 and to 12 percent in 2013. The percentage of children receiving body mass index screening increased from 3 percent in 2011 to 11 percent in 2012 and to 13 percent in 2013. Data were reported by North Carolina and not independently validated.

Note: Data were reported by North Carolina and not independently validated.

Practices improved delivery of recommended preventive services

North Carolina used a learning collaborative model to educate 23 practices on strengthening their medical home characteristics for children, especially children with special halth care needs (CSHCN). Participating practices—

  • Built their QI capacity. Participating practices developed QI teams charged with improving the practices’ quality of care. The teams attended group in-person and virtual learning sessions delivered by the State and received individualized assistance. Clinicians were eligible to receive Maintenance of Certification (MOC) credit for completing training modules. Initially, the State encountered challenges both in maintaining practice participation in learning activities and motivating practices to use data to drive QI. In response, the State offered practices (1) financial incentives tied to participation in learning activities and (2) individualized assistance to help practices run and use data reports from their electronic health records (EHRs). The State also developed a video series and clinical toolkit to help practices engage adolescents in their own care.3
  • Implemented care process improvements. Participating practices undertook a variety of process improvements to improve the delivery of preventive care, particularly for CSHCN. For example, many practices instituted the use of validated, State-recommended mental health and developmental screening questionnaires, (2) regular measurement and recording of children’s body mass index, (3) the use of motivational interviewing to help families improve nutrition and increase physical activity among children at risk of obesity, and (4) dental varnishing for children.

Practices used new EHR features for improved capture of information about children

The Model Children’s EHR Format (Format) is a set of recommended requirements for EHR data elements, data standards, and functionality released by the United States Department of Health and Human Services in February 2013.4 To assess the Format’s usefulness, North Carolina worked concurrently with national EHR vendors and State-level child-serving practices that used the vendors’ products. With support from the demonstration, the State hired four coaches to help the practices participate in the assessment. The State—

  • Helped more than 25 practices use their EHRs more effectively. The EHR coaches surveyed practices and vendors to understand how practices’ existing EHR functionality compared with the Format. Coaches and vendors then helped practices use existing features that met Format requirements. For example, coaches helped practices access tools that enabled direct entry of data from screening questionnaires into EHRs.
  • Encouraged vendors to develop new functionalities and provide training. To drive changes in EHR functionality that met Format requirements, State demonstration staff developed specifications to guide vendors in making modifications to their existing EHRs. As a result, some vendors enhanced their systems’ reporting capabilities; others produced tools to capture data at the point of care and created report views to assist practices in population management and QI. While some vendors made changes, the State indicated that working with them was a slow and difficult process. Some vendors were reluctant to add functionalities given competing demands, including meeting meaningful use requirements, and concerns that project participation might adversely affect their competitive edge.

Key demonstration takeaways

  • By improving practice-level quality reporting and facilitating QI, the State supported modest improvements to statewide rates of routine adolescent, autism, and obesity screening and the provision of dental varnishing.
  • North Carolina leveraged existing infrastructure to implement a statewide model of QI coaching. QI specialists trained practices to use quality measure reports to identify priorities, conduct QI activities, track progress, and standardize processes to improve the delivery of preventive services.
  • Practices participating in the learning collaborative reported enhanced QI capacity and implementation of new care processes.
  • Although some EHR vendors improved their systems to conform with the Format, many were slow to build Format requirements into their products. In the interim, providing direct assistance to practices helped improve their use of EHR functionality.


  1. For more information on the Child Core Set, visit
  2. We conducted a cross-sectional survey of pediatricians and family physicians who provide primary care to publicly insured children in North Carolina. The final sample included responses from 235 clinicians (46.9 percent response rate). Survey weights were used to calculate univariate statistics.
  3. The Engaging Adolescents Video Series is available at
  4. For more information on the Format, visit
Continuing Efforts in North Carolina

North Carolina will continue to pursue its CHIPRA quality demonstration activities until February 2016 under a grant extension approved by CMS. North Carolina plans to—

  • Continue tracking most child-focused measures as part of the State’s Medicaid performance measurement.
  • Continue employing QI specialists in the networks for children and start using specialists to improve care for adults.
  • Continue offering MOC Webinars, perhaps asking clinicians to pay a fee to view them.
  • Develop new training modules for child-serving practices on healthy weight and the medical home for children in foster care.
  • Seek additional funding to continue working with practices and engaging vendors to improve EHR functionality for children.

Learn More

North Carolina's CHIPRA quality demonstration experiences are described in more detail on the national evaluation Web site at

The following products highlight North Carolina's experiences—

The information in this brief draws on interviews conducted with staff in North Carolina agencies and participating practices, a survey of child-serving providers, and a review of project reports submitted by North Carolina to CMS.

The following staff from Mathematica Policy Research and the Urban Institute contributed to data collection or the development of this summary: Dana Petersen, Mynti Hossain, Rachel Burton, and Christal Ramos.

Page last reviewed July 2015
Page originally created July 2015
Internet Citation: Spotlight on North Carolina. Content last reviewed July 2015. Agency for Healthcare Research and Quality, Rockville, MD.