Spotlight on Pennsylvania

National Evaluation of the CHIPRA Quality Demonstration Grant Program

July 2015

This brief highlights the major strategies, lessons learned, and outcomes from Pennsylvania’s experience during the first 5 years of the quality demonstration funded by the Centers for Medicare & Medicaid Services (CMS) through the Children’s Health Insurance Program Reauthorization Act of 2009 (CHIPRA). For this demonstration, CMS awarded 10 grants that supported efforts in 18 States to identify effective, replicable strategies for enhancing the quality of health care for children. With funding from CMS, the Agency for Healthcare Research and Quality is leading the evaluation of the program.

Pennsylvania’s Goals: Partner with large health care systems to improve the quality of care for children by—

  • Implementing electronic screening questionnaires.
  • Encouraging improvement on child-focused quality measures.
  • Improving electronic health record (EHR) functionality.

Health systems developed and implemented electronic screening questionnaires

Pennsylvania partnered with the Children’s Hospital of Philadelphia (CHOP) and Geisinger Health System to develop and implement electronic screening for developmental delays, autism, attention deficit hyperactivity disorder, adolescent depression, and postpartum depression. Using CHIPRA quality demonstration funds, Pennsylvania, CHOP, and Geisinger—

  • Developed electronic screening questionnaires that parents or other caregivers complete through a secure online portal at home or on a tablet computer in the waiting room. The system automatically scores and uploads the screener to the electronic health record (EHR) so that the provider can review results during the visit. When appropriate, the EHR automatically suggests tools that providers can use to address concerns (such as automated referral letters to early intervention programs).
  • Implemented electronic screening in 86 primary care sites at CHOP and Geisinger. The two health systems administered more than 115,400 electronic screenings. While promising for pediatric conditions, providers indicated that electronic screening for postpartum depression remains challenging because child-serving providers are often unable to bill for this service.
  • Helped five additional health care organizations in the State begin implementation of electronic screening. The additional health care organizations started developing electronic templates, working with vendors to integrate screeners into their EHRs and encouraging providers to use the new screeners. Some systems are slower to make changes than others because of EHR limitations and competing organizational priorities.

Pennsylvania financially rewarded health care organizations for improvement on child-focused quality measures

Pennsylvania provided incentive payments to six health systems and a federally qualified health center for reporting selected measures from the Core Set of Children’s Health Care Quality Measures for Medicaid and CHIP (Child Core Set) using their EHRs and demonstrating improvement on those measures.1, 2 Health systems received $935,000 in incentive payments, ranging from $65,000 to $260,000. With this support and their own resources, these health systems—

  • Reported on 10 to 18 Child Core Set measures. Health systems indicated that two tasks in particular were resource intensive: (1) changing how patient data were recorded in EHRs in order for measures to be calculated, and (2) programing EHRs to generate quality measures. Compared with their counterparts, health systems using EHRs with advanced reporting capabilities were able to report more measures. Providers in health systems that used internal clinical and information technology staff to program measures indicated that the measures more accurately reflected actual performance than did measures programmed by contractors or EHR vendors.
  • Engaged practices in quality reporting. Child-serving physicians in health systems participating in the State’s financial incentive program were more likely than their counterparts in other health systems to generate internal quality reports and indicate that quality reports were effective (Figure 1).
  • Improved performance on quality measures, including childhood immunization status, body mass index assessment, well-child visits, and dental preventive services.

Figure 1. Child-serving Physicians’ Reported Experiences with and Attitudes Toward Quality Reporting in Pennsylvania

Figure 1 shows that child-serving physicians in participating health systems (n = 52) report more experience with and more positive attitudes toward quality reporting than physicians outside participating health systems (n = 178). Significantly more physicians in participating health systems reported generating internal quality reports (76 percent compared with 42 percent of physicians outside these systems). In addition, physicians in participating systems were more likely to feel that quality reports are effective (85 percent compared with 72 percent of physicians outside these systems). Both of these findings are statistically significant (p<0.05). Ninety-two percent of physicians in participating systems reported receiving external quality reports, and 88 percent reported pursuing quality improvement (QI) efforts in the last 2 years. In comparison, 87 percent of practices outside participating systems reported receiving external quality reports, and 78 percent reported pursuing efforts in the last 2 years. Data were reported by Pennsylvania and not independently validated.

Source: Cross-sectional survey of pediatricians and family physicians who provide primary care to publicly insured children in Pennsylvania.

Providers implemented new EHR features to better capture information about children

Pennsylvania worked with four health systems and a federally qualified health center to test the usefulness of CMS’s Model Children’s EHR Format (Format), a set of recommended requirements for EHRs used by child-serving providers.1, 4 With support from the CHIPRA quality demonstration, the health systems:

  • Incorporated new Format requirements into their EHR systems, including patient portals, alerts for immunizations, and fields for tracking social and family history. Health systems indicated that changing their EHR was a slow, difficult process. Challenges included vendor resistance to making changes, staff availability to implement changes, and provider resistance to change. Providers were more receptive to EHR changes when clinical and information technology staff worked together to prioritize and implement changes.

Key demonstration takeaways

  • Health systems implemented electronic screeners to improving screening rates.
  • Under the Demonstration, health systems started tracking additional Child Core Set measures and implemented quality improvement efforts to improve performance. Health systems indicated that programming their EHRs to calculate measures was resource intensive and would have been difficult without the financial assistance provided through the grant.
  • Improving existing or implementing new electronic systems was challenging for health systems, given competing demands for resources and providers’ time, as well as vendors’ resistance.
  • Health systems that involved cross-functional teams—including clinical staff at different levels, information technology experts, and administrators—in quality measurement and information technology efforts made more improvements than their counterparts that relied on vendors or less integrated teams to complete the work.
Continuing Efforts in Pennsylvania

Pennsylvania will continue its CHIPRA quality demonstration activities until February 2016 under a grant extension approved by CMS. Moving forward—

  • All participating health care organizations plan to develop electronic screening questionnaires for more conditions, implement screeners in additional practices, or both.
  • Most health systems plan to continue tracking some Child Core Set measures, though their efforts will be more limited when financial incentives end after the grant.
  • Health systems plan to continue to improve their EHRs to better serve children.


  1. Geisinger and CHOP helped to lead the quality measurement and electronic screening work. The following health care organizations also participated in these project activities and piloted the EHR Format: St. Christopher’s Hospital for Children, Hamilton Health Center, Hershey Medical Center, Children’s Hospital of Pittsburgh, and Pocono Medical Center.
  2. For more information on the Child Core Set, visit
  3. We conducted a cross-sectional survey of pediatricians and family physicians who provide primary care to publicly insured children in Pennsylvania. The final sample included responses from 54 providers in participating health systems (32 percent response rate) and 178 in nonparticipating health systems (44 percent response rate). Survey weights were used to calculate univariate statistics. The test of proportions was used to compare responses from intervention versus nonintervention physicians.
  4. The current version of the Format is available at

Learn More

Pennsylvania’s CHIPRA quality demonstration experiences are described in more detail on the national evaluation Web site available at

The following products highlight Pennsylvania’s experiences—

The information in this brief comes from interviews conducted with staff at Pennsylvania and the participating health care organizations, a survey of child-serving providers, and review of project reports Pennsylvania submitted to CMS.

The following staff from Mathematica Policy Research contributed to data collection or the development of this summary: Grace Anglin, Leslie Foster, and Mynti Hossain.

Page last reviewed July 2015
Page originally created July 2015
Internet Citation: Spotlight on Pennsylvania. Content last reviewed July 2015. Agency for Healthcare Research and Quality, Rockville, MD.