How are CHIPRA Quality Demonstration States encouraging health care providers to put quality measures to work?

Author: Leslie Foster

Contents

• Key Messages
• Background
• Findings
• Conclusions
• Implications
• Learn More
• Endnotes

This Evaluation Highlight is the fifth in a series that presents interim findings from the Children’s Health Insurance Program Reauthorization Act of 2009 (CHIPRA) Quality Demonstration Program. The Highlight compares and contrasts two projects that use quality measures to drive quality improvement. It features a pay-for-performance program in Pennsylvania that encourages health systems to use electronic health records (EHRs) and a State-led learning collaborative in South Carolina that encourages primary care practices to be more quality oriented. Our analysis covers the first 2 to 3 years of these 5-year demonstration projects, including a year of planning and 1 to 2 years of implementation.

Key Messages

The early experiences of Pennsylvania and South Carolina suggest that as States encourage health care providers to use the Core Set of Children’s Health Care Quality Measures for Medicaid and CHIP (Child Core Set) to drive quality improvement (QI), they should:

Background

In early 2011, the Centers for Medicare & Medicaid Services (CMS) released the Child Core Set measures to track the quality of care provided to children in Medicaid and the Children’s Health Insurance Program (CHIP). Measures in the Child Core Set cover a range of health domains, including prevention and health promotion, management of acute and chronic conditions, the availability of care, and family experiences of care. The measures are generally calculated as the percentage of qualifying patients that received a recommended service (for example, the percentage of 6-year-old children with a well-child visit in their 6^th year). Since 2011, CMS has encouraged all States to report to CMS annually on these measures for children enrolled in Medicaid and CHIP.¹

Ten of the 18 States participating in the CHIPRA quality demonstration are using a portion of their grant funds to develop valid and reliable procedures for the annual reporting required by the grant teams. In addition, eight of these 10 States are testing the application of practice-level versions of some or all measures in the Child Core Set. The first Highlight in this series described the technical and administrative steps States are taking to calculate quality measures at the practice level. This Highlight expands on that story by exploring how two of those States are applying measures from the Child Core Set to promote QI within primary care practices and health systems that include multiple practice sites. It describes (1) each project’s key design features, (2) changes that occurred within health systems and practices when measurement began, and (3) the ingredients thought to sustain improvements.

This Highlight draws information from semi-structured, in-person interviews conducted by the national evaluation team in spring and summer 2012. The evaluation team interviewed each State’s CHIPRA demonstration staff, staff of the health systems and primary care practices participating in the CHIPRA projects, and other stakeholders. This Highlight also draws on progress reports that States submitted to CMS in February and August 2012 and February 2013.

Findings

Pennsylvania providers responded positively to pay-for-improvement incentives

Pennsylvania financially rewards the use of EHRs for measuring and improving quality. Specifically, the project rewards participating health systems for reporting any of the Child Core Set measures from their EHRs in a base year and then for improved performance on a subset of eight measures (Figure 1) that were prioritized by project stakeholders (particularly clinicians).² The health systems participating in Pennsylvania’s CHIPRA demonstration include affiliated hospitals, primary care practice sites, and other facilities. In this project, health systems report quality data on behalf of their primary care practice sites.

Figure 1. Child Core Set Measures Selected for Pennsylvania’s Pay-for-Improvement CHIPRA Project

Participating health systems submit data to the Pennsylvania Department of Public Welfare (DPW), which administers the State’s Medicaid program, using measure specifications that DPW developed specifically for EHRs. The DPW checks the integrity of the data and then calculates the measures for children continuously enrolled in Medicaid or CHIP. Performance is measured annually, but the DPW encourages semiannual data submissions so that problems can be identified and resolved in advance. Health systems receive $10,000 per measure reported from an EHR for the base year (up to 18 measures or $180,000). They subsequently receive $5,000 for each absolute percentage point improvement per measure (up to 5 points or $25,000 per measure capped at a total payment of $100,000 per health system). The health systems generally use the payments to fund QI projects related to the measures.

The project is being phased in. Two large health systems with established EHRs and experience in QI initiatives were asked to participate first, followed by five smaller health systems. Two of these smaller systems adopted an EHR only recently. The smaller health systems are meant to benefit from lessons learned by their predecessors. Periodic conference calls and a newsletter produced by CHIPRA demonstration staff are the main vehicles for imparting lessons.

Both of the larger health systems have earned incentive payments. One earned $120,000 for baseline reporting and $70,000 for improvements on four measures in the first follow-up year. The other earned $180,000 for baseline reporting and $50,000 for improvements in the first follow-up year, also on four measures. In both cases, the health systems had decided to pursue improvement projects only for measures with a baseline value of 89 percent or lower. Table 1 lists some of the QI strategies that practice sites implemented.

Table 1: Selected QI Strategies Pennsylvania Practices Used to Improve Performance

South Carolina used the Child Core Set Measures as a foundation for assisting primary care practices

In South Carolina, the CHIPRA project requires participating primary care practices to design, execute, and document plan-do-study-act (PDSA) cycles using the quality-of-care concepts established by the Child Core Set but adjusted to the time period selected for the PDSA cycle. In PDSA cycles, the practices define a quality improvement aim related to a Child Core Set measure, test an approach to achieving that aim, measure and reflect on the results, and then refine the approach in a series of short-term cycles. Practices document their PDSA cycles (at least three or four per quarter) on a project blog, along with minutes from internal QI meetings.

To educate practices about the Child Core Set and PDSA cycles (among other QI topics), the South Carolina CHIPRA demonstration convenes semiannual in-person learning collaborative sessions, and demonstration staff visit individual practice sites. Midway through the demonstration, a QI specialist was hired to advise practices on how to implement effective QI activities.

Each practice selects measures to address through PDSA cycles. As of December 2012, three measures accounted for half of all documented PDSA cycles. These were: developmental screening in the first 3 years of life, asthma-related emergency department (ED) visits, and preventive dental services. CHIPRA demonstration staff had reviewed 14 of the 24 Child Core Set measures during learning collaborative sessions by December 2012.

CHIPRA demonstration staff observed mixed performance across the 18 practices participating in the project. Some practices contribute fully to collaborative sessions and blog postings. Others meet minimum requirements. Some practices successfully use PDSA cycles for QI and transform successes into new office protocols. Others use PDSA cycles as a mechanism for documentation but fail to do the followup work to demonstrably improve, according to demonstration staff. Table 2 lists some of the QI strategies practices implemented.

Table 2: Selected QI Strategies South Carolina Practices Used to Improve Performance

Practices in both States grappled with clinical documentation limitations and worse-than-expected baseline performance

When health systems and practices began their projects, they commonly found they could not accurately assess their current or recent performance because of incomplete or inconsistent documentation in EHRs and paper charts. Before systems or practices could measure improvement, they first had to improve documentation and convey the importance of improved documentation to all relevant staff. Physicians who paid proper clinical attention to a matter were displeased when documentation did not match their performance. To the Pennsylvania CHIPRA demonstration staff, this realization is precisely the point of their project. One team member said, “Until you get your EHR up to speed with what you’re doing clinically, your EHR is falling short.”

Even when documentation was more complete, most practices found their performance was worse than expected on some measures. A South Carolina physician said, “Last year, a point of emphasis was developmental screening. We were already tipping our toe into that, but [the requirement to do PDSA cycles] helped formalize the process, and gave me an incentive to measure where we were. I knew what I was doing for developmental screening and I had told other people what we should be doing, but I didn’t really go and look. I think we all assume we are doing a really good job but until you capture those metrics you don’t know. It can become a big ‘ah ha’ moment.”

Friendly internal competition and teamwork were useful improvement strategies

Efforts by Pennsylvania and South Carolina to get health systems and practices to improve spurred a healthy rivalry among individual providers. For example, one of the Pennsylvania health systems believes annual measurement of system-level performance will not motivate improvement. The health system is designing a dashboard of physician-level measures, produced monthly or quarterly, expressly to promote friendly competition among physicians. In South Carolina, one practice leader who shared physician-level performance statistics among physician colleagues said, “There is nothing punitive about it, but I let them know in front of everybody that their ratio is low. That usually fixes the problem.”

The health systems and practices we visited said many QI activities require the involvement of physicians, nurses, and administrative staff to succeed. To be involved effectively, all staff must be aware of quality measures and why they matter. While some practices were struggling to increase awareness and teamwork, most mentioned these as explicit goals. A South Carolina physician commented, “Everybody has to understand that change is not one person’s job, it is the practice’s job.” In Pennsylvania, one practice’s success at making all members of its staff aware of its goal of increasing well-child visits led it to adopt changes suggested by administrative staff. Specifically, when office staff tried to make reminder phone calls to parents to schedule visits, they noticed that many parents had run out of cell phone minutes by month’s end and were not receiving calls or voicemails. Aware of the practice’s improvement goal, the office staff suggested making reminder calls earlier in the month.

Clinicians and States identified key ingredients to sustaining QI efforts

The keys to sustaining measure-driven QI efforts were:

• Investment in both the human and automated components of data extraction and reporting. To conduct QI activities efficiently, practices must be able to query EHRs to extract and report quality measures. In South Carolina, some participating practices still need to invest in this level of automation because their EHRs do not generate the reports they need. In Pennsylvania, by contrast, these basic functions were mostly automated, but investment in programming and analysis by humans continues. Each time an EHR is modified, for example, programmers and analysts must reconsider data coding and measure calculation and modify procedures. When it comes to sustaining QI efforts, not everything can be automated.
• Commitment to EHRs as the quality infrastructure. Regular and better use of EHRs is needed to make measure-driven QI activities the “normal way of doing things.” The need for an EHR focus was built into Pennsylvania’s pay-for-performance project, but several South Carolina practices also mentioned it. The idea is to make the EHR the infrastructure that ensures “best practice” care by building into the EHR functionality that supports appropriate clinical workflow and decision support. Otherwise, as a South Carolina practice explained, QI efforts are too easily disrupted when a practice must respond to events such as illness outbreaks or staffing changes.
• Broader family engagement. Many of the denominators of the Child Core Set measures include all eligible children. As practices began to accept this level of accountability, they took steps to promote patient awareness, knowledge, and compliance and paid more attention to communicating effectively with patients and families. Previously, some practices and health systems said they had been satisfied providing high-quality care to children who visited the office (as opposed to all eligible children).
• Reimbursement for delivering recommended services. Practices are most likely to sustain QI interventions related to services for which they are reimbursed. In South Carolina, the three measures most commonly used in PDSA cycles measure the receipt of services for which practices can claim reimbursement through Medicaid. In contrast, the CHIPRA demonstration staff had more difficulty getting practices to engage in motivational interviewing (to help persuade patients and families to modify behaviors linked to being overweight), despite offering training in the technique. Practices do not have a way to bill the South Carolina Medicaid program for providing that service, according to the CHIPRA demonstration staff.

Conclusions

This Highlight describes two projects that are pursuing similar ends but through different means. The Pennsylvania project sets its sights on EHRs as the infrastructure for quality measurement and improvement and rewards health systems that use EHRs accordingly. Participating health systems and practice sites decide for themselves how to bring about improvement. In contrast, the South Carolina project encourages but does not yet require EHR use. (Practices that originally contracted with the South Carolina project were informed that they must allow data abstraction from their EHR or use a data registry, but this requirement has not been enforced.) Moreover, CHIPRA demonstration staff members are directly involved in teaching practices to measure and improve quality by providing a set of collaborative and customized technical assistance supports and requiring the use of PDSA cycles.

Despite design differences, both projects use Child Core Set measures as a foundation for practice-based QI, and health system and practice staff in both States described similar experiences in becoming more measure driven. These staff found it initially discomfiting to learn that, on some measures, their past performance was not at the level anticipated and that documentation of clinical procedures was far from complete. But these staff also commonly described exhilaration that they were able to demonstrate improvement because of consistent measurement, teamwork, friendly competition, and changes in office processes. Finally, staff articulated what they need to sustain improvement—EHRs that facilitate documentation and measurement and help institutionalize best practices, ways to engage all families served by the practice, and reimbursement for recommended services.

Implications

The first Highlight in this series alerted States that testing the Child Core Set measures for practice-level reporting is a time- and resource-intensive task. This Highlight identifies several further considerations for States as they encourage health care providers to use the Child Core Set measures to support QI efforts:

• Letting providers select measures for QI interventions has the benefit of encouraging provider buy-in. To ensure that providers also focus on measures that align with State policy goals, however, States could consider offering financial incentives or modify Medicaid and CHIP reimbursement policies to cover high-value services.
• States must think carefully about what combination of technical assistance and financial incentives will encourage providers to become more measure- and quality-driven. South Carolina opted to provide extensive technical assistance but few financial incentives. Conversely, Pennsylvania opted to rely on financial incentives and provide relatively little technical assistance. Needs for technical assistance may be greater when the targeted primary care providers are not organized into health systems (i.e., standalone practice sites), have limited QI and information technology expertise, and little experience with an EHR.
• Using the Child Core Set measures at the practice or health system level means holding practices and health systems more accountable for all of the patients in their practice, not just those who come in for visits. The providers interviewed for this study were beginning to accept and adapt to that change. In addition, States should consider supporting providers’ efforts to increase patient and family engagement. This might include, for example, producing or disseminating educational materials to help families in Medicaid and CHIP proactively manage their health care.

Endnotes

¹ In January 2013, CMS published its first annual changes to the Child Core Set in State Health Official Letter #13-002. “2013 Children’s Core Set of Health Care Quality Measures.” January 24, 2013. Measures may be updated or retired by their stewards as technical specifications are changed, new clinical evidence emerges, or the measure’s performance changes. http://www.medicaid.gov/Federal-Policy-Guidance/downloads/SHO-13-002.pdf

² When planning its CHIPRA quality demonstration, Pennsylvania considered whether a pay-for-improvement project would be redundant with the Medicaid Stage 1 Meaningful Use Incentive Program established by the Health Information Technology for Economic and Clinical Health (HITECH) Act as part of the American Recovery and Reinvestment Act of 2009. It concluded that such a project would offer additional important benefits. These included having quality measures reported for both Medicaid and CHIP, exploring the feasibility of patient-specific reporting (not required by HITECH for Stage 1), and encouraging reporting on the full Child Core Set (as opposed to a subset of common Stage 1 and Child Core Set measures).