Employer Health Care Alliance Cooperative Adapts AHRQ-Funded Survey Tool to Evaluate Pay-for-Performance Program

The Employer Health Care Alliance Cooperative (The Alliance) in Madison, Wisconsin, a not-for-profit cooperative, used a modified version of an AHRQ-funded survey tool to evaluate its pay-for-performance program. The tool, called Physician Attitudes on Incentives (PAI-23), was developed by Gary Young, Principal Investigator for the National Evaluation Team for Rewarding Results at Boston University School of Public Health.

Under the Rewarding Results initiative, AHRQ, the Robert Wood Johnson Foundation, The Commonwealth Fund, and the California HealthCare Foundation all collaborated to fund, develop, implement, and evaluate seven innovative demonstration projects to examine the impact of physician incentives on addressing barriers to quality medicine. As early lessons have been learned under the initiative, expert evaluators at both the local and national levels have been working alongside the project leaders of these demonstration projects.

John Bott, Value-Based Purchasing Manager for The Alliance, first heard about the survey tool in an AHRQ Webinar. He was immediately interested in using the survey to evaluate his organization's pay-for-performance pilot program, which had been in place for two years. After contacting AHRQ, he was put in touch with Young, and the project blossomed. Young was happy that The Alliance wanted to use the survey and had no qualms about helping Bott adapt it for their needs.

"Because the survey had been created based on rewards at the physician level, and our program rewarded hospitals and medical groups, we had to modify it. Gary helped us adapt it to use with administrators who are responsible for contracting and quality. He approved all our revisions, to maintain the validity of the tool," Bott explains.

In August 2006, Bott conducted a phone survey of 15 key people in two medical groups and two hospitals participating in the pay-for-performance pilot project. Fourteen people participated, giving thoughtful responses. Respondents were uniformly pleased at the opportunity to share their views and appreciated being asked for their input.

Results were many and varied; for example:

• In general, respondents were supportive of pay-for-performance concepts of rewarding high quality care.
• Some thought the financial incentive was not high enough to affect behavior.
• Overall, no one thought the incentive caused any negative unintended consequences.
• Even within the same organization, opinions varied about whether working to get the incentive was cost-effective for the organization.

Some of the findings were a bit surprising, such as the following:

• The level of awareness among key people engaged in quality improvement about the pay-for-performance pilot program was lower than expected.
• Hospital staff perceived more difficulty than medical group staff in gaining cooperation within their organizations to reach the quality targets.
• The acts of quality measurement and comparing performance across providers were seen as important to improving patient care.
• Some respondents requested more frequent measurements and feedback; once a year wasn't often enough for them.

Based on the survey findings, The Alliance has built in several revisions to its pay-for-performance work plan. Because awareness of incentives and measures wasn't as high as hoped, The Alliance is working to engage key people who affect quality, specifically focusing on "more educating and awareness-building with physicians and those engaged in quality improvement," according to Bott. Other possibilities are more ongoing quality improvement activities and more frequent measurements and feedback.

Furthermore, Bott says, "We need to do more work to show provider organizations that there is a business case for working on quality: not only does the organization get the incentive, but improvement in many areas actually saves the medical group or hospital money."

Although the program will remain voluntary, according to Bott, The Alliance is designing an expansion of the pay-for-performance program, building on results from the pilot program. "We learned a great deal from this tool and plan to repeat the survey in the future," says Bott.

An abstract by Bott and Bert White, Rewarding Results Project Director, has been accepted for a collaborative presentation on the survey instrument and The Alliance's use of the tool in February 2007 at the National Pay-for-Performance Summit.

Founded in 1990, the Employer Health Care Alliance Cooperative helps employer-members manage costs and the well-being of their workforce. It has over 140 self-funded employers as members, representing over 75,000 employees and dependents throughout Wisconsin.