Health Plan Uses AHRQ Data and Surveys to Improve Service to Patients and Their Families

MetroPlus Health Plan in New York City is using CAHPS® data and surveys to improve the service it provides to patients and their families. To address some relatively poor scores in the State's biennial CAHPS Health Plan Surveys, MetroPlus launched an improvement plan which has had a ripple effect in improving care at the clinic level.

"Our member satisfaction numbers had been low for years on some of the composite scales," notes Debra Corbett, Senior Associate Executive Director of Quality Management at MetroPlus. "We wanted to understand what was going on."

While the State's CAHPS survey helped the plan identify a real weakness, it was not sufficient to help them address it, because it drew from only a sample of the plan's membership. To get the level of detail required to pinpoint the source of its performance problem, the plan needed to field the survey itself to acquire data on specific facilities. "We look at the CAHPS results as the gold standard in evaluating members' experiences," comments Ben Stein, PhD, a statistician for MetroPlus. "It provides a wealth of data."

In 2003, MetroPlus commissioned its vendor to run a survey of the plan's larger primary care locations. When they analyzed the data, MetroPlus administrators found evidence of both network-wide weaknesses as well as performance deficits in certain primary care facilities. Plan administrators then contacted the four facilities that had clear opportunities to improve their performance.

"We asked them to submit either an improvement plan or a proposal to pay for a facilitator who would help them run a quality improvement project," explains Corbett. Two facilities submitted proposals, including the Woodhull Pediatric Clinic, which is part of the North Brooklyn Health Network.

"What it [the CAHPS survey] said was not that surprising to us," notes Bernadette McKetney-Brown, Network Senior Associate Director of Managed Care at North Brooklyn Health Network. The clinic's administrators had some basic satisfaction data as well as anecdotal evidence of service-related problems, and were disappointed to learn that their efforts to address the problems had not made more progress. "But we had not seen CAHPS results before," admits McKetney-Brown, adding that she was impressed by what they were able to learn from the data.

The Woodhull Clinic began by asking the New York University (NYU) Center for Health and Public Service to delve more deeply into the survey results. The NYU Center conducted quantitative analyses and qualitative research that helped to identify both the factors that contributed to poor ratings and opportunities to improve. For example, the clinic staff learned that low ratings of doctors and nurses often reflected things surrounding the visit, such as staff interactions and the practice environment, rather than the actual professional interactions.

NYU also showed graphically how the clinic compared to other facilities. "That helped us see the data in a way that everyone could understand," McKetney-Brown says.

The next step was to act upon the data. The Woodhull Clinic contracted with the Primary Care Development Corporation (PCDC), a nonprofit agency that supports quality improvement initiatives in New York City targeting underserved communities. PCDC reviewed the analyses from the NYU Center. The CAHPS data were "much richer and more reliable than what [the clinic] had before," notes Patricia Simino Boyce, Director of Emerging Initiatives at PCDC.

Based on statistical analyses of the data, PCDC developed a plan to improve three specific indicators:

• The rating of the doctor/nurse.
• The helpfulness of office staff.
• The respectfulness of office staff.

In collaboration with a team from the clinic, PCDC devised a multi-step process aimed at creating behavioral changes that would improve the patients' experiences. The team consulted The CAHPS Improvement Guide and worked with a national expert in improving service in medical practices to identify appropriate best practices, organize training, and create actionable tools and strategies, such as scripts and patient hand-offs.

"We used rapid-cycle testing to get immediate feedback on the usefulness of those tools," explains Boyce. "We had measurement periods where we would rehearse what the staff had learned, using scripts and tools, and then survey patients to see what impact we were having." PCDC found that results were improving. "Patients were reporting a better experience," Boyce says.

The next step for the Woodhull Clinic is to roll out the pilot project to the entire staff. "Implementation is the hardest part of this work," Boyce notes. "You're trying to change people's behavior. That's a serious challenge."

McKetney-Brown says that the clinic is ready for the implementation process and has made progress in overcoming some of the obstacles to change. The clinic will track its progress on its own "patient report card," but the real test will come when MetroPlus surveys its members again in 2005.

MetroPlus provides health care services to New York City residents through New York State's Medicaid Managed Care, Child Health Plus, Family Health Plus and Partnership in Care, an HIV Special Needs Program.