This is the second event in a series of two web-assisted audio conferences on healthcare quality. These events are designed for state and local health policymakers and other decision makers interested in measuring and improving the quality of healthcare in the United States.
Cindy DiBiasi: Good afternoon. Welcome to Using Measurement Data to Improve Quality. This is the second event in a series of two web-assisted audio conferences on healthcare quality. These events are designed for state and local health policymakers and other decision makers interested in measuring and improving the quality of healthcare in the United States.
The series is sponsored by the U.S. Department of Health and Human Services Agency for Healthcare Research and Quality, often referred to by the acronym A-H-R-Q or AHRQ. My name is Cindy DiBiasi and I will be your moderator for today's session.
Healthcare quality is a timely and important topic in today's environment of rising healthcare costs. All players in the U.S. healthcare system are looking for ways to ensure that all Americans receive the safest, highest quality healthcare services possible when these services are needed. To be as effective as possible, it is critical that all decision makers in the healthcare process are able to measure, analyze and compare quality-related data. It is also important that the use of these data become an integral part of the quality improvement process. This will ensure the delivery of safe, high quality healthcare services to all Americans.
AHRQ has developed a number of tools to help state and local health policymakers, providers, purchasers and health plan and system decision makers better measure and improve quality. In last week's call, we discussed two recently released AHRQ reports on healthcare quality in the United States and some AHRQ-sponsored tools that can be used to benchmark and measure performance. Today we will look at the ways different stakeholders are using two of these AHRQ tools to actually improve healthcare quality and outcomes. Each of today's panelists represents a different group of stakeholders and each is actively engaged in using AHRQ tools to improve the delivery of healthcare services. The two tools that are the focus of today's conference are the CAHPS® and AHRQ Quality Indicators.
The CAHPS® is a family of tools for collecting, analyzing, and reporting information about consumer experiences in the healthcare system. The purpose is to provide reliable and valid information to help consumers, purchasers and policymakers assess healthcare system performance and choose among health plans. More information on the CAHPS® is available at www.ahrq.gov\qual\cahpsix.htm.
The AHRQ Quality Indicators or QIs are performance measures that are produced from hospital administrative data. They can be used to highlight potential quality concerns, identify areas that need further study and track changes over time. There are three sets of these measures: prevention, inpatient and patient safety. More information on the QIs is free and publicly available at www.qualityindicators.ahrq.gov.
Let me begin today by introducing our panelists. In the studio with me I have Joseph Anarella, assistant director of the Bureau of Quality Management and Outcomes Research for the New York State Department of Health; Dr. Michael Belman, staff vice president and medical director of Blue Cross of California; Bruce Boissonnault, president of the Niagara Health Quality Coalition in Buffalo, New York; and Vi Naylor, executive vice president of the Georgia Hospital Association. Welcome everyone.
Before we begin our discussion, I would like to tell the audience a bit about the format of this audio conference. First we will talk with our four panelists and then open up the lines to take your questions. We will give instructions on how to send your questions to us later in the program.
Although we don't anticipate that you will experience any technical problems, I would like to give you a few tips on dealing with them just in case they come up. If you are on the web and experience any problem viewing your slides, click the "Help" button in the bottom right hand corner of your screen to troubleshoot your web connection. If it appears that the slides are not advancing, you may need to restart your browser and log on again. If you experience any difficulty with the audio stream, you may access the audio by phone by dialing 1-888-840-0794 and use the password "quality tools". These dial-in instructions are posted on the right-hand side of your screen in the small, black box. If you are on the phone and need technical assistance, just dial "*0". For people who are listening by phone but watching the slides on the Internet, please be aware that you will notice a delay in the slide changes because they are timed to match the Internet broadcast.
Now I think we are ready to discuss today's topic. Let's begin with Joe Anarella, assistant director of the Bureau of Quality Management and Outcomes Research for the New York State Department of Health. Joe, I understand the State of New York has been using AHRQ tools to improve the quality of care delivered in your Medicaid Managed Care Program. Tell us about that program and how it is structured.
Joseph Anarella: Sure Cindy. New York State is at the very end of a six-year phase in of an 1115 Waiver. We have approximately 31 plans serving over 2.2 million enrollees currently, 20 plans alone in New York City. Enrollment plans ranges from several hundred to close to 200,000. As you can imagine, that has implications for the plan's infrastructure and their ability to measure and process health data. The plans are fully capitated. We have a pharmacy carve-out. We also exclude certain populations such as the SSI and HIV populations. We have just started a demonstration project for HIV-infected Medicaid recipients, the HIV Special Needs Plans, and those have started enrolling people over the last several months.
In addition to the fully-capitated programs, we have a Primary Care Case Management Program that serves upstate counties and enrollment in those programs is approximately 20,000. Then we have a few very rural counties up in the Adirondack area where we still have a fee-for-service program in place.
The second slide is going to show the change in enrollment over the last three years. As you can see here, we have basically tripled the number of Medicaid recipients we have enrolled in managed care plans. In March '01, we had approximately 700,000, two-thirds of which were children under the age of 20. Now in early '04, we have the 2.2 million I initially described and that makes us very close to 50-50 adults and children. With that change in mix, we are now able to look at different types of measurement tools that allow us to look more deeply at some adult measures and some chronic disease measures as well.
Cindy DiBiasi: Now what AHRQ-sponsored tools are you using in New York and how are you using them?
Joseph Anarella: We use a variety of tools to measure quality. I think our sophistication in using these tools has evolved over time. Back in 1994, we were using primarily HEDIS® measures with a few New York State-specific measures added on to address concerns of importance in the state. Over time we have added these other tools that you see on the slide here, including the CAHPS®, the Consumer Assessment of Health Plans Survey, which is what I will be talking about for the rest of the presentation.
The New York State Department of Health sponsors a bi-annual CAHPS® survey and we are just in the implementation stages of our third survey right now this February. We used the CAHPS® 3.0 for Medicaid tool. We deviate slightly from the prescribed standards of the CAHPS® measurement. We used slightly different, we sampled slightly different numbers. We sampled 750 adults and 750 parents of Medicaid children. We usually use an incentive and we actually did some incentive research the last time we did our CAHPS® survey in '02 where we offered some portion of the group $2, another portion $1 and another portion no incentive at all and we found we got the best response with our $1 incentive. That is just a little aside for those of you who are considering using a monetary incentive to boost your response rate. We found that the $1 worked the best.
We also delete some of the questions that we feel are very provider-specific. We don't feel some of them have, the plans have that much control over. Of course we add some New York State-specific questions. We had questions related to chronic disease conditions so we can, at the end of the survey, sort some of the responses by those conditions. Lastly, we enclosed an English and a Spanish survey when we mailed the survey to the recipients.
When the survey is completed, we feed back the data to the health plans so that they can review it and act on it for quality improvement purposes. I think Dale Shaller, your speaker from last week talked about the new quality improvement guide that folks at AHRQ and CAHPS® have been working on and we have looked at that and think it is going to be a great tool for the plans to use this year when the receive their new CAHPS® data.
We also used the CAHPS® data in something called the Quality Incentive. That is a program where plans can earn an additional 1% of premium if they do well on a series of HEDIS® and CAHPS® measures. The plans that score above a certain threshold are eligible to earn either a full percentage of additional premium or .75, .50, .25 or no premium. That has really, I think, been quite a motivator for some of the plans in the state. We used CAHPS® data in our public reporting. We have three major products. We have something called The Consumer's Guide to Medicaid Managed Care that we share with Medicaid recipients at the time their county is going mandatory so they have a little information on the plans that they must choose from. We post the data on our website under something called "Equar". We also furnish, provide the data in a hard copy managed care report that is available through our office and it is also posted on our website.
We use the data in a quality matrix, which I will describe a little bit more in a minute, but it is basically a method to prioritize improvement areas for health plans. A future use for the CAHPS® data is to use it in our auto-assignment algorithm. I will just briefly describe, for recipients who don't choose a plan when it is time for them to enroll, they are auto-assigned to a plan. We have a formula in New York that allows us to consider quality in that auto-assignment algorithm. We had been using, up until now, just quality measures but this year we hope to add CAHPS® wait to that algorithm as well.
Cindy DiBiasi: As you have been using these tools, have you noticed any overall improvement in HMO performance?
Joseph Anarella: We have. I think we have seen the CAHPS® performance improve slightly and again we are measuring every other year, though I think this will be a (unclear) year and we will have some new data. We have noticed a lot more interest from the plans on part of the CAHPS® data so I think that the interest comes first and then you see the improvement over time.
What this slide that is currently on the screen shows is the quality improvement matrix. If you can imagine a plan receiving the data back in this format, we placed the majority of the measures they report on, the HEDIS® measures, in this matrix and one of the CAHPS® measures. Actually it is a composite measure. There is a trend factor for the plan from year to the next. Then there is a place that the measure is also placed where they compared to the statewide average. So if you can imagine a series of HEDIS® and CAHPS® measures in this matrix, it will give you an idea of the feedback we provide to the plans. The CAHPS® measure we currently use is a composite measure. It is called "Problem with Service". The three questions from that are: Was the problem defined or understand info on written materials? Was it a problem getting help when you called Customer Service? Did you have a problem with any paperwork? For the plans that score on the, in the DNF category, those plans owe us an action plan to describe what barriers they think are causing the score and what plans they have in place to correct it.
Cindy DiBiasi: What types of changes have the individual plans made?
Joseph Anarella: I can give you an example of a plan who scored low on this problem of service measure and presented us I think a pretty comprehensive plan for addressing it. They laid out a five-point plan. The first area they were going to address is they were going to increase the number of customers service representatives and train those representatives to more quickly assess the nature of calls and assess the next steps that needed to be taken to address the concerns.
The second action they were going to take was to broaden the range of language capabilities of staff performing the Q&A and to increase the number of calls audited in additional languages. As you might imagine in New York City, where this plan is located, they serve a very large number of recipients with different language capabilities.
The third thing they were going to do was increase their provider maintenance file and update provider information on a more timely basis so members wouldn't be calling doctors with full panels and with offices that were closed to new enrollees.
The fourth action was to develop and run a member advisory board. Their advisory board was actually going to start reviewing some of the materials and some of the information packets that new members received when they enrolled in the plan.
Finally, they were going to develop and execute a training curriculum for all their customer service representatives and increase the frequency of staff meetings and communication with their customer service representative. As I mentioned earlier, we will see shortly if these actions have impacted their scores.
Cindy DiBiasi: How do you know it is the use of the CAHPS® tool that is motivating the plans to improve the quality and stimulate the changes?
Joseph Anarella: Well, we actually did a survey a little less than a year ago and asked the plans what motivated them to act on some of the issues that we presented to them in the matrix and what department-sponsored events actually caused them to work on certain low measures. They identified a number of things the department does, including the Quality Matrix, which as I mentioned, has a CAHPS® component. The Quality Incentive, which also has a CAHPS® component and the auto-assignment algorithm. So each of these major initiatives coming out of the department feature a CAHPS® piece to them so I think we infer that plans are interested in improving those scores and to do that they have to improve the CAHPS® scores.
Cindy DiBiasi: OK, and just a final note. More information about New York's use of the CAHPS® may be obtained directly from Joe and he can be reached at the phone number and E-mail address to be shown. Information is also available on the web at www.health.state.ny.us and Joe, we will be back to you for questions.
But first we are going to move on to Dr. Michael Belman, staff vice president and medical director at Blue Cross of California where they have been using the CAS, which is based in the CAHPS®, as one of the bases for a quality incentive payment to providers. Michael, you have got quite a unique program.
Dr. Michael Belman: Yeah, we have been in the performance measurement business for close on ten years now and we have done this in our Blue Cross of California plan, which is the California subsidiary of Well Point Health Networks. We have had a report card in place for our medical groups for a considerable period of time. In 2002, we expanded this report card to include additional measures and also linked it to a greatly increased financial incentive, which is paid to the medical groups.
In 2003, we introduced a PPO physician quality incentive program, which is a web-based program that reports information back to individual physicians on the web. This information is accessible only by the individual physician. This too is part of a pilot program funded by The Robert Wood Johnson Foundation and the first incentives will be in fact paid at the end of the first quarter of this year.
In general, our goal has been to emphasize the business case for quality. We know that there are very good medical groups that function in California and the feeling is that by linking performance measurement and financial incentives that this can greatly amplify the case for quality and in fact increase the investment within the medical group to pursue quality performance.
Cindy DiBiasi: Now what AHRQ tools are you using in your pay for performance program?
Dr. Michael Belman: One of the AHRQ tools we use is the CAS, Consumer Assessment Survey, which is a derivative of the CAHPS® survey. This survey is directed more at the individual medical group level rather than the health plan, which was the original design of the CAHPS®. So the survey is administered annually to random samples of members of medical groups and information specifically related to the medical groups is elicited.
The four domains that are included are similar to what is derived from the CAHPS®. That is, getting needed care, getting care quickly, physician communication and overall rating. We have found that this provides useful information about the groups. It is also, the information is presented to members of Blue Cross through our website and is also presented through a website that is sponsored by the Pacific Business Group on Health that has a website that presents this information to the public.
Cindy DiBiasi: Now I understand that you have combined the CAHPS®/CAS information with other information to assess different aspects of the physician group performance. How and why do you do that?
Dr. Michael Belman: That is right. We use the CAHPS®, and that comprises close to 40% of the total of our report card, but in addition we use other measures which include clinical quality measures, which includes many of the well known HEDIS® measures, the preventive healthcare and chronic disease indices, which are well known I am sure to everybody.
We also reinforce the need for certain administrative service performance interactions between Blue Cross and the medical groups through measures geared to facilitate our collaboration. We have another measure, which actually provides an additional point score to those groups who institute individual physician incentive programs within the medical group. The California model, as it is well known by many, is with a health plan contract with medical groups rather than individual physicians. In order to drive the incentives and the information down to the individual physician level, we found it helpful to provide additional points to those groups that use similar types of measures within their medical groups so that they can then provide individual incentives to the physicians. That way I think we are providing a much stronger link between the physician, the medical group and the health plan.
Cindy DiBiasi: How do you know this program is making a difference?
Dr. Michael Belman: Well, I think we have seen some very encouraging responses to the introduction of our enlarged program and our enlarged financial incentives. As I mentioned, part of the, one of the goals is to emphasize the business case for quality and we have seen, for example, the introduction of open access programs in several of the medical groups during the past year. Open access programs emphasize easy access of members to physicians, usually aiming for same-day appointments. This is something that requires a great deal of organization within a medical group, but with the added incentive that is provided to improve member satisfaction, we think this is an appropriate incentive that the medical groups have used in order to justify the time and effort in order to introduce these programs.
We have also seen an improvement in the HEDIS® scores and also I think what is encouraging is an improvement in the information technology investment, which is critical to improving medical group performance. That also is rewarded within our program.
Our program is also, I should mention, in synergy with the integrated healthcare association pay for performance program, which is an all-California health plan program. Six major health plans in California collaborate to use a common scorecard, which makes life a lot easier for the medical groups and this pay for performance program is in fact also funded through a Robert Wood Johnson grant and is currently being evaluated for its impact.
Cindy DiBiasi: What has been the reaction of physicians and other stakeholders to this payment method?
Dr. Michael Belman: The reaction has been positive. It has increased, I think the acceptability has increased probably five or six years ago when the scorecards were newer and the incentive was smaller, I am not sure that many physicians were aware of it. With the fact that there is a bunch more expanded scorecard together with a significant incentive, and by "significant" I mean that we are probably now paying something in the order of about 5% of the professional capitations of a medical group. In fact, Blue Cross of California paid close to $30 million in bonuses last year based on quality to the medical groups.
I think people have recognized that these measures are of importance to the delivery of healthcare to the members and now they are seeing as well that they can benefit within the medical group. I think they are also seeing that the CAHPS® member survey is reliable and that it provides good, objective information. We make a point of actually circulating the survey to our medical groups prior to the time that it is mailed out to make sure that physicians are aware of the types of questions that are in the CAS survey. Now more physicians actually know what the questions are. This may be somewhat seen as "teaching to the test", but it does indeed help physicians understand what the public ranks as important. Also, before many of our expansions were put into the program, we did get input from physician leadership within California.
Cindy DiBiasi: OK, well Michael we will be back to talk more about this. But to view the results of BCC's efforts with the CAS, go to www.bluecrossca.com. Once you are there, click "Find a Provider or Hospital". Then follow the link on the left-hand column to "Provider Performance Profiles". More information is also available at healthscope.org the website of a Pacific business group on health.
Let's turn now to Bruce Boissonnault. Bruce is the president of the Niagara Health Quality Coalition based in Buffalo, New York. The Niagara Health Quality Coalition is comprised of employers, providers, physicians and insurers, all working together to improve regionalized healthcare quality and maximize cost effectiveness. Bruce, why don't you talk about the project a bit?
Bruce Boissonnault: The Niagara Health Quality Coalition in collaboration with another organization, The Alliance for Quality Healthcare, publishes the New York State Hospital Report Card, whose goal is not merely publishing or accountability. Really, the goal is to improve care continuously in a collaborative model.
As I mentioned, one of the stakeholders is The Alliance for Quality Healthcare. That alliance is comprised of 3,600 businesses and 35 health plans, a member of the Institute of Medicine Report on Medical Errors, healthcare consumer groups and probably the nation's leading actuarial healthcare consulting firm. So it is quite a good group.
The analysis is performed in our shop with help from some of those stakeholders. The goals, really we want the data to serve a couple of purposes. One, we want the data to be useful to patients in their search for high quality healthcare. We want public policy decision making to improve and not be based just on cost. If only cost data are available, of course all of your public policy decisions will be based on that. We want to avoid the confusion created by multiple reports and by having virtually every health plan of consequence in the State of New York as part of this collaboration. We don't have every health plan doing their own separately and differently. We also work very closely with the hospitals to complement their own internal quality improvement processes statewide.
Cindy DiBiasi: Which of the AHRQ quality improvement tools did you use for this project?
Bruce Boissonnault: We used all three: the inpatient, the prevention, which are the community-wide quality indicators; and the patient safety indicators. We do not publish the patient safety indicators, but instead share them with the hospitals on a selective basis quietly. If there are questions, we can get into why. But basically, there are perverse incentives if you publish the patient safety indicators in that you might keep people from reporting things that they should, if those became widely published.
Cindy DiBiasi: Let's talk about some of the benefits of publicly reporting health quality data.
Bruce Boissonnault: We were the first in the country to do this. We did it collaboratively with Texas and we were surprised at some of the benefits we found. One of the things that we hadn't considered was the benefit of every hospital in New York having the ability to just pick up the phone and say, "You know, I am looking at your data." For another hospital in the state, they can look at every other hospital. They can pick up the phone and they can say, "I am looking at your data for heart attack. Can you tell me how you are doing that?" We do know that there is a good deal of that happening. So the hospital-to-hospital comparisons being published actually were very helpful.
Cindy DiBiasi: And the hospitals have been cooperative in that regard?
Bruce Boissonnault: You know, we think it is going to be a work in progress. We know that many are doing it and we think it is a growing trend. Of course, we know from the Hibbard Study that in Wisconsin what they did is they published data similar to these indicators in one-third of the state and released them publicly. In another third of the state, they published them, sent them to the hospital CEO and in another third of the state they did none of those. They found that you got about twice as much quality improvement activity when the data were provided to the hospital CEOs, but you got about twice as much more than that when it was publicly released. So you get a four-fold improvement by publishing in the Hibbard Study. We are seeing the same sort of things anecdotally. Hospitals want to serve the communities that they are in and so they also want uniform information about quality, which I think is one of the real advantages of this methodology.
I think it is fair to say that some of the hospital board meetings are more interesting now that the board members are empowered with uniform measures that are publicly available of specific quality indicators. Some of the hospital CEOs have asked that their compensation be tied to these measures. Basically the boards are saying, "Tell me about this" and the hospital CEOs are saying, "I am going to work on this and I am going to make a big difference and I would like to be compensated for it." In fact, in our region, virtually every hospital CEO has some of our measures tied to their compensation. That flows right through the organizations.
Cindy DiBiasi: What are some of the ways the quality data has been used to improve service delivery?
Bruce Boissonnault: You know, it is the first year we have published so we can't give you the "this year versus last year" comparison. We were the first state, as I said, along with Texas and the country to do it, but I can give you a couple of anecdotal examples that I think make the point.
We have been working closely with hospitals around the state since the late 1990's. I remember when the gall bladder measure first came out. It is the laproscopic colosusectomy measure. When that first came out, there were a group of hospitals that were tied together under a corporate umbrella that we noticed had very low rates of using this less invasive type of surgery and safer type of surgery for gall bladder removal. So I picked up the phone even before the data were published and I called one of the senior medical people. That senior medical person said, "Publish the data. I have met with the chief of surgery and the chief of surgery has told me that we don't need to learn anything. We are as good as any other hospital." "And frankly", he said, "by publishing this, it will make my job easier as a senior medical officer with the institution to get this addressed."
Another example was in acute stroke, which is one of the inpatient quality indicator measures as well where in yet another part of the state we noticed that one of the stroke centers where they were actually diverting ambulances and making patients wait an extra 20 minutes so they could go to the stroke center and yet that stroke center's outcomes were statistically significantly worse than the state average. Our business leaders mentioned that in a non-confrontational way to the leadership of the hospital. In some quiet dialogs, we learned that 24/7 radiology had been discontinued at the stroke center. It was repaired within a number of weeks, maybe even days after it came to light that it was having this kind of impact.
Cindy DiBiasi: Now this seems like really powerful information. Do you know if consumers are using this information?
Bruce Boissonnault: You know, believe it or not it is a raging debate in healthcare whether consumers use these data. I always have to laugh because when we published this report in November of 2002 for the first time, we had approximately 15,000 discrete users. That is people, not hits, per hour for three million hits a day on our website. So one of the things I think that is really important about this is by having uniform measures that are credible and that are widely understood, people, once they get used to using these measures, will use them.
The other thing I will say is part of the formula is to get the kind of coverage you need and we received coverage in places like The New York Times and National Public Radio.
Cindy DiBiasi: Now you must have faced some challenges. How did you make all of this work?
Bruce Boissonnault: One of the biggest challenges anyone using these data in public reporting is going to face is the sort of chicken and egg issue of gee, patients don't know that these data even exist so how could they know how valuable they could be in their search for high quality healthcare? The other thing is, even when they start to use them, they might think that they are more perfect than they are. We tell folks at the beginning and the end of almost every conversation, take these data to your doctor. Have a better dialog with your doctor because there are no perfect data. Your physician is on your side and so you are going to want to have a dialog with someone who knows the system and knows where the data might not be telling you the whole story. So the chicken and the egg issue is there as far as people recognizing the value in the data.
It is also there in that you are using administrative data and I will be honest, we have found that the problems were much smaller than we expected in both New York and Texas. But there were instances in some, I would call them narrow frames, where the hospitals were reporting their billing data in a way which maximized their billing rather than maximized, for example, the accuracy of what they wanted to report. That washes, both of those issues washed out very, very quickly. So how we met that challenge was by getting it published. The sooner you do it, the sooner those problems wash out. There is no substitute for actually doing the report.
If you are not going to outsource this, there are, you do have to have some systems expertise and if you are going to bring all of that expertise in, you are going to have to hire someone and buy computers. So you may end up spending a fair amount of money, but you can shortcut that by working with someone who already has some of the capacity. We have been asked by the states of Florida and Colorado to help them through their first couple of iterations of public reports they are intending to do. We do it for just a few thousand dollars.
The last thing actually we expected to be a bigger challenge than it was, we really expected the providers to be concerned about this. Actually, in New York State, they, I don't want to say they were wonderful, they weren't active partners but they, our hospitals embraced the use of the measures and I don't think anyone was threatened by them. As long as you are careful to note their limitations.