Quality Improvement: Tools to Benchmark and Measure Quality. Transcript of Web Conference (continued)

Cindy DiBiasi: Now Dale, let's talk some more about some of the data analysis and the quality improvement strategies that are included in this guide.

Dale Shaller: OK, at the front end of the guide, we have spent some time in Section Two walking through a series of analytic steps that a plan or a medical group might want to first do before they really know where to hone in and focus their improvement efforts. So the first chart here that we show is a benchmarking chart that basically describes how a health plan might compare their own results in a CAHPS® survey to a national or local comparison norms. This happens to come from the National CAHPS® Benchmarking Database.

The next slide shows another step in the analytic process, which is a kind of a focusing step. It is sometimes called the key driver analysis where you can figure out which of the domains of the CAHPS® survey are most likely to yield you the biggest bang for the buck. So what this chart does is shows you the relationship between things that correlate highly with overall ratings of care and your sort of measurement, your performance on those specific attributes. The basic idea here is if you end up in the upper left hand quadrant of this matrix, that is where you are doing poorly on areas that are highly correlated with overall ratings and so that is probably where you want to focus your attention.

Finally, this next slides illustrates in a little bit more detail this idea of the improvement process cycle, which really does suggest that quality improvement isn't something that you just do and then you are over. It is a continuous process. You start it and you start it again and you take small steps and you do small tests of change. What this chapter does or this section of the guide is walk through organizations on how you do that improvement process. Again, specifically with reference to the CAHPS® survey and drawing on a lot of case examples of how plans and how medical groups have actually done this in practice to kind of balance people in the real world.

Cindy DiBiasi: Well Dale, specifically, how can someone improve the CAHPS® score?

Dale Shaller: They are listed in the guide in the section that talks about specific intervention. Over a dozen strategies that correlate to the CAHPS® core questions that are measured on the health plan survey. Some apply to the plan levels, some apply to the group level, some apply to both plan and medical group levels and each one of these specific strategies has a description of the problem that they are designed to address, what the purpose of the strategy is, pros and cons of using it, evidence that it actually works and a list of resources and literature that plans and practices can consult for further information.

One specific example drawing from the area of doctor communication, which is actually one of the areas that is most highly correlated to overall ratings of performance and is actually one of the areas that is most difficult to improve, lists seven very specific steps that a health plan or medical group might take to improve doctor-patient communications. There is a whole list here of both kinds of strategies such as training of physicians to help them improve their communication skills. Tools to help patients become better communicators at their end when they visit the doctor giving them for example, things like "Doc Talk" cards, stuff to take into the visit, Coached Care Program, things of that nature. Shared decision making strategies, support groups and self care programs, how evidence-based information could actually be delivered to patients for their use in the care process using modes of communications such as the Internet. Finally, things that are known as planned visits or group visits which proactively create opportunities for patients, especially those with chronic conditions to learn about ways to self manage their care and work in a team way with their physicians. All these things are actually designed to improve that doctor-patient communication which is so important. Leads to higher scores on the patient satisfaction, patient ratings and really, very importantly, leads to higher scores on doctor satisfaction because one of the things that we find is that doctors are as beleaguered as patients in the system today. Anything that they can find that helps them manage better will improve their quality of life and their satisfaction on the job and lead to better quality of care.

Cindy DiBiasi: Dale, how is the guide now being used?

Dale Shaller: Because we just published it in the fall of last year, we don't really know. But we do know that CMS distributed it to all of their Medicare/Managed Care plans, all of their regional offices in both the print version and the CD-ROM version. We have presented it at least a couple of major national conferences. The IHI Forum, which is a big national event. We had a learning lab there. We are doing newsletters. We are doing an audio conference like today.

One of the most interesting early applications of the guide is underway in a collaborative in Minnesota through an organization called the Institute for Clinical Systems Improvement where we have eight medical groups from the State working together. They are testing a new version of CAHPS® designed for use at the medical group level to give them a baseline sense of what their performance is and then they are using the CAHPS® Improvement Guide as part of a twelve-month collaborative where they are going to identify and work on specific aims that are going to be supported, at least in part, by the resources that are included in the guide.

The guide is available for downloading at the Website that is indicated on the screen. We are very interested as people have the opportunity to download and use the guide to get their feedback on ways that we can improve this because we are sort of trying to follow our own advice here and do some improvement as we go. We consider this the first version of the guide and based on our experience with it in the field, intend to improve it and eventually develop it into a web-based tool so that it can be rapidly updated and made actually for interactive use by plans and medical groups seeking to improve their performance.

Cindy DiBiasi: Great. It sounds like you have got a lot of wonderful ideas. We are going to come back and ask you some more questions about it. But first we are going to open up the lines for questions from the listening audience. Let me tell you how to communicate with us. There are two ways you can send in your questions. The first is by telephone. If you are already listening on a phone, press "*1" to indicate that you have a question. If you are listening through your computer and want to call in with a question, dial 1-888-496-6261 and use the password "AHRQ tools." Then press "*1". While asking your question on the air, please do not use a speakerphone or cell phone to ask your question. Please be sure to speak loudly and clearly and if you are listening through your computer, it is important that you turn down the volume after speaking with the operator because there is a significant time delay between the Web and telephone audio.

If you want to send a question via the Internet, simply click the button marked "Q&A" on the event window on your computer screen, type in your question and then click the "Send" button. One important thing, if you prefer not to use your name when you speak with us, that is fine. But we would like to know what State you are from and the name of your department or organization so please provide those details regardless of whether your question comes in by phone or the Internet.

As you are formulating your questions or queuing up on the phone line, I would like to say a few words about our sponsor. The mission of AHRQ is to improve the quality, safety, efficiency and effectiveness of health care for all Americans. The operating component producing this series of audio conferences is AHRQ's User Liaison Program. The ULP serves as a bridge between researchers and State and local policymakers. ULP not only brings research-based information to policymakers so that you are better informed, but we also take your questions back to AHRQ researchers so that they are aware of priorities at the State and local levels. Hundreds of State and local officials participate in ULP workshops every year. ULP hopes that today's Web assisted audio conference and the remaining event in this series will provide a forum for a productive discussion between today's audience and researchers. We'd appreciate any feedback you have on this Web assisted audio conference and at the end of today's broadcast, a brief evaluation form will appear on your screen. Easy-to-follow instructions are included on how to fill that form out so please be sure to take the time to complete the form.

For those of you that have been listening by telephone only and not using your computer, we ask that you stay on the line. The operator will ask you to respond to the same evaluation questions using your telephone keypad. Your comments on this audio conference will provide us with some valuable tools in planning future events that better suit your needs. Also, you could E-mail your comments to the AHRQ User Liaison Program at info@ahrq.gov.

Let's now go to some questions from the audience and from Maryland we have on the phone, Linda Bartanista. I hope I said that name right, Linda. I'm sorry.

Linda Bartanista: That's OK.

Cindy DiBiasi: Do you want to repeat it so your family will know who you are?

Linda Bartanista: Sure. It is Linda Bartanista.

Cindy DiBiasi: Good. OK, thank you. You have a question?

Linda Bartanista: Yes, actually I have two questions. I spent some time yesterday perusing the Website and the first question I have is for Dr. Kelley. I was trying to find the workbook for Crossing the Quality Chasm for State leaders and diabetes care. I couldn't find it anywhere.

Dr. Ed Kelley: Hmmm. Well there is a really good reason for that and that is because it is not done yet. Typically we like to talk about things we haven't finished yet (Laughs.) because by the time we finish them, people have forgotten about them. The workbook is in draft form right now and we are hoping that it will be available within the next month or so. It has been discussed and I should mention we did produce this in partnership with Medstat and also with the Council of State Governments and they have done some meetings at their annual convention and so we have gotten some input and we have had some partners weigh in on it, but the final form is not yet available but it will be up on the Quality Tools Website as soon as it is.

Cindy DiBiasi: You are saying the next three or four weeks.

Dr. Ed Kelley: Right. That is what we hope.

Cindy DiBiasi: Or else we are going to give everybody your cell phone number.

Dr. Ed Kelley: (Laughs.) Right, exactly. I think I should mention probably three or four weeks might be a little aggressive. That is when we hope to finish it at AHRQ. I think we were planning on it being available spring of this year.

Jean Slutsky: And this is a plead to sign up for the What's New? Update at Quality Tools because then you will get an E-mail letting you know what has been added new to the Website.

Linda Bartanista: OK because we are putting out an RFP right now that relates to a project getting baseline measures for diabetes care. My second question is I was also a little confused about the difference between the National Quality Measures Clearinghouse™ Website and the Quality Tools Website. They seem to get the same information from both so maybe I am not kind of clear on what the difference is between the two.

Jean Slutsky: That is a great question. The National Quality Measures Clearinghouse™ is a clearinghouse just for quality measures whereas Quality Tools is a clearinghouse for all sorts of tools of which the Quality Measures are one part. So if you are looking for a much broader subset, a broader group of tools, Quality Tools will encompass not only quality measures, clinical practice guidelines, tools developed by the agencies such as the Child Health Tool Box, HCUP Quality Indicators; it is a much broader group.

Linda Bartanista: OK, thank you.

Cindy DiBiasi: Thank you. From Missouri, we have Ann Mangeldoff on the phone. Hello?

Ann Mangeldoff: Hello. Yes, I am calling from the Arthritis Foundation and one of the things, and I don't know exactly whom my question is towards. Probably both Dr. Kelley and Ms. Slutsky. A lot of the information that has been shared has really been focused on findings as well as on targets specific to health care providers or health care provider systems and I am wondering if there are; I know there was some brief reference to consumer information on the Website, but I wondered if you could speak a little bit more to what consumers, what part consumers can play in this process?

Jean Slutsky: That is a great question too. If you were to go to the Quality Tools Website, you will notice that you can enter the Website as a consumer or a patient. If you were to do that, it would actually give you access to many tools that have been sponsored and developed by the Agency that are targeted explicitly at patients.

Ann Mangeldoff: Those tools can be utilized by anyone who wished to use them?

Jean Slutsky: You bet. They are actually tools that we make available to you free of charge.

Ann Mangeldoff: Great.

Cindy DiBiasi: From Georgia, we have Patricia Haley on the phone. Hello?

Patricia Haley: Hello. I am from Atlanta, Georgia with the Division of Aging Services and the question that I have is for Ms. Slutsky. Can you tell me a little how these tools can be used in measuring performance in quality in homecare services, like home and community-based services?

Jean Slutsky: I am going to let Ed talk about the measurement information, but in terms of the tools that are available that are not measures, they actually help you once you have identified a problem or you want to avoid a problem in quality. Ed can talk to you about how the measures can be applied to improve quality.

Dr. Ed Kelley: Right, it is a good point and we didn't have to much time to get into specific areas of the measurement that the reports get into, but the reports do deal with home health care and nursing home care. We draw from a broad group of measures that come from the minimum data set for nursing homes and from the Oasis data set for home health care. Those have had, we put together a technical expert panel on measurement that would be applicable for home health care quality.

So at this stage, we have begun tool development off of the reports and we started with diabetes, but I think that we are kind of considering that we are going to get some input on what future tools and workbooks might be perhaps in the offing. There is one on home health care, but for right now the tools that are available on the Website include mostly ones associated with measurement. There is the ability with the data that is in there, like I said, the final version of that will be available shortly to look at national averages as well as State averages to see how Georgia, for instance, compares with other States in our region and with other States around the country.

Patricia Haley: OK, thank you very much.

Cindy DiBiasi: Thank you and before we take the next call, just a reminder not to use a speakerphone. We can hear you much more clearly if you just pick up the handset.

Our next caller is from Pennsylvania. Fred Ferguson. Hello?

Fred Ferguson: Hi. I have a couple of questions for you. First question is, we are listening to about a lot of the reports that are provided by AHRQ and the information, but there are some questions regarding actually what is the true problem and it is people treating people. I would like to know how your programs are training people to actually treat people because in areas of EMS and health care in general, that seems to be an ongoing problem.

The second question is, and I believe this is an American question in comparison to the rest of the industrialized world, is that Americans are very poor in prevention and we like to respond afterwards and actually like to have a quick fix for the problems that we have. I would like to know what areas of prevention that AHRQ is trying to improve upon so people can respond to that?

Jean Slutsky: Those are just wonderful questions. I am going to actually combine your question into one if I may. Slipping the prevention question into the people treating people question, the Agency sponsors the U.S. Preventive Services Task Force, which actually makes recommendations on clinical preventive services. I don't know if you are familiar with those, but they are evidence-based recommendations that really guide clinicians in providing the most accurate evidence-based preventive services to their patients and we actually have quite a program in trying to disseminate that information to providers. We have The Put Prevention into Practice Program as well as making these recommendations available through the Quality Tools Website, the Guideline.gov Website and through various other venues. I think you are quite right that people treating people is very important that we make sure that health care providers are aware of the best available evidence on how to base their treatment recommendations and their policies.

Dale Shaller: Can I just add I think that that question is also, it is not really a stretch at all to say that The CAHPS® Improvement Guide is an answer in part to that question because it is about systems of care, treating their patients and improving the patient experience of care, which we know through practical experience as well as studies that there is a pretty high correlation between patient satisfaction with their care and clinical outcomes. I think that combined with sort of the technical evidence and clinical measures of care really does provide the professional providers of care with some resources to actually improve the way that they treat people.

Cindy DiBiasi: Dale, a question for you on The CAHPS® guide. "Is the CAHPS® guide also intended for Medicaid SCHIP Managed Care Plans?"

Dale Shaller: Absolutely. Even though CMS, the Medicare program was the funding agency for it, they did not want us to limit our scope to just Medicare plans. So the kinds of strategies that are outlined in the guide are applicable to the Medicaid, the SCHIP environment, the commercial environment and as I mentioned earlier not just at the plan level but oftentimes it is a medical group level which tends to not distinguish between payer source in terms of improvement strategies. So yes, it has broad applicability to programs outside of Medicare.

Cindy DiBiasi: And a question from Rosemary Fields. "Can the CAHPS® guide be accessed on the AHRQ Website?"

Dale Shaller: That is a question that I don't know the answer to.

Cindy DiBiasi: Looks like you have stumped the panel, Rosemary.

Dale Shaller: We know that it can be accessed through the Web URL that we gave you on that slide.

Jean Slutsky: We are actually working to get all of the CAHPS® tools available through qualitytools.ahrq.gov.

Cindy DiBiasi: OK. From Rhode Island on the phone we have Tricia Washburn. Hello?

Tricia Washburn: Hi. My question is for Dale Shaller. The CAHPS® survey, it is noted in the slides that they are going to be under development for hospitals, medical groups and individual clinicians. I was wondering if there was some timeframe on when that would be developed and if we could access that through the Website?

Dale Shaller: Yes. We can provide information through the same URL that was given for The CAHPS® Improvement Guide. We actually embedded in what is called the SUN Website. That stands for Survey User's Network. So once you are there, you are able to, through that Website, look at the various other CAHPS®-related tools that have been developed or are under development. One of the major efforts underway right now is in the hospital sector to develop a hospital CAHPS® survey. There was a major push by CMS over the last year to get that done. It is moving forward.

At the medical group level, the example I mentioned a little bit earlier of the collaborative on quality improvement in Minnesota is using one of the first generation surveys designed to take CAHPS® to the medical group level. We have actually designed this to address individual clinicians and we are rolling that information up to look at an overall medical group score. Items that are similar to what you see in the health plan survey on some of these earlier screens, but go beyond that to questions related to coordination of care, particularly office management systems.

What is really important about this in terms of again our commitment to improvement not only of things related to what CAHPS® measures but the CAHPS® surveys themselves is we had a lot of input in the development of that group level survey by the individual medical groups that are participating in the collaborative. As a result of that, it is a hugely improved survey that they like and because they like it they are using it and once you embed it on an ongoing basis in their improvement process so it becomes not just a one-time thing. You measure CAHPS® scores once a year and you go away and you go back and measure it another year. What we are moving to with applying CAHPS® in the context of quality improvement is a combination of doing sort of a baseline assessment and then some periodic routine surveys, small samples, continuous collection and examination of the data so it becomes much more embedded in the improvement process so there is a lot of work underway at the hospital level, the medical group level.

Timelines are hard to predict, but I would say by the end of this year we will have something close to a fully-developed embedded instrument for the medical group level that can be widely accessed. But we are not waiting for that to happen before people are encouraged to use what we currently have. So if you have any further questions about that, you can reach us through that SUN site and we'd be happy to follow up with you and encourage your use of some of these early tools.

Tricia Washburn: Thank you very much.

Dale Shaller: Thank you.

Cindy DiBiasi: Ann Mangeldoff from Missouri is on the phone with a comment. Ann?

Ann Mangeldoff: Yes, I just wanted to say that in addition to health systems and providers and such that I think this is very, very important information to share with providers-to-be through medical schools and such because I think always coming in on the back end makes it difficult and getting people oriented to the ideas of quality improvement right from the start could be a help.

Cindy DiBiasi: All in agreement on the panel, I think. A question for Ed from Illinois. "How does the NHQR differ from the National Quality Forum Guidelines?"

Dr. Ed Kelley: That is a good question. I think that gets to the heart of this measurement issue. For those who work in a quality measurement field, sometimes it feels like we are drowning in alphabet soup and drowning in measures and there is always another group that has another measure that is out there. Not to belittle the efforts of different groups, but one of the benefits of the report is that for the first time there is an effort to bring together and harmonize different measurement efforts that are out there. So we work real hard at incorporating consensus-based measurement sets out there like the National Quality Forum Sets. They have a set on diabetes and that is incorporated. They have measures on nursing homes and that is incorporated. We also work with public and private organizations that have developed measures to consider what of those measures met a set of pretty stringent validity criteria and what of those measures could provide national data over time. So those were our two big cut points.

But basically I guess the answer is that we see the NHQR as an umbrella under which a lot of different measurement efforts are collected and reported on.

Cindy DiBiasi: We have a caller from New York. Foster Geston is on the phone. Hello?

Foster Geston: Hi, yes, thank you. I am sorry I can't take this off speaker so I apologize for the echo. I guess I was wondering since there was a presentation about disparities and a presentation about CAHPS® about the link between the two. My question is first simply is do the levels of disparities seen in the disparity reports seen in satisfaction as well and can Dale or someone else comment on what is seen with respect to individual's experience of care by those groups that are described in the disparity report? Is there reason to think that improvement in narrowing the gap in disparities will improve people's experience of care or vice versa? That improvements in experience of care between various might actually improve technical quality?

Dr. Ed Kelley: Those are really good questions. I think, I will comment on it and maybe I can let Dale comment on sort of the general issue of disparities and patient experience of care and disparities seen and measurement thereof.

Basically, The National Healthcare Disparities Report did track differences between racial and ethnic groups and socioeconomic groups. We would consider these patient experience of care measures and specifically those fall into our patient-centeredness discussion as well as our timeliness discussion. We would probably be better served by checking out on The Quality Tools Website a set of detailed tables that the report has that goes through and basically outlines four different racial and ethnic groups in kind of Consumer Reports-like fashion, a table of the different measures and where do different subgroups, racial and ethnic groups and socioeconomic groups do better or worse than the reference population across those measures.

In general, the disparities that you see in some of the effectiveness areas of care that we track, you do see in disparities as well that there are some real important exceptions to that. More detail is available on that on the web.

Dale Shaller: I think that improvements in the patient experience of care will, there is every reason to do that across racial and ethnic groups. While we do see disparities by those categories, I think what we are generally targeting is what medical groups, provider systems can do to improve their overall approach. I do believe that overall disparities in the measures of patient experience will decline as improvement strategies are implemented and those gains are made.

Cindy DiBiasi: A question from Linda Russell from Madison, Wisconsin. She says, "I am working on developing needs assessments to gather data on the number of deaf and hearing-impaired people needing mental health services. Can I access your tools to develop appropriate need assessments as well as these formulas to measure and issue recommendations?"

Dr. Ed Kelley: I can start just by saying that the two reports do track mental health care quality. This is, however unfortunately, one of the priority areas that we have for finding new measures because there is just not that much out there that provides good information on mental health care quality at the national level. There are a lot of good reasons for that. Some of it related to the diseases associated with mental illness or captured under the rubric of mental illness and some of it associated with the state of the art in terms of quality measurement. But there are some promising efforts out there led by groups like The Washington Circle Group, Emory University and others who are working on quality measures that we hope will if not in the very next report, in future reports be out there for people to use.

There are some limited measures that do exist in terms of assessing mental health care quality and that would be one place to start would be looking at those measures, looking at the discussion that is in the two reports and also the gaps that exist as well as some of the best practices that we highlight in the reports as a starting point.

Cindy DiBiasi: Jean?

Jean Slutsky: I would also suggest doing a search on the Quality Measures Clearinghouse site, on the NQMC site under "mental illness" or "mental diseases". It is categorized like the mesh tree or the medical subject headings. If they are not on there now, we are very shortly going to be adding some mental health measures.

Cindy DiBiasi: Dale?

Dale Shaller: Well, since my role is always to add in the patient experience, dimensions through CAHPS® I would mention that there is an instrument that has been developed and tested and is used by a number of behavioral health organizations. It is a behavioral health survey based on CAHPS®. It goes by the acronym ECHO which stands for Experience of Care and Health Outcomes. That is another specific tool. It has actually been implemented in quite a number of plans and behavioral health organizations with a lot of success and discerning ways in which at least that aspect of the experience of care can be improved.

Cindy DiBiasi: A question from Trudy Mathews. She says, "In this difficult budget environment for States and health care organizations, new initiatives are hard to fund. Are there any quality improvement strategies you know of that also have the potential to produce a return on investment for States and for other purchasers?"

Dr. Ed Kelley: I think that I will start. I'm sorry, I meant to jump in. I was writing down Trudy's question, which I think is a really good point. I mean when we went out with the diabetes workbook, at least the draft of it that was designed to take some of the information we had here and retarget it to State folks as to what they could do. What we heard over and over again was how bad the problem is. I know how bad the problem is. I want to do something about it and I don't have a lot of money to do it. Part of the issue for those of you either listening on the Web or on the phone who work in this environment is selling, spending money on improving quality of care. It is not always, even though we think it is the best thing that you could possibly spend your money on, it is not the first thing that jumps to your State legislator's mind in terms of what they want to spend money on in terms of health care. Right now coverage is really the sort of prominent discussion that exists out there. I think in the diabetes workbook that will be coming out soon, we highlight some instances where the business case for quality improvement can really be made in terms of for instance, if we just take diabetes for an example. Improving primary and secondary prevention of diabetes results in major gains in terms of reductions of hospital costs and lower extremity amputations in terms of short admission for short and long-term complications for diabetes. That is not even to say, obviously, just even taking amputations. The personal costs to families and the emotional toll that goes on there and there are ways to put, because unfortunately that is the way we work, there are ways to put dollar figures on those types of advances that the workbook, when it comes out, will help spell some of that out and AHRQ has some other tools that help spell that out.

Return to Web Conference
Proceed to Next Section