Using the AHRQ Medical Office Survey on Patient Safety Culture
Questions and Answers
Joann Sorra: Now we'll begin the question-and-answer session.
For your information, after the webinar, you will be able to access a replay of today's webinar, an audio recording, a written transcript, and the presenters' slides.
I'd like to turn to the audience's questions now. Again, please go ahead and submit through the online form. And our first question here, I'm going to direct to you, John and L.J. Why were you expecting differences between job positions if all job positions are assessing the same safety culture items and the same working places at the same time? I think they're curious about why we did this analysis expecting differences in the responses across the different types of staff positions.
John Hickner: I think the best answer is the elephant analogy, which is that people have different views of the elephant and they see things from different perspectives. Certainly, in the offices that I've worked in I can see differences in attitudes and perspectives between the receptionists, the nurses, and the physicians, so I think it's not surprising that we would see this. Ideally, if there is a lot of open discussion in offices about the specific issues that are measured on the safety culture survey, over the course of time people will come to more of a common view. It is a good sign, however, I think, that teamwork was pretty uniformly rated highly in all categories. And I think that's a good start.
Joann Sorra: L.J., you have anything to add?
Lyle J. (L.J.) Fagnan: Just to kind of parallel what John is saying, the physician leadership and managers oftentimes have a little rosier picture and are not as connected to what is going on. They're not sitting in the lunchroom hearing about what the real issues are. So I think it is some indication of not having your finger on the pulse and I think those practices that schedule regular times to sit down and tell stories and reflect on things where there is a more horizontal relationship are going to have better positive scores. There's lots of things to think about here and we hope to look at that in more detail.
Joann Sorra: Thank you. Our next question is again for you, John and L.J., about surveying practices staffed by residents and faculty who see patients on a part-time basis. Would you recommend surveying those residents and faculty?
John Hickner: Yes, I certainly would, even though they may be there a third of the time or half of the time. I think their opinion is important, so I would include them.
Joann Sorra: L.J., how about you?
Lyle J. (L.J.) Fagnan: I would agree with that. I think that we are seeing increasing numbers of those types of practices and this survey is quite valuable for those practices and meaningful.
Joann Sorra: Thank you. Our next question is, can you talk a little bit about the manpower hours needed to complete the process, the survey from start to finish and also about the costs? I don't know if, L.J., you have a sense of this from the 300 medical offices that you helped administer the survey.
Lyle J. (L.J.) Fagnan: We did this in 11 networks and there are probably 11 different responses to this. It does take a fair amount of time. We had support to do this. Our sense is that it takes some outside facilitation to engage the practices both in administering the survey and being there face to face, getting the surveys done. Then, it's nice to get the results back, but in order to get practices to reflect on those and if you are working with a set of practices, you actually need to go sit down with them because otherwise it is going to sit in a stack on their desk and not get looked at. So I think it does take a fair amount of time. I think it's only going to be as much value as the effort and time that you put into it and I do not have any economic data of saying this is how many dollars it would take and what staff FTE it would take to do that.
Joann Sorra: John, any experiences from you with Cleveland Clinic?
John Hickner: Clearly, it takes effort. Consider that those who complete the survey spend between 10 and 20 minutes to complete it, so that's their full obligation. The time involved is the one that is doing the administration of the survey. We found when we did it at Cleveland Clinic that having one person responsible in each individual practice was sufficient to get the work done to pass out the questionnaires and pick them up in a confidential fashion; we used sealed envelopes. And then there are the data entry issues, so if you do it with a paper form, somebody has to enter the data into a database. AHRQ provides a terrific database that you can put the results into and the database has macros that will do the analysis. You also will have the option, I believe, and Joann can answer this, of doing this online, so that is another option. I don't know if that is available yet.
Joann Sorra: The Agency for Healthcare Research and Quality does not support a sort of central Web survey. What was done in the pilot test was that one of the systems actually had very good Internet access in their medical offices and they administered this survey themselves through the Web, so we know that the Web is going to be a mode that increasingly medical offices are going to be able to access. What we found in the pilot was that we had a better response rate on paper and that you still are going to have a little more difficult time, even in hospitals, getting the response rate that you can get on paper with the Web. Paper is still the best way to go for a higher response rate.
Next question. Once a survey is administered and results are received, will there be help from AHRQ to address areas of concerns? I can probably answer this question. Westat is the support contract for AHRQ to provide assistance to users of the survey. Right now, there is a medical office resource list that is on the AHRQ Web site and it lists dozens of free online resources that address the various areas that are assessed in the survey. So I recommend that you go to the AHRQ Web site to the Medical Office Survey page and then look at the medical office resource list and check out some of those resources that address things like access and information exchange and followup and I would recommend you start there and then if you have questions, you can send them to the safety culture surveys mailbox and technical assistance line.
The next question. Are there identified best practices that are linked to the survey areas? For example, how to address work pressure and pace. John and L.J., I don't know what your experience has been in terms of once you have the results, what your next steps have been with the medical offices.
John Hickner: Unfortunately, the research is thin in this area and although some organizations do very well and have developed their own internal best practices, I don't think anybody has published much in the way of best practices. As an example, my research group has been working on best ways to follow up on test results, which is a real safety issue. Our group has also done a little work on ensuring accurate medication lists, another big safety area, but by and large I would say that safety research in the office setting when it comes to implementation and best practices is still at an early stage and there is great need for work in that area.
Joann Sorra: L.J., any comments?
Lyle J. (L.J.) Fagnan: I am going to speak as an individual network. We were one of the 11 networks and so we did 36 practices in our network. I think there is an opportunity here. Again, you have to find some support and funding for this, but we have exemplar practices. I'm looking at work pace and pressure, so the average is about 44 percent positive responses but I had a couple practices that were 80 and 90 percent that had positive responses. Others had 22 percent and 15 percent, so it would actually take time to figure out what is going on in those practices. What are the characteristics of those practices that allow them to really do really well or to be more challenged in those areas? I think there is plenty of opportunity to look at it, but we are going to have to get down to the macro system level to understand that.
Joann Sorra: Thank you. Our next question is, if you have an office with fewer than 10 staff, what do you recommend doing with these offices in terms of breaking out the results? Naomi, perhaps you can address this in terms of the rules that we use for the comparative database on hospital versus medical office and kind of the differences in the required Ns.
Naomi Dyer: Sure. For medical office, to get results, all you need are 5 respondents, not 10. With hospital, it is the rule of 10. For the breakout of the different staff positions, if you are looking at your own results, we would say don't do anything with less than three for confidentiality reasons. For the comparative database, as long as you have one, we will include it in the benchmark, since some of the staff positions may have fewer than three in a given medical office.
Joann Sorra: Thank you. Our next question is, do you anticipate that payers or other groups will expect office practices to conduct these surveys as part of recognition, pay for performance, contractual obligations? John or L.J., any thoughts on that?
John Hickner: The answer right now is yes, if the practices belong and are certified by the Joint Commission, because the Joint Commission now does require periodic surveys of safety culture. For example, Cleveland Clinic just did their most recent survey this past summer, which included not just the hospital employees but all the offices. In smaller offices, I think no, but I would guess that eventually all accredited practices will need to do some kind of assessment of safety culture on a periodic basis. I don't know if that's once a year, once every 2 years, or what.
Lyle J. (L.J.) Fagnan: To my knowledge and maybe there are other tools out there, but this is the only tool I'm aware of that really measures the experience of care from the people that actually work in the office. We have patient experience of care tools, and we have clinical measure qualities, but this one actually has people respond based on their perceptions of what's going on in these various domains. I think it's incredibly valuable. I think it is mostly useful internally for practices to start to reflect on how they are doing and to enhance their communication structure. I am not sure how much payment's going to be attached to this, but I think it's a tremendous tool for practices to engage each other in talking about these various quality and safety domains.
Joann Sorra: Thank you, L.J., John, and Naomi. Before we close we ask that you please take a minute to complete an evaluation of today's webinar which will automatically appear on your screen in just a moment. Your feedback is very important to us. We are also providing a technical assistance E-mail address and toll-free number if you have any further questions or comments on the Medical Office Survey and we've also provided the link to the medical office survey pages on the AHRQ Web site. A big thank you to our speakers and I definitely want to thank you all for participating in today's webinar.