Questions and Answers
Joann Sorra: Now we'll begin the question-and-answer session.
For your information, after the webinar, you will be able to
access a replay of today's webinar, an audio recording, a written transcript,
and the presenters' slides.
I'd like to turn to the audience's questions now.
Again, please go ahead and submit through the
online form. And our first question here,
I'm going to direct to you, John and L.J.
Why were you expecting differences between job positions if all job
positions are assessing the same safety culture items and the same working
places at the same time? I think they're
curious about why we did this analysis expecting differences in the responses
across the different types of staff positions.
John Hickner: I
think the best answer is the elephant analogy, which is that people have
different views of the elephant and they see things from different
perspectives. Certainly, in the offices
that I've worked in I can see differences in attitudes and perspectives between
the receptionists, the nurses, and the physicians, so I think it's not
surprising that we would see this.
Ideally, if there is a lot of open discussion in offices about the
specific issues that are measured on the safety culture survey, over the course
of time people will come to more of a common view.
It is a good sign, however, I think, that
teamwork was pretty uniformly rated highly in all categories.
And I think that's a good start.
Joann Sorra: L.J., you have anything to add?
Lyle J. (L.J.) Fagnan: Just to kind of parallel what
John is saying, the physician leadership and managers oftentimes have a little
rosier picture and are not as connected to what is going on. They're not
sitting in the lunchroom hearing about what the real issues are. So I think it
is some indication of not having your finger on the pulse and I think those
practices that schedule regular times to sit down and tell stories and reflect
on things where there is a more horizontal relationship are going to have
better positive scores. There's lots of
things to think about here and we hope to look at that in more detail.
Joann Sorra: Thank
you. Our next question is again for you,
John and L.J., about surveying practices staffed by residents and faculty who
see patients on a part-time basis. Would
you recommend surveying those residents and faculty?
John Hickner: Yes, I certainly would, even though
they may be there a third of the time or half of the time. I think their
opinion is important, so I would include them.
Joann Sorra: L.J., how about you?
Lyle J. (L.J.) Fagnan: I would agree with that.
I think that we are seeing increasing numbers
of those types of practices and this survey is quite valuable for those
practices and meaningful.
Joann Sorra: Thank you.
Our next question is, can you talk a little
bit about the manpower hours needed to complete the process, the survey from
start to finish and also about the costs?
I don't know if, L.J., you have a sense of this from the 300 medical
offices that you helped administer the survey.
Lyle J. (L.J.) Fagnan: We did this in 11 networks and
there are probably 11 different responses to this.
It does take a fair amount of time.
We had support to do this.
Our sense is that it takes some outside
facilitation to engage the practices both in administering the survey and being
there face to face, getting the surveys done. Then, it's nice to get the
results back, but in order to get practices to reflect on those and if you are
working with a set of practices, you actually need to go sit down with them
because otherwise it is going to sit in a stack on their desk and not get
looked at. So I think it does take a
fair amount of time.
I think it's only going to be as much value
as the effort and time that you put into it and I do not have any economic data
of saying this is how many dollars it would take and what staff FTE it would
take to do that.
Joann Sorra: John, any experiences from you with
John Hickner: Clearly, it takes effort.
Consider that those who complete the survey
spend between 10 and 20 minutes to complete it, so that's their full
obligation. The time involved is the one
that is doing the administration of the survey.
We found when we did it at Cleveland Clinic that having one person
responsible in each individual practice was sufficient to get the work done to
pass out the questionnaires and pick them up in a confidential fashion; we used
sealed envelopes. And then there are the
data entry issues, so if you do it with a paper form, somebody has to enter the
data into a database. AHRQ provides a terrific database that you can put the
results into and the database has macros that will do the analysis.
You also will have the option, I believe, and
Joann can answer this, of doing this online, so that is another option.
I don't know if that is available yet.
Joann Sorra: The
Agency for Healthcare Research and Quality does not support a sort of central Web
survey. What was done in the pilot test
was that one of the systems actually had very good Internet access in their medical
offices and they administered this survey themselves through the Web, so we
know that the Web is going to be a mode that increasingly medical offices are
going to be able to access. What we found in the pilot was that we had a better
response rate on paper and that you still are going to have a little more
difficult time, even in hospitals, getting the response rate that you can get
on paper with the Web. Paper is still
the best way to go for a higher response rate.
Next question. Once a survey is administered and results are
received, will there be help from AHRQ to address areas of concerns?
I can probably answer this question.
Westat is the support contract for AHRQ to
provide assistance to users of the survey.
Right now, there is a medical office resource list that is on the AHRQ Web
site and it lists dozens of free online resources that address the various
areas that are assessed in the survey.
So I recommend that you go to the AHRQ Web site to the Medical
Office Survey page and then look at the medical office resource list and check
out some of those resources that address things like access and information
exchange and followup and I would recommend you start there and then if you have
questions, you can send them to the safety culture surveys mailbox and
technical assistance line.
The next question.
Are there identified best practices that are linked to the survey
areas? For example, how to address work
pressure and pace. John and L.J., I don't know what your experience has been in
terms of once you have the results, what your next steps have been with the
John Hickner: Unfortunately, the research is thin in
this area and although some organizations do very well and have developed their
own internal best practices, I don't think anybody has published much in the
way of best practices. As an example, my
research group has been working on best ways to follow up on test results,
which is a real safety issue. Our group
has also done a little work on ensuring accurate medication lists, another big
safety area, but by and large I would say that safety research in the office
setting when it comes to implementation and best practices is still at an early
stage and there is great need for work in that area.
Joann Sorra: L.J., any comments?
Lyle J. (L.J.) Fagnan: I am going to speak as an
individual network. We were one of the 11 networks and so we did 36 practices
in our network. I think there is an opportunity here. Again, you have to find
some support and funding for this, but we have exemplar practices.
I'm looking at work pace and pressure, so the
average is about 44 percent positive responses but I had a couple practices
that were 80 and 90 percent that had positive responses.
Others had 22 percent and 15 percent, so it
would actually take time to figure out what is going on in those practices. What
are the characteristics of those practices that allow them to really do really
well or to be more challenged in those areas? I think there is plenty of
opportunity to look at it, but we are going to have to get down to the macro
system level to understand that.
Joann Sorra: Thank you.
Our next question is, if you have an office
with fewer than 10 staff, what do you recommend doing with these offices in
terms of breaking out the results? Naomi, perhaps you can address this in terms
of the rules that we use for the comparative database on hospital versus
medical office and kind of the differences in the required Ns.
Naomi Dyer: Sure.
For medical office, to get results, all you need are 5 respondents, not
10. With hospital, it is the
rule of 10. For the breakout of the different
staff positions, if you are looking at your own results, we would say don't do
anything with less than three for confidentiality reasons.
For the comparative database, as long as you
have one, we will include it in the benchmark, since some of the staff
positions may have fewer than three in a given medical office.
Joann Sorra: Thank you.
Our next question is, do you anticipate that
payers or other groups will expect office practices to conduct these surveys as
part of recognition, pay for performance, contractual obligations?
John or L.J., any thoughts on that?
John Hickner: The
answer right now is yes, if the practices belong and are certified by the Joint
Commission, because the Joint Commission now does require periodic surveys of
safety culture. For example, Cleveland
Clinic just did their most recent survey this past summer, which included not
just the hospital employees but all the offices.
In smaller offices, I think no, but I would guess
that eventually all accredited practices will need to do some kind of
assessment of safety culture on a periodic basis.
I don't know if that's once a year, once
every 2 years, or what.
Lyle J. (L.J.) Fagnan: To my knowledge and maybe there are other tools out there, but
this is the only tool I'm aware of that really measures the experience of care
from the people that actually work in the office.
We have patient experience of care tools, and
we have clinical measure qualities, but this one actually has people respond based
on their perceptions of what's going on in these various domains. I think it's incredibly
valuable. I think it is mostly useful
internally for practices to start to reflect on how they are doing and to
enhance their communication structure. I
am not sure how much payment's going to be attached to this, but I think it's a
tremendous tool for practices to engage each other in talking about these
various quality and safety domains.
Joann Sorra: Thank
you, L.J., John, and Naomi. Before we
close we ask that you please take a minute to complete an evaluation of today's
webinar which will automatically appear on your screen in just a moment.
Your feedback is very important to us.
We are also providing a technical assistance
E-mail address and toll-free number if you have any further questions or
comments on the Medical Office Survey and we've also provided the link to the
medical office survey pages on the AHRQ Web site.
A big thank you to our speakers and I definitely
want to thank you all for participating in today's webinar.
Return to Contents
Current as of June 2011
Using the AHRQ Medical Office Survey on Patient Safety Culture. Transcript of a webinar held April 29, 2011. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/qual/mosurvey11/motrans42911.htm