Staff Perception Survey before and after EHR/CPOE Implementation (Text Version)

Slide presentation from the AHRQ 2009 conference

Slide Presentation from the AHRQ 2009 Annual Conference


On September 16, 2009, Jean Loes made this presentation at the 2009 Annual Conference. Select to access the PowerPoint® presentation (2.39 MB) (Plugin Software Help).


Slide 1

Slide 1. Staff Perception Survey before and after EHR/CPOE Implementation

Staff Perception Survey before and after EHR/CPOE Implementation

Jean Loes
Marcia Ward, Douglas Wakefield, John O'Brien

 

Slide 2

Slide 2. Introduction

Introduction

  • Among the most notable challenges to implementing clinical information systems are the varying levels of acceptance and use by healthcare providers and employees:
    • Research has shown that experiences shape the degree to which users will accept the technology initially (Dixon, 1999; Herbert & Benbasat, 1994).
    • Research has shown that users' attitudes regarding risks to service quality and disruptions in workflow hinder implementation (Hu, et al., 2002; Zheng, et al., 2005).
  • All of the significant models of information technology use suggest that perceptions of the impact on work and outcomes are significant determinants of technology use and adoption (Kufafka, et al., 2003).

 

Slide 3

Slide 3. Background on Measures

Background on Measures

  • Measures have been created to explore attitudes toward technology including perceived usefulness and ease of use (Davis, 1989).
  • Recently we developed a measure of healthcare worker perceptions of the effect of clinical information systems on workflow processes and outcomes (Wakefield et al., 2007).
  • This Information Systems Expectations and Experiences (I-SEE) survey was administered before and after "Go-Live" of a comprehensive system change in several hospitals.
  • Analysis of the survey factor structure identified scales that assessed respondents' perceptions related to communication changes, changes in selected work behaviors, perceptions of the implementation strategy, and the impact on quality of patient care (Wakefield et al., 2007).

 

Slide 4

Slide 4. Objective

Objective

  • The purpose of this study is to examine changes in perceptions about quality before and after implementation of a comprehensive clinical information system.
  • This study is the first to explore these changes in multiple small hospitals and across various employee categories.

 

Slide 5

Slide 5. Survey Methods

Survey Methods

  • The I-SEE survey instrument was modified. Respondents were asked to indicate their current perceptions at each wave.
  • The survey was administered at three times:
    • Wave 1 - March 2007 - before any changes
    • Wave 2 - March 2008 - during training and preparation
    • Wave 3 - March 2009 - after implementation
  • Implementation "Go-Live" occurred:
    • July 2008 for 3 hospitals
    • September 2008 for 4 hospitals

 

Slide 6

Slide 6. Survey Responses Number and (Response Rate)

Survey Responses Number and (Response Rate)

HospitalWAVE 1
2007
WAVE 2
2008
Wave 3
2009
Ellsworth Municipal Hospital73 (28%)59 (23%)41 (16%)
Franklin General County Hospital93 (51%)56 (31%)43 (24%)
Hancock County Memorial Hospital79 (65%)63 (52%)34 (28%)
Kossuth Regional Health Center54 (28%)47 (24%)35 (18%)
Mercy Medical Center - New Hampton58 (50%)62 (54%)48 (42%)
Mitchell County Regional Medical Center45 (21%)49 (23%)44 (21%)
Palo Alto County Health Systems90 (50%)42 (23%)49 (27%)
Total Returned Surveys511 (40%)385 (30%)305 (24%)

 

Slide 7

Slide 7. Survey Items with an Average Response of "Strongly Agree"

Survey Items with an Average Response of "Strongly Agree"

The table describes the highest rated (6-point Likert scale: strongly disagree to strongly agree) items in descending order. 

Survey ItemsOverall Mean Response
31. Patient care is consistently given according to the "9 Rights"5.35
25. I enjoy my job5.30
14. Overall patient care is safe in the areas I work5.26
1. Access to information to make good patient care decisions is available5.12
18. I get a great deal of professional satisfaction from my job5.12
29. Communications ensuring high quality and safe patient care routinely occur when patients are transferred to other facilities5.05

 

Slide 8

Slide 8. Items that Changed across Waves

Items that Changed across Waves

The line graph illustrates the items that indicated significant changes across the three waves of surveys.

Participants' response to question 15 (Staff are alerted to potential patient care errors before they occur) indicate a decline in favorable perception over the three waves.

Participants' response to question 22 (Patient related clinical data are available to decision makers in a timely manner) indicate no notable difference in perception between Wave 1 and 2, but an increase at Wave 3 in favorable perception.

Participants' response to question 3 (I can quickly access information that I need to share with patients and families) indicate an increase in favorable perception over the three waves.

Participants' response to question 26 (Patient care orders are consistently legible and clear) indicate a slight decline in perception between Wave 1 and 2, and a substantial increase at Wave 3.

Participants' response to question 8 (Too many verbal orders are made on my unit) indicate no notable difference in perception between Wave 1 and 2, but an increase at Wave 3 in favorable perception.

 

Slide 9

Slide 9. Number of Respondents in Each Staff Group

Number of Respondents in Each Staff Group

Professional/Employee GroupsNumber
Providers
(physicians and midlevel providers)
49
Registered Nurses343
Other-Clinical
(professional, technical, patient support)
458
Other-NonClinical
(clerical, senior management, marketing)
293

 

Slide 10

Slide 10. Comparison across Staff Groups

Comparison across Staff Groups

  • To explore whether the staff groups differed, we compared their responses across the three waves.
    • The Other - NonClinical group was excluded because they had minimal if any contact with the clinical information system and were not directly involved in patient care.
  • Significant interactions were found for three survey items shown on the next slides. The pattern from before to after implementation was:
    • Other - Clinical respondents showed no change or increases in survey responses after implementation
    • Registered nurses showed no change in survey responses after implementation
    • Providers showed sizable decreases in survey responses after implementation.

 

Slide 11Slide 11. Significant Differences among Staff Groups

 

Significant Differences among Staff Groups

The line graph illustrates the change in response to question 10 (I spend about the right amount of time recording diagnoses and symptoms) between three staff groups across the three waves. The providers show the most change in perception and are the only group to decrease after implementation. The Registered Nurses show little change while the Other-Clinical group shows an increase after implementation.

 

Slide 12

Slide 12. Significant Differences among Staff Groups

Significant Differences among Staff Groups

The line graph illustrates the change in response to question 11 (I spend about the right amount of time preparing discharge documents) between three staff groups across the three waves. The providers show the most change in perception and are the only group to decrease after implementations. Prior to implementation, the Registered Nurses show a slight increase while the Other-Clinical group shows a slight decrease.

 

Slide 13

Slide 13. Significant Differences among Staff Groups

Significant Differences among Staff Groups

The line graph illustrates the change in response to question 19 (The work processes I commonly use are efficient) between three staff groups across the three waves. The providers show the most change in perception and are the only group to decrease after implementation. The Registered Nurses show a slight decline and the Other-Clinical group shows a slight increase across waves.

 

Slide 14

Slide 14. Comparison between Physicians and Mid-level Providers

Comparison between Physicians and Mid-level Providers

  • To further explore whether subsets of the provider group differed, we compared their responses across the three waves.
    • Physicians included 11 supervisory and 13 non-supervisory physicians.
    • Mid-level Providers included 22 nurse practitioners, physicians assistants, CRNAs, etc.
  • Significant interactions were found for four survey items. As shown on the next slide, the pattern from before to after implementation was:
    • Mid-level Providers showed increases in survey responses after implementation to two survey items
    • Mid-level Providers showed no change in survey responses after implementation to two survey items
    • Physicians showed increases in survey responses after implementation to two survey items
    • Physicians showed decreases in survey responses after implementation to two survey items.

 

Slide 15

Slide 15. Significant Differences between Physicians and Mid-Level Providers

Significant Differences between Physicians and Mid-Level Providers

The line graphs illustrates the change in physicians' and mid-level providers' response across the three waves. Significant differences were found between the two groups for questions 3, 4, 14, and 22. In each case the physicians had a lower rating than the mid-level providers.

 

Slide 16

Slide 16. CPOE Rates after Implementation

CPOE Rates after Implementation

The line graph illustrates computerized provider order entry (CPOE) use rate within each of the seven hospitals over a nine month period, from Aug. 2008 to June 2009. The average CPOE use rate is about 60% and if fairly consistent across the nine-month period.

 

Slide 17

Slide 17. Relationship between Survey Items at Wave 3 and CPOE Use Rates

Relationship between Survey Items at Wave 3 and CPOE Use Rates

  • Correlations between survey items at Wave 3 and CPOE use rates indicate that staff at hospitals with higher CPOE use rates also tended to respond more in agreement to three survey items:
    • "Patients are rarely asked the same questions by the staff" (r=.93)
    • "Access to information to make good patient care decisions is available" (r=.79)
    • "I get a great deal of professional satisfaction from my job" (r=.73)
  • The strongest relationship indicated that staff at hospitals with higher CPOE use rates tended to strongly disagree with the survey item "Too many verbal orders are made on my unit" (r= -.96)

 

Slide 18

Slide 18. Relationship between Survey Items at Wave 2 and CPOE Use Rates

Relationship between Survey Items at Wave 2 and CPOE Use Rates

  • Correlations between survey items at Wave 2 and CPOE use rates indicate that staff at hospitals with higher CPOE use rates tended to respond more in agreement to three survey items before Go-Live:
    • "I spend about the right amount of time documenting patient care" (r=.69)
    • "Patients are rarely asked the same questions by the staff" (r=.63)
    • "Overall patient care is safe in the areas I work" (r=.59)
  • Thus, higher agreement with these items may predict CPOE use rates after implementation.

 

Slide 19

Slide 19. References

References

Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and use acceptance of information technology. MIS Quarterly, 13 (3), 318-340.

Dixon, D. R. (1999). The behavioral side of information technology. International Journal of Medical Informatics, 56, 117-123.

Halbesleben JRB, Wakefield DS, Ward MM, Brokel J, Crandall D. (2009). The relationship between super users' attitudes and employee experiences with clinical information systems. Medical Care Research & Review, 66: 82-96.

Hebert, M., & Benbasat, I. (1994). Adopting information technology in hospitals: the relationship between attitudes/expectations and behavior. Hosp Health Serv Adm, 39 (3), 369-383.

Hu, P. J., Chau, P. Y. K., & Sheng, O. R. L. (2002). Adoption of telemedicine technology by health care organizations: An exploratory study. Journal of Organizational Computing and Electronic Commerce, 12 (3), 197-221.

Kukafka, R., Johnson, S. B., Linfante, A., & Allegrante, J. P. (2003). Grounding a new information technology implementation framework in behavioral science: A systematic analysis of the literature on IT use. Journal of Biomedical Informatics, 36, 218-227.

Wakefield DS, Halbesleben JRB, Ward MM, Qiu Q, Brokel J, Crandall D. (2007) Development of a measure of clinical information systems expectations and experiences. Medical Care, 45: 884-890.

Zheng, K., Padman, R., Johnson, M. P., & Diamond, M. S. (2005). Understanding technology adoption in clinical care: clinician adoption behavior of a point-of-care reminder system. International Journal of Medical Informatics. 74(7-8), 535-543.

 

Slide 20

Slide 20. Acknowledgments

Acknowledgments

  • This work was supported by funding from AHRQ - 5UC1HS016156 - "EHR Implementation for the Continuum of Care in Rural Iowa"
  • Mercy Medical Center - North Iowa
  • Mercy North Iowa Network
  • The University of Iowa - Center for Health Policy and Research
  • The University of Missouri - Center for Health Care Quality
Current as of December 2009
Internet Citation: Staff Perception Survey before and after EHR/CPOE Implementation (Text Version). December 2009. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/news/events/conference/2009/loes/index.html