Staff Perception Survey before and after EHR/CPOE Implementation (Text Version)

Slide Presentation from the AHRQ 2009 Annual Conference

On September 16, 2009, Jean Loes made this presentation at the 2009 Annual Conference. Select to access the PowerPoint® presentation (2.39 MB) (Plugin Software Help).

Slide 1

Staff Perception Survey before and after EHR/CPOE Implementation

Jean Loes
Marcia Ward, Douglas Wakefield, John O'Brien

Slide 2

Introduction

• Among the most notable challenges to implementing clinical information systems are the varying levels of acceptance and use by healthcare providers and employees:
• • Research has shown that experiences shape the degree to which users will accept the technology initially (Dixon, 1999; Herbert & Benbasat, 1994).
  • Research has shown that users' attitudes regarding risks to service quality and disruptions in workflow hinder implementation (Hu, et al., 2002; Zheng, et al., 2005).
• All of the significant models of information technology use suggest that perceptions of the impact on work and outcomes are significant determinants of technology use and adoption (Kufafka, et al., 2003).

Slide 3

Background on Measures

• Measures have been created to explore attitudes toward technology including perceived usefulness and ease of use (Davis, 1989).
• Recently we developed a measure of healthcare worker perceptions of the effect of clinical information systems on workflow processes and outcomes (Wakefield et al., 2007).
• This Information Systems Expectations and Experiences (I-SEE) survey was administered before and after "Go-Live" of a comprehensive system change in several hospitals.
• Analysis of the survey factor structure identified scales that assessed respondents' perceptions related to communication changes, changes in selected work behaviors, perceptions of the implementation strategy, and the impact on quality of patient care (Wakefield et al., 2007).

Slide 4

Objective

• The purpose of this study is to examine changes in perceptions about quality before and after implementation of a comprehensive clinical information system.
• This study is the first to explore these changes in multiple small hospitals and across various employee categories.

Slide 5

Survey Methods

• The I-SEE survey instrument was modified. Respondents were asked to indicate their current perceptions at each wave.
• The survey was administered at three times:
  • Wave 1 - March 2007 - before any changes
  • Wave 2 - March 2008 - during training and preparation
  • Wave 3 - March 2009 - after implementation
• Implementation "Go-Live" occurred:
  • July 2008 for 3 hospitals
  • September 2008 for 4 hospitals

Slide 6

Survey Responses Number and (Response Rate)

Slide 7

Survey Items with an Average Response of "Strongly Agree"

The table describes the highest rated (6-point Likert scale: strongly disagree to strongly agree) items in descending order. 

Slide 8

Items that Changed across Waves

The line graph illustrates the items that indicated significant changes across the three waves of surveys.

Participants' response to question 15 (Staff are alerted to potential patient care errors before they occur) indicate a decline in favorable perception over the three waves.

Participants' response to question 22 (Patient related clinical data are available to decision makers in a timely manner) indicate no notable difference in perception between Wave 1 and 2, but an increase at Wave 3 in favorable perception.

Participants' response to question 3 (I can quickly access information that I need to share with patients and families) indicate an increase in favorable perception over the three waves.

Participants' response to question 26 (Patient care orders are consistently legible and clear) indicate a slight decline in perception between Wave 1 and 2, and a substantial increase at Wave 3.

Participants' response to question 8 (Too many verbal orders are made on my unit) indicate no notable difference in perception between Wave 1 and 2, but an increase at Wave 3 in favorable perception.

Slide 9

Number of Respondents in Each Staff Group

Slide 10

Comparison across Staff Groups

• To explore whether the staff groups differed, we compared their responses across the three waves.
  • The Other - NonClinical group was excluded because they had minimal if any contact with the clinical information system and were not directly involved in patient care.
• Significant interactions were found for three survey items shown on the next slides. The pattern from before to after implementation was:
  • Other - Clinical respondents showed no change or increases in survey responses after implementation
  • Registered nurses showed no change in survey responses after implementation
  • Providers showed sizable decreases in survey responses after implementation.

Slide 11

Significant Differences among Staff Groups

The line graph illustrates the change in response to question 10 (I spend about the right amount of time recording diagnoses and symptoms) between three staff groups across the three waves. The providers show the most change in perception and are the only group to decrease after implementation. The Registered Nurses show little change while the Other-Clinical group shows an increase after implementation.

Slide 12

Significant Differences among Staff Groups

The line graph illustrates the change in response to question 11 (I spend about the right amount of time preparing discharge documents) between three staff groups across the three waves. The providers show the most change in perception and are the only group to decrease after implementations. Prior to implementation, the Registered Nurses show a slight increase while the Other-Clinical group shows a slight decrease.

Slide 13

Significant Differences among Staff Groups

The line graph illustrates the change in response to question 19 (The work processes I commonly use are efficient) between three staff groups across the three waves. The providers show the most change in perception and are the only group to decrease after implementation. The Registered Nurses show a slight decline and the Other-Clinical group shows a slight increase across waves.

Slide 14

Comparison between Physicians and Mid-level Providers

• To further explore whether subsets of the provider group differed, we compared their responses across the three waves.
  • Physicians included 11 supervisory and 13 non-supervisory physicians.
  • Mid-level Providers included 22 nurse practitioners, physicians assistants, CRNAs, etc.
• Significant interactions were found for four survey items. As shown on the next slide, the pattern from before to after implementation was:
  • Mid-level Providers showed increases in survey responses after implementation to two survey items
  • Mid-level Providers showed no change in survey responses after implementation to two survey items
  • Physicians showed increases in survey responses after implementation to two survey items
  • Physicians showed decreases in survey responses after implementation to two survey items.

Slide 15

Significant Differences between Physicians and Mid-Level Providers

The line graphs illustrates the change in physicians' and mid-level providers' response across the three waves. Significant differences were found between the two groups for questions 3, 4, 14, and 22. In each case the physicians had a lower rating than the mid-level providers.

Slide 16

CPOE Rates after Implementation

The line graph illustrates computerized provider order entry (CPOE) use rate within each of the seven hospitals over a nine month period, from Aug. 2008 to June 2009. The average CPOE use rate is about 60% and if fairly consistent across the nine-month period.

Slide 17

Relationship between Survey Items at Wave 3 and CPOE Use Rates

• Correlations between survey items at Wave 3 and CPOE use rates indicate that staff at hospitals with higher CPOE use rates also tended to respond more in agreement to three survey items:
  • "Patients are rarely asked the same questions by the staff" (r=.93)
  • "Access to information to make good patient care decisions is available" (r=.79)
  • "I get a great deal of professional satisfaction from my job" (r=.73)
• The strongest relationship indicated that staff at hospitals with higher CPOE use rates tended to strongly disagree with the survey item "Too many verbal orders are made on my unit" (r= -.96)

Slide 18

Relationship between Survey Items at Wave 2 and CPOE Use Rates

• Correlations between survey items at Wave 2 and CPOE use rates indicate that staff at hospitals with higher CPOE use rates tended to respond more in agreement to three survey items before Go-Live:
  • "I spend about the right amount of time documenting patient care" (r=.69)
  • "Patients are rarely asked the same questions by the staff" (r=.63)
  • "Overall patient care is safe in the areas I work" (r=.59)
• Thus, higher agreement with these items may predict CPOE use rates after implementation.

Slide 19

References

Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and use acceptance of information technology. MIS Quarterly, 13 (3), 318-340.

Dixon, D. R. (1999). The behavioral side of information technology. International Journal of Medical Informatics, 56, 117-123.

Halbesleben JRB, Wakefield DS, Ward MM, Brokel J, Crandall D. (2009). The relationship between super users' attitudes and employee experiences with clinical information systems. Medical Care Research & Review, 66: 82-96.

Hebert, M., & Benbasat, I. (1994). Adopting information technology in hospitals: the relationship between attitudes/expectations and behavior. Hosp Health Serv Adm, 39 (3), 369-383.

Hu, P. J., Chau, P. Y. K., & Sheng, O. R. L. (2002). Adoption of telemedicine technology by health care organizations: An exploratory study. Journal of Organizational Computing and Electronic Commerce, 12 (3), 197-221.

Kukafka, R., Johnson, S. B., Linfante, A., & Allegrante, J. P. (2003). Grounding a new information technology implementation framework in behavioral science: A systematic analysis of the literature on IT use. Journal of Biomedical Informatics, 36, 218-227.

Wakefield DS, Halbesleben JRB, Ward MM, Qiu Q, Brokel J, Crandall D. (2007) Development of a measure of clinical information systems expectations and experiences. Medical Care, 45: 884-890.

Zheng, K., Padman, R., Johnson, M. P., & Diamond, M. S. (2005). Understanding technology adoption in clinical care: clinician adoption behavior of a point-of-care reminder system. International Journal of Medical Informatics. 74(7-8), 535-543.

Slide 20

Acknowledgments

• This work was supported by funding from AHRQ - 5UC1HS016156 - "EHR Implementation for the Continuum of Care in Rural Iowa"
• Mercy Medical Center - North Iowa
• Mercy North Iowa Network
• The University of Iowa - Center for Health Policy and Research
• The University of Missouri - Center for Health Care Quality