Measuring Improvement in Hospital Teamwork: Diffusion of TeamSTEPPS in Critical Access Hospitals (Text Version)

On September 15, 2009, Katherine Jones, Wendi Nordhausen, Mark Goodridge made this presentation at the 2009 Annual Conference. Select to access the PowerPoint® presentation (2.7 MB). Plugin Software Help.

Slide 1

The University of Nebraska Medical Center
AHRQ Annual Meeting Sept. 15, 2009
Measuring Improvement in Hospital Teamwork: Diffusion of TeamSTEPPS in Critical Access Hospitals

Katherine Jones, PT, PhD
Wendi Nordhausen, RN, BSN
Mark Goodridge, RT (R) (CT)

Slide 2

Acknowledgements

Our Team

• Anne Skinner, RHIA
• Robin High, MS, MBA
• Andrea Bowen, BA
• 99 Master Trainers from 24 Critical Access Hospitals

Our Funding

• AHRQ Office of Communications and Knowledge Transfer
• Nebraska Dept of Health and Human Services
• Good Samaritan Health Systems Network
• St. Elizabeth CAH Link
• Direct funds from 14 Critical Access Hospitals

Medicare Rural
Hospital
Flexibility
Program
(Flex Program)

Slide 3

Objectives

• Describe a collaborative approach to implementing TeamSTEPPS within a state/region
• Use the AHRQ Hospital Survey on Patient Safety Culture (HSOPS) to plan and evaluate the implementation of TeamSTEPPS
• Use 'Diffusion of Innovations,' Kirkpatrick's Taxonomy, and decision frame to explain variations in success implementing TeamSTEPPS
• Implement lessons learned from two Critical Access Hospitals to facilitate adoption of TeamSTEPPS

Slide 4

TeamSTEPPS Background

• 05 - 07 AHRQ Partnerships in Implementing Patient Safety Grant (1 U18 HS015822)
  • Purpose: Implement patient safety practices of voluntary medication error reporting and organizational learning in 24 CAHs
  • Aim: Develop organizational infrastructure for reporting and analyzing medication errors needed to identify system sources of error
  • Evaluate impact of this infrastructure change on safety culture with HSOPS
    • HSOPS results revealed need for teamwork

Slide 5

Implementation Background

• 3/2008 initial funding through AHRQ Office of Communications and Knowledge Transfer
• Purpose: Implement the patient safety practice of teamwork and communication training in 25 Critical Access Hospitals
• Aim: Evaluate impact of TeamSTEPPS training program on safety culture using our rural-adapted version of the AHRQ HSOPS
• Collaborative funding through 12/2010

Slide 6

Collaborative Funding

1. AHRQ Office of Communications and Knowledge Transfer (3/2008 - 11/2008) $36,190.00
2. Nebraska Dept of Health and Human Services (6/2008 - 12/2010) $79,848.00
3. Two Critical Access Hospital Networks (6/2008 - 6/2009, 3/2009 - 12/2010) $30,856.00
4. Direct Payments from 14 Hospitals (2008 - 2010) $23,487.53

Total $170,381.53

Slide 7

Implementation Cycle

Slide 8

Diffusion of TeamSTEPPS in Nebraska

Image: This picture is an outline of the state of Nebraska showing the locations of the 35 Critical Access Hospitals participating in the TeamSTEPPS Collaborative

Slide 9

Measuring to Implement TeamSTEPPS

TeamSTEPPS Tools to bridge gap between belief and behavior.

• Situation Monitoring
• Mutual Support... Seeking and offering Task Assistance
• Briefs, Huddles, Debriefs

Slide 10

Measuring to Implement TeamSTEPPS

Communication Openness Chase County Community Hospital

TeamSTEPPS Tools to bridge gap between belief and behavior.

• Advocacy and assertion
• I'm Concerned, I'm Uncomfortable, Stop the procedure (CUS)

Slide 11

TeamSTEPPS Tools to improve structured communication across shifts and departments.

• SBAR, Closed loop communication, Seeking Clarification
• Huddles and WalkRounds after shift change
• I PASS the BATON

Slide 12

Measuring to Evaluate TeamSTEPPS

Slide 13

Evaluation: Adoption of Behavior

Bar graph entitled: Adoption of Five Team Behaviors in 24 Critical Access Hospitals. A bar graph indicating that 4/24 hospitals had a False Start, 5/24 Hospitals are Late Majority, 12/24 Hospitals are Early Majority, and 3/ 24 Hospitals are Early Adopters. We used respondents' ratings of the frequency of five team behaviors in their departments to create adopter categories (Tables 6, and Appendix B, Figure 3). Thus, a respondent who indicated that all five team behaviors were performed "most of the time or always" in their department was considered an adopter. On average, 8.4% of 2,021 respondents from the 23 hospitals were adopters. Each hospital was assigned to a category- False Start, Late Majority, Early Majority, and Early Adopter- according to its proportion of adopters.

Slide 14

Implementing TeamSTEPPS at Clarinda Regional Health Center
Clarinda, Iowa

Mark Goodridge, RT (R) (CT)

Slide 15

TeamSTEPPS at Clarinda Regional Health Center

• Critical Access Hospital - 25 Beds
• Average daily census 7-8
• Census can vary from 4-14 in 24 hours
• 85% of services are out-patient
• 400-500 ED visits per month
• 600-700 specialty clinic visits per month
• 225 employees - FT & PT

Slide 16

TeamSTEPPS Training-Master Trainers

• 3 Master Trainers trained April 2008 with UNMC Collaborative
  • Senior Staff member
    • Elaine Otte COO
  • Frontline staff
    • Mark Goodridge RT (R) (CT)
    • Jennifer Chambers RN (ED)

Slide 17

TeamSTEPPS Training-Leadership

Leadership Development Training

• Department managers
• Senior Staff members
• Board of Trustees
• Fundamentals Course
• One time training session off campus
• Managers required to submit action plans to COO

Also shown on the slide is a photo titled "Role Play during Leadership Development".

Slide 18

TeamSTEPPS Training-All Staff

• Nov & Dec 2008
• 15-20 staff per class
• All classes interdisciplinary
• Essentials course
• Team building exercises
• Goal to train all staff within 2 weeks by Master Trainers & Education Director

Also shown on the slide is an image titled: Team Building Exercise during Staff Training

Slide 19

We Defined TeamSTEPPS as a Change

• We created a Sense of Urgency
  • Results from the 2006 Patient Safety Survey
  • Sue Sheridan video
• We ensured staff viewed TeamSTEPPS as consistent with our mission to provide exceptional care in a safe environment
• TeamSTEPPS is better than our "old way of communicating"
  • Shared stories of impact of our "old way"
  • TeamSTEPPS videos and role playing

Slide 20

We Obtained Management Support

• Senior leaders are educated and supportive of the TeamSTEPPS initiative
  • COO trained as Master Trainer
• The board is educated and supportive of the TeamSTEPPS initiative
  • Included in the Leadership Fundamentals Training Session
• Medical Staff education-in progress; goal is to shift from "I" to "We"

Slide 21

Our Champions Led the Way

• Mark (Radiology) & Jennifer (Nursing)- front line champions
  • Led the organization by training staff & mentoring department managers
  • Use TeamSTEPPS language
  • Overcome resistance by engaging key employees and managers

Slide 22

Resources Used for Implementation

• UNMC's support
  • Conference calls
  • Sharing tools
  • Lessons Learned Conference Nov 2008
• Senior Staff support
• Funds allocated for the program by COO

Also shown on slide is an image titled: Our Poster at UNMC Lessons Learned Conf, Nov. 2008.

Slide 23

We are Sustaining TeamSTEPPS

• "Not a flavor of the month"
• Senior Staff and Board of Trustees buy-in
• Use TeamSTEPPS tools and language-role models
• Focus on Debriefs for drills and code alerts
• Part of new employee orientation
  • COO introduces concept to all new employees
  • Biannual Essentials Course
  • All receive a pocket guide

Slide 24

Lessons Learned and Next Steps

• Support of Board of Trustees
  • Attended Leadership training
    
• Next Steps
  • Medical Staff training
  • Sustainment — Use TeamSTEPPS tools in specific areas
  • Communicate use of TeamSTEPPS by professional organizations (AORN)

Slide 25

We are Measuring to Identify Improvement

• How do we know our training program resulted in change in culture, learning and behavior?
  • Data from HSOPS
  • Observed Changes in process and behavior

Slide 26

Implementing TeamSTEPPS at Chase County Community Hospital
Imperial, Nebraska

Wendi Nordhausen, RN, BSN

Slide 27

TeamSTEPPS at Chase County Community Hospital

• 25 Bed - Critical Access Hospital
• Average Daily Census - 2 to 6 patients
• Staff 105 employees
• Attached clinic
• 3 physicians, 2 physician assistants, 2 nurse practitioners

Also shown is an aerial image of the hospital and a map of the county with the text enclosed: Chase County, Pop. 3,269, Density 4/sq mi.

Slide 28

TeamSTEPPS Training

• 4 Master Trainers - April 23-25th, 2008 as part of UNMC Collaborative
• Included ALL staff and medical staff
• Board informed
• Included all modules in Fundamentals Course- adapted to our specific needs
• Offered 4 to 5 times each week in 60 - 90 minute sessions for 7 weeks
• Included one 6 hour make-up day

Slide 29

We Defined TeamSTEPPS as a Change

• We created a sense of urgency.
• We ensured staff viewed TeamSTEPPS as consistent with our mission and vision
• We ensured staff saw TeamSTEPPS as better than our "old way of communicating"
  - Started with SBAR and trauma debriefs

Slide 30

We Obtained Management Support

• Senior leaders are educated and supportive of the TeamSTEPPS initiative
• The board is educated and supportive of the TeamSTEPPS initiative
• Medical Staff is educated and supportive of the TeamSTEPPS initiative

Slide 31

Our Champions Led the Way

• CEO - Master Trainer, Leader
• Physician - QI background
• Linda (Resp. Therapist), Lori (Lab Coord.), Wendi (QI Coordinator) — Interdisciplinary Master Trainers

Slide 32

We are Sustaining TeamSTEPPS

• Employees know TeamSTEPPS is a priority
  • Use the tools and language
  • Scenarios brought to manager & dept meetings
• TeamSTEPPS changed day to day processes
  - SBAR
  - Trauma Debriefs
• Our organization supports and rewards involvement in TeamSTEPPS

Slide 33

Resources Used for Implementation

• UNMC conference calls
• Administrative Support
• Lessons Learned Conference
• Critical Access Hospital Network Meeting
• Additional Master Trainers could make a difference

Also shown on the slide is a photo titled: Our Poster at UNMC Lessons Learned Conf, Nov. 2008.

Slide 34

Lessons Learned and Next Steps

• Most effective aspect of implementation- trained all staff in Fundamentals
• Least effective aspect... change team function
• Current and Future Focus - Orient new employees, Quarterly refresher courses, higher level of implementation and integration of the tools.

Slide 35

We are Measuring to Identify Improvement

• How do we know our training program resulted in change in culture, learning and behavior?
  • Data from HSOPS
  • Observed Changes in process and behavior... mails structured by SBAR, conversations about "processes" and communication

Slide 36

Measuring to Evaluate for Individual Hospitals and the Collaborative

Katherine Jones, PT, PhD

Slide 37

Measuring to Evaluate

Image: A pyramid to conceptualize Kirkpatrick's Taxonomy of Training Criteria. Now that we understand that it is changes in the structure and process of care that produce changes in outcomes, we must think about how we can evaluate the effect of our training programs on the structure and process of care delivered by our learners.

Reference: Alliger et al. A meta-analysis of the relations among training criteria. Personnel Psychology. 2006, 50: 341-358.

Slide 38

Rural HSOPS Spring 2009

• Population Surveyed
  • 24 Hospitals evaluate impact of TeamSTEPPS Implementation 2008 - 2009 (2,137 respondents)
  • 13 Hospitals obtain baseline prior to TeamSTEPPS Implementation (1,328 respondents)
  • Added Teamwork Related Items to HSOPS
• Overall Response Rate for 37 Hospitals 3465/4601 = 75.3%
• Range 51% - 96%

Slide 39

Added HSOPS Knowledge & Behavior Items

Slide 40

Graph: TeamSTEPPS Training, Knowledge, and Behavior April 2009

Slide 41

Hospital Survey on Patient Safety Culture Composite Positive Responses Comparison Over Time

Slide 42

Hospital Survey on Patient Safety Culture Composite Positive Responses Benchmarking Hospital Results to the 2009 Comparative Database

This graph compares Chase County Community Hospital results on the HSOPS to the 10th and 90th percentiles reported in the 2009 National Database. This graph demonstrated the use of HSOPS for External Benchmarking to the National Database.

Slide 43

This bar graph reports team-related item level results from the HSOPS for Case County Community Hospital in 2005, 2007, 2009.

People support on another in this department. BELIEF

• 2009: 76%
• 2007: 81%
• 2005: 78%
  

When one in this department gets really busy others help out.*

• 2009: 63%
• 2007: 57%
• 2005: 41%

(R) We work in "crisis mode" trying to do too much, too quickly.*

• 2009: 69%
• 2007: 63%
• 2005: 48%

Staff will freely speak up in they see something that may negatively affect patient care. BELIEF

• 2009: 66%
• 2007: 65%
• 2005: 62%

Staff feel free to question the decisions or actions of those with more authority.*

• 2009: 50%
• 2007: 36%
• 2005: 40%

* ≥5% Change from 2005 Baseline
(R) Reverse Worded Item

Slide 44

This bar graph entitled "Teamwork-Related HSOPS Items Chase County Community Health Center" reports team-related item level results from the HSOPS for Case County Community Hospital in 2005, 2007, 2009.

Hospital departments work well together to provide the best care for patients.* BELIEF

• 2009: 69%
• 2007: 73%
• 2005: 62%
  

(R) Problems often occur in the exchange of information across hospital departments*

• 2009: 59%
• 2007: 43%
• 2005: 29%

(R) Shift changes are problematic for patients in this hospital.*

• 2009: 65%
• 2007: 54%
• 2005: 44%

Mistakes have left to positive changes here.* BELIEF

• 2009: 71%
• 2007: 76%
• 2005: 59%

* ≥5% Change from 2005 Baseline
(R) Reverse Worded Item

Slide 45

Decision Frame Revealed in HSOPS

• Decision frame: mental structures people use to organize the world
  • Reference point changes with knowledge
• If behaviors change to reflect change in knowledge. Belief may not change
  • Consider item level scores not just dimension scores to track change over time
• If behavior not consistent with new knowledge... HSOPS results less positive after training
  • Seek higher standard based on new knowledge

Tversky A, Kahneman D. Science. 1981;211:453-458.
Wright G. Goodwin, P. Strategic Management Journal, Strat Mgmmt J. 2002;23:1059-1067.

Slide 46

Teamwork-Related HSOPS Items Clarinda Regional Health Center

This bar graph entitled "Teamwork-Related HSOPS Items Clarina Regional Health Center" reports team-related item level results from the HSOPS for Clarinda Regional Health Center 2009 and 2007.

Hospital departments work well together to provide the best care for patients.

• 2009: 71%
• 2007: 76%
  

(R) Problems often occur in the exchange of information across hospital departments.

• 2009: 46%
• 2007: 46%

(R) Shift changes are problematic for patients in this hospital.*

• 2009: 59%
• 2007: 49%

Mistakes have left to positive changes here.*

• 2009: 76%
• 2007: 67%

* ≥5% Change from 2005 Baseline
(R) Reverse Worded Item

Slide 47

Percent Reporting Training in Some, All Modules or Master Trainer in 24 Critical Access Hospitals

Bar graph showing the percent reporting training in 24 critical access hospitals. Percentages range from 5% up to 95%.

Slide 48

Evaluation: Training - Knowledge

Line graph titled: Association Between TeamSTEPPS Training and Knowledge of TeamSTEPPS Tools. It shows the percent of respondents reporting training in some or all TeamSTEPPS Modules versus the percent of respondents who correctly answered 3/4 of the TeamSTEPPS Knowledge Questions.

Slide 49

Evaluation: Knowledge - Behavior

Line graph title: Association Between Knowledge of TeamSTEPPS Tools and Self-Reported Adoption of Five Team Behaviors.

Slide 50

This is a table reporting the results of a Generalized Linear Mixed Model to model the odds ratio of reporting performing 5 Behaviors most of time/always by levels of TeamSTEPPS training. We used this model to adjust for the random effect of each hospital (nesting of employees within 24 hospitals).

* Reference Group is those reporting no training in TEAMSTEPPS Modules
**P<0.0001, ***p=0.0012

Slide 51

Evaluation: Behavior - Safety

Line graph titled: Association Between Self-Reported Adoption of Five Team Behaviors and HSOPS Overall Perceptions of Safety.

Slide 52

Measuring Improvement Summary

Graphic shows how training leads to knowledge leading to changes in behavior leading to HSOPS Overall Perceptions of Safety.

Slide 53

Diffusions of Innovation Theory

• Explains why training/knowledge does not always result in changes in behavior
• Change clearly defined; better than old way
  • Trialable, Observable
• Management is supportive;
  • Change is a clear priority and is rewarded
  • Resources are available
• Champion(s) overcome resistance
• Policy/procedure/job descriptions sustain
• Effectiveness is evaluated

Rogers EM. Diffusion of Innovations. 5th ed. New York: Free Press; 2003.
Helfrich et al. Med Care Res Rev. 2007;64:279-303.
Saint S et al. Jt Comm J Qual Patient Saf. 2009;35:239-246.

Slide 54

Summary and Next Steps

• Collaboration across state and local organizations can leverage resources to diffuse TeamSTEPPS across a state and region
• Use AHRQ HSOPS to plan and evaluate TeamSTEPPS as a patient safety innovation
• Diffusion of innovations theory, Kirkpatrick's Taxonomy of Training Criteria, and decision frame are concepts needed to interpret measurement of teamwork with HSOPS
• Next Steps: More training, physician engagement, link teamwork to patient outcomes

Slide 55

Contact Information

Katherine Jones, PT, PhD
kjonesj@unmc.edu

Wendi Nordhausen, RN, BSN
wnccch@gpcom.net

Mark Goodridge, RT (R) (CT)
goodridge_mark@yahoo.com

Web site where safety culture tools and rural-adapted version of HSOPS are posted
www.unmc.edu/rural/patient-safety