TeamSTEPPS 2.0: Module 10. Measurement

Classroom Slides

TeamSTEPPS is a teamwork system developed jointly by the Department of Defense (DoD)and the Agency for Healthcare Research and Quality (AHRQ) to improve institutional collaboration and communication relating to patient safety.

Contents


Slide 1: Measurement

Cover slide: Measurement. TeamSTEPPS

Return to Contents


Slide 2: Objectives

Objectives

  • Describe the importance of measurement.
  • Describe the Kirkpatrick model of training evaluation.
  • Identify measures that can be used to assess the impact of TeamSTEPPS.
  • Describe the AHRQ Surveys on Patient Safety Culture.
  • Prepare a plan for determining if TeamSTEPPS worked.

Return to Contents


Slide 3: TeamSTEPPS Phases

TeamSTEPPS Phases

The shift process shown in this slide has three phases. 

Phase I: Assessment. Pretraining assessment includes site assessment, culture survey and data/measures. Are these ready? If no, pass through climate improvement and return to pretraining assessment. If yes proceed to action plan and then move on to Phase II. 

Phase II: Planning, training and implementation. Training leads to intervention. Intervention includes testing and leads to Phase III. 

Phase III: Sustainment. This phase includes culture change: coach and integrate, monitor the plan, and continuous improvement. 

In summary: Set the stage, decide what to do, make it happen and make it stick.

The sections concerning pretraining assessment, training, and culture change are circled in red and captioned "Requires Measurement."

Return to Contents


Slide 4: How to Measure—Kirkpatrick

How to Measure - Kirkpatrick

This slide depicts a multi-level approach to Kirkpatrick's Model of Training Evaluation:

  • Level I—Reaction—Like It and Useful.
  • Level II—Learning—Think, Do, Feel.
  • Level III—Behavior— Transfer to the Job.
  • Level IV—Results—Organizational Results.

Return to Contents


Slide 5: Available Measures

Available Measures

  • Reactions:
    • Course Evaluation Form.
  • Learning:
    • Teamwork Attitudes Questionnaire (T-TAQ).
    • Learning Benchmarks.
    • Team Performance Observation Tool.
    • Teamwork Perceptions Questionnaire (T-TPQ).
  • Behavior:
    • Team Performance Observation Tool.
    • Teamwork Perceptions Questionnaire (T-TPQ).
    • AHRQ Surveys on Patient Safety Culture.
  • Results:
    • Patient outcome and clinical process measures.
    • AHRQ Surveys on Patient Safety Culture.
    • Patient Safety Indicators.

Return to Contents


Slide 6: Course Evaluation Form

Course Evaluation Form

  • Level I Reactions.
  • Found in Tab F.
  • Includes evaluation questions for all available course modules.
  • Customizable.
  • Provides information about what trainees thought about the training.

Return to Contents


Slide 7: TeamSTEPPS Teamwork Attitudes Questionnaire (T-TAQ)

TeamSTEPPS Teamwork Attitudes Questionnaire (T-TAQ)

  • Level II Learning.
  • Found in Tab F.
  • 30-item self-report tool.
  • Respondents rate their agreement with items on a 5-point Likert scale.
  • Measures attitudes toward:
    • Team Structure.
    • Leadership.
    • Situation Monitoring.
    • Mutual Support.
    • Communication.

TeamSTEPPS Materials and Tools

Return to Contents


Slide 8: Learning Benchmarks

Learning Benchmarks

  • Level II Learning.
  • Found in Tab F.
  • A 23-item multiple-choice test.
  • Uses:
    • To assess knowledge of teamwork.
    • To measure changes in knowledge.
  • Caution:
    • Items tend to be too easy.

Return to Contents


Slide 9: Team Performance Observation Tool

Team Performance Observation Tool

  • Level II Learning and Level III Behavior.
  • Found in Tab F.
  • Tool for observing team performance:
    • Site assessment.
    • Measure training effectiveness.
  • Observers should practice using the tool.
  • Can be adapted to a particular unit.

Return to Contents


Slide 10: TeamSTEPPS Teamwork Perceptions Questionnaire (T-TPQ)

TeamSTEPPS Teamwork Perceptions Questionnaire (T-TPQ)

  • Level II Learning and Level III Behavior.
  • Found in Tab F.
  • 35-item self-report tool.
  • Respondents rate their agreement with items on a 5-point Likert scale.
  • Measures staff perceptions of:
    • Team Structure.
    • Leadership.
    • Situation Monitoring.
    • Mutual Support.
    • Communication.

TeamSTEPPS Materials and Tools

Return to Contents


Slide 11: Level III: Behavior

Level III: Behavior

  • Measure whether information learned during training is transferred to the job.
  • Two important factors in transfer are:
    • Whether there is an opportunity to use the new TeamSTEPPS tools or strategies on the job.
    • Whether use of the TeamSTEPPS tools and strategies is valued and reinforced.

Return to Contents


Slide 12: Behavior: Measures

Behavior: Measures

  • Team Performance Observation Tool.
  • Teamwork Perceptions Questionnaire (T-TPQ).
  • AHRQ Surveys on Patient Safety Culture.

Return to Contents


Slide 13: Level IV: Results

Level IV: Results

  • Patient Outcome Measures:
    • Examples: Complication rates, infection rates, measurable medication errors, and patient perceptions of care and satisfaction with their care.
  • Clinical Process Measures:
    • Examples: Length of patient wait time, time to intubate, medication administration delays, compliance with preventive screenings, number of misdiagnoses, number of structured handoffs used.

Return to Contents


Slide 14: AHRQ Surveys on Patient Safety Culture

AHRQ Surveys on Patient Safety Culture

  • Free, easy-to-use surveys to assess patient safety culture.
  • Available for various contexts:
    • Hospital.
    • Medical Office.
    • Nursing Home.
    • Pharmacy.

Available on the AHRQ Web site

Return to Contents


Slide 15: Hospital Survey on Patient Safety Culture

Hospital Survey on Patient Safety Culture

  • Level III Behavior and Level IV Results.
  • Composed of 51 items and 12 composites.
  • Comprehensive instrument that assesses staff perceptions of patient safety culture:
    • Hospitalwide.
    • Specific work area or unit.
  • Comparative database:
    • Voluntarily submitted data from U.S. hospitals.

Return to Contents


Slide 16: How Can Hospitals Use the Survey on Patient Safety Culture?

How Can Hospitals Use the Survey on Patient Safety Culture?

  • Raise awareness about patient safety issues.
  • Assess patient safety culture.
  • Track changes in patient safety culture over time.
  • Evaluate the impact of patient safety interventions (e.g., TeamSTEPPS).

Return to Contents


Slide 17: Survey Measures: 12 Areas of Patient Safety and Two Overall Items

Survey Measures: 12 Areas of Patient Safety and Two Overall Items

Unit-level safety areas

  • Overall perceptions of safety.
  • Frequency of events reported.
  • Supervisor/manager expectations and actions promoting safety.
  • Organizational learning—continuous improvement.
  • Teamwork within units.
  • Communication and openness.
  • Feedback and communication about error.
  • Nonpunitive responses to error.
  • Staffing

Hosptialwide safety areas

  • Hospital management support for patient safety.
  • Teamwork across hospital units.
  • Hospital handoffs and transitions.

Overall items

  • The patient safety "grade" respondents would assign their work area/unit.
  • The number of events the respondent reported in the last 12 months.

Return to Contents


Slide 18: What Support Is Available?

What Support Is Available?

Return to Contents


Slide 19: AHRQ Patient Safety Indicators (PSIs)

AHRQ Patient Safety Indicators (PSIs)

  • Level IV Results.
  • Provide information on potential in-hospital complications and adverse events.
  • Can help identify potential adverse events that might need further study.
  • Include indicators for in-hospital complications that may represent patient safety events.
  • Quality Indicators.

Return to Contents


Slide 20: Evaluating TeamSTEPPS Exercise

Evaluating TeamSTEPPS Exercise

Three nurses and the infection control officer at a 16-bed post-surgical intensive care unit attend the 2-day TeamSTEPPS Master Training course. During the implementation planning session, the team agrees that hand hygiene compliance is a major problem, especially physician compliance. As a result, the unit has a relatively high surgical site infection rate. The team decides to use the CUS and Two-Challenge Rule tools as strategies to address this issue. As part of their implementation, they train all the nurses and the employed physician staff on these two tools, but struggle with getting surgeons who are not employed by the hospital to participate. Once the training is complete, they decide to launch the implementation of CUS and the Two-Challenge Rule.

Return to Contents
Return to Index

Current as of March 2014
Internet Citation: TeamSTEPPS 2.0: Module 10. Measurement: Classroom Slides. March 2014. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/professionals/education/curriculum-tools/teamstepps/instructor/fundamentals/module10/slmeasure.html