TeamSTEPPS® Teamwork Attitudes Questionnaire Manual

Prepared for:
U.S. Department of Defense, Tricare Management Activity
Subcontract agreement # SK-443-07-AIR-HCTCP
Prime Contract # W81XWH-06-F-0526

Prepared by:
David P. Baker, Ph.D.
Kelley J. Krokos, Ph.D.
Andrea M. Amodeo, M.S.
American Institutes for Research
1000 Thomas Jefferson Street, NW
Washington, DC 20007

Original: 24 September 2008
Revised: 7 October 2008

Contents

Introduction
Development of the T-TAQ
   Literature Review
   Item Development
   Pilot Test
   Final Item Selection
Scoring
Administration
Customization of Use
References
Appendix A
Appendix B

Introduction

This document serves as a comprehensive manual for the TeamSTEPPS Teamwork Attitudes Questionnaire (T-TAQ). It describes the development of the T-TAQ and provides detailed instructions for administering the instrument.

Background

Since the Institute of Medicine’s 1999 report, To Err is Human (Kohn, Corrigan, & Donaldson, 1999), the Agency for Healthcare Research and Quality (AHRQ) and the Department of Defense (DoD) have been Federal leaders in the patient safety movement. A major focus of these agencies has been supporting research and development activities centered on improving team performance in the delivery of care. Many organizations, such as The Joint Commission, the Institute for Healthcare Improvement, the National Quality Forum, and the Accreditation Council for Graduate Medical Education, have cited the importance of teamwork in patient safety. The National Defense Authorization Act of 2001 requires the DoD to deliver such training for staff at DoD Military Treatment Facilities (MTFs) through the Health Care Team Coordination Program, and the Joint Commission recommends hospitals establish team training programs.

In November 2006, AHRQ and DoD in collaboration released TeamSTEPPS as the national standard for team training in health care. TeamSTEPPS, which stands for Team Strategies and Tools to Enhance Performance and Patient Safety, is the result of a multiyear research and development project. Since 2005, numerous organizations and individuals have contributed to the refinement of the TeamSTEPPS curriculum.ⁱ

A cornerstone of the TeamSTEPPS approach is the three-phase process that defines the requirements for implementing the curriculum. Phase I involves conducting a site assessment to determine the teamwork-related needs within the unit. Phase II involves implementing the selected TeamSTEPPS tools and strategies, and Phase III involves sustaining the TeamSTEPPS intervention. Therefore, a critical cog in this process is measurement. Knowing that the TeamSTEPPS tools and strategies enhanced participant attitudes toward teamwork, developed participant knowledge about effective team practice, and improved team skills on the unit is critical to knowing the intervention worked.

The T-TAQ was designed to measure individual attitudes related to the core components of teamwork that are captured within TeamSTEPPS. Specifically, individual attitudes toward team structure, leadership, mutual support, situation monitoring, and communication are measured. The T-TAQ can be used to assess specific needs within the unit or health care institution and whether the TeamSTEPPS intervention produced the desired attitude change.

^i. For a complete list of contributors consult the TeamSTEPPS instructor guide, which can be ordered at http://www.ahrq.gov/teamstepps/.

Return to Contents

Development of the T-TAQ

The development of the T-TAQ involved a series of systematic steps: literature review, item development, pilot testing, item selection, and psychometric testing. Each step is described below.

Literature Review

Our search of the relevant literature revealed that few measures exist that provide assessments of individual attitudes toward teamwork, particularly ones oriented toward health care. Furthermore, none are aligned with the core components of teamwork (Salas, Sims, & Burke, 2005) that are the basis for the TeamSTEPPS program. For example, in aviation, the Cockpit Management Attitudes Questionnaire (Helmreich, 1984; Gregorich, Helmreich, & Wilhelm, 1990) assesses constructs related to crew resource management (CRM) including leadership, coordination, and communication. In health care, the Safety Climate Survey measures perceptions of organizational commitment to patient safety through constructs such as commitment to safety, leadership, interpersonal interactions, attitudes toward stress, and knowledge of how to report adverse events (Sexton, et al., 2006). The Safety Attitudes Questionnaire measures hospital providers’ attitudes about teamwork climate, safety climate, perceptions of management, job satisfaction, working conditions, and stress recognition (Sexton & Thomas, 2003).

Perhaps the most germane measure to the current discussion is AHRQ’s Hospital Survey on Patient Safety (HSOPS). HSOPS measures 12 dimensions related to patient safety culture. Two of these scales focus specifically on teamwork: teamwork within units and teamwork between units. Hundreds of institutions have administered HSOPS, and national norms are available. TeamSTEPPS recommends institutions consider using HSOPS as part of their site assessment for determining their teamwork needs and as an evaluation tool to determine whether HSOPS scores improve as a function of TeamSTEPPS implementation. However, the HSOPS focuses on safety culture, and the teamwork scales do not partition out the critical subdomains of team performance. Therefore, our review of the literature supported the need for the development of the T-TAQ.

Return to Contents

Item Development

We developed items on the basis of a test specification that listed the domains to be assessed and the number of items to write for each domain. A test specification ensures that content validity is built into the item development process. Content validity relates to whether the items provide appropriate coverage of the core constructs one is attempting to measure. In this case, we were interested in assessing attitudes toward the core components taught within TeamSTEPPS: team structure, leadership, situation monitoring, mutual support, and communication.

We developed items through an extensive item-writing process that included multiple item writers who were experienced in survey and test development and were also knowledgeable in the principles of teamwork and, more specifically, the TeamSTEPPS training curriculum upon which the items were to be based. As items were drafted, writers linked each item to a specific TeamSTEPPS curriculum module and identified the page in the manual from which the item was written. Throughout the item-writing process, this linking ensured that items captured key teamwork concepts. Item-writing efforts resulted in a pool of 110 items.

Next, we conducted an item review. Item review criteria included social desirability concerns. For example, we used caution with including the phrase "it is important to" in an item, as this may trigger a positive response (i.e., agreement with the item), regardless of the true attitude of the respondent. We also reviewed items to ensure that they asked the respondent about his/her attitude and not that of other team members or the team as a whole. Further reviews sought to eliminate items that were repetitive in terms of teamwork concepts and to limit the number of items within each of the constructs to approximately 10 items. At the end of this review, 49 items remained in the pool. Table 1 illustrates the constructs measured by the piloted T-TAQ and the corresponding number of items.

Table 1. Number of Pilot Test Items by TeamSTEPPS Construct

Return to Contents

Pilot Test

The process and outcome of the pilot test are described in more detail below.

Data Collection. We administered the pilot version of the T-TAQ to TeamSTEPPS training participants (n=346) at various MTFs throughout the DoD between December 2007 and April 2008. We also distributed the pilot version to participants (n=149) in a mid-Atlantic critical care conference that occurred in Spring 2008. For purposes of this research, we identify this sample as our "civilian" sample to distinguish it from the DoD sample. Participants for the DoD sample completed the pilot version of the T-TAQ for either prior to or immediately following TeamSTEPPS training at the discretion of the individual training facilitators. The T-TAQ for the civilian sample was administered during the conference. All questionnaires were completed in paper form.

Data Quality. To ensure the integrity of the data, we conducted analyses to remove cases with excessive missing data and anomalous response patterns. Specifically, cases in which respondents did not respond to at least 95 percent of the items or answered at least 95 percent with the same response were removed from the data set. In addition, when respondents provided multiple responses to an item, or the response was unclear, the data point was coded as "missing." This resulted in a loss of 46, or 9.8 percent, of the cases and subsequent reduction in the total sample size from 495 to 449.

Final Sample. As a result of the data "cleaning" described above, 449 cases were included in the analyses, with n=311 from the DoD and n=138 from the civilian participants. The large majority of the respondents (N=408 or 91.7 percent) reported that they deliver direct care to patients. Of these, n=175 or 44.3 percent reported they deliver inpatient care, n=89 or 22.5 percent reported they deliver outpatient care, and n=131 or 33.2 percent reported they deliver both inpatient and outpatient care. A total of 211 respondents reported their position as physicians or dentists.ⁱⁱ Eighty-five respondents reported they work as registered nurses and 27 as advanced practice nurses. Respondents reported working predominantly in intensive care: 260 respondents reported working in some type of intensive care unit (ICU), such as the general ICU, a cardiac ICU, or a pediatric ICU. An additional seven respondents reported working specifically in a neonatal ICU. Eighteen respondents reported working in an emergency department and an equal number in a pediatric unit. Both the DoD and civilian administrations asked respondents whether they had prior team training experience, of which the majority (n=242 or 54.3 percent) reported they did not.

Return to Contents

Final Item Selection

We used a classical item statistics approach and criteria regarding the desired length of the final T-TAQ to select items. First, we computed and analyzed means, item-total correlations, and standard deviations. We reviewed items and constructs using these analyses and Cronbach’s Alpha "If Item Deleted" results. In addition, we considered the number of items per construct, as we desired scales of equal numbers of items, as well as a final T-TAQ that would be of suitable length (i.e., not requiring much participant time). This process resulted in the deletion of 19 items from our pilot measure.

The final questionnaire (presented in Appendix A) includes a total of 30 items; 6 items for measuring each teamwork construct. Among the final items, four are reverse coded: three in mutual support and one in communication.ⁱⁱⁱ Final constructs and their alpha values are provided in Table 2. Finally, we examined construct independence by intercorrelating the five subscales that comprise the T-TAQ. Table 3 provides Pearson correlation coefficients among the constructs, with coefficients ranging from .36 (mutual support and team structure) to .63 (situation monitoring and communication). These results suggest that while the constructs overlap to some degree, they do also exhibit unique variance.

Table 2. T-TAQ Reliability Coefficients

Table 3. T-TAQ Construct Inter-Correlations

*p < .01, two-tailed.

ii. DoD and civilian questionnaires included different demographic items. This was due to the specific needs and interests of each group. Some items (e.g., respondent position, department) were the same but provided different response options. In addition, some respondents provided multiple responses for the position and department in which they worked. Therefore, information about each position or department or unit in which respondents work is not necessarily unique to that position or department or unit and may have been collapsed into a more general response category for reporting purposes. Due to this issue, valid percentages are not reported.

iii. The T-TAQ reverse-coded items are mutual support items 20, 21, 24, and communication item 30.

Return to Contents

Scoring

T-TAQ scoring can take one of two forms. A total score may be calculated for each teamwork construct or an average score may be derived. Table 4 presents means, ranges, and standard deviations for each construct captured in the T-TAQ. As can be seen in Table 4, respondents had very favorable attitudes toward teamwork across constructs.

Table 4. Means, Range, and Standard Deviations

Return to Contents

Administration

The T-TAQ may be administered as a stand-alone measure to assess attitudes toward the core components of teamwork, as part of an institution’s site assessment to determine training needs, or as a tool to evaluate TeamSTEPPS training. In all of these cases, the basic administration of the T-TAQ should remain the same (refer to Appendix B). What will vary, as noted in the administration options below, is when the T-TAQ is administered. Detailed administration instructions are presented in Appendix B.

Stand-Alone. As an independent assessment of teamwork attitudes, the T-TAQ may be administered at any point in time. In this capacity, the T-TAQ may be administered organization-wide, unit-wide, or to some combination of units. Careful consideration should be given to identifying the population of interest to be surveyed so the correct staff members receive the T-TAQ.

TeamSTEPPS Site Assessment. The T-TAQ may be used as one component of the TeamSTEPPS site assessment process. In this capacity, the T-TAQ should be administered prior to TeamSTEPPS training. Results can be used to identify where poor attitudes toward teamwork exist within a unit or institution and can therefore assist the organization’s TeamSTEPPS change team in selecting specific TeamSTEPPS interventions.

TeamSTEPPS Evaluation. The T-TAQ may be used to assess TeamSTEPPS effectiveness. The basic question answered here is whether the TeamSTEPPS intervention produces desirable attitude changes regarding teamwork. To answer this question, the T-TAQ should be administered immediately prior to and after TeamSTEPPS training. Results from these two data collections should then be compared. The T-TAQ may also be administered several months after TeamSTEPPS training to see if the attitude changes that were achieved at the end of training are sustained.

Return to Contents

Customization of Use

Organizations may have different needs with regard to teamwork development. This section briefly describes appropriate options for customization of the T-TAQ.

Background Questions. Background questions regarding a participant’s current job, professional training, age, experience, etc. are not included on the T-TAQ. We have found through our experience that various health care units are referred to differently across organizations, so it is best if organizations develop customized background questions. We recommend that each institution develop its own questions regarding participant characteristics and append these questions to the back of the questionnaire.

Item Modification. Items on the T-TAQ should not be modified. Changing the items can affect the reliability and validity of the instrument.

Scale Use. Scales from the instrument can be used separately if desired. For example, if an organization was interested only in attitudes toward leadership, then this scale could be administered independently. The administration instructions found in Appendix B would still apply.

Return to Contents

References

Gregorich, S. E., Helmreich, R. L., & Wilhelm, J. A. (1990). The structure of cockpit management attitudes. Journal of Applied Psychology 75, 682-690.

Helmreich, R. L. (1984). Cockpit management attitudes. Human Factors, 26, 583-589.

Kohn, L. T., Corrigan, J. M., & Donaldson, M. S. (1999). To err is human. Washington, DC: National Academy Press.

Salas, E., Sims, D. E., & Burke, C. S. (2005). Is there a big 5 in teamwork? Small Group Research 36(5), 555-599.

Sexton J. B., Helmreich, R. L., Neilands, T. B., Rowan, K., Vella, K., Boyden, J., et al. (2006). The safety attitudes questionnaire: Psychometric properties, benchmarking data, and emerging research. BMC Health Services Research, 6(44).

Sexton, J. B., & Thomas, E. J. (2003). The safety climate survey: Psychometric and benchmarking properties (Technical Report 03-03). Houston, TX: The University of Texas Center of Excellence for Patient Safety Research and Practice.

Return to Contents

Appendix A: TeamSTEPPS Teamwork Attitudes Questionnaire

The purpose of this survey is to measure your impressions of various components of teamwork as it relates to patient care and safety.

Instructions: Please respond to the questions below by placing a check mark (√) in the box that corresponds to your level of agreement from Strongly Agree to Strongly Disagree. Please select only one response for each question. 

Please provide any additional comments in the space below.

Thank you for your participation!

Return to Contents

Appendix B: TeamSTEPPS Teamwork Attitudes Questionnaire Administration Instructions

Read the following instructions aloud prior to administering the T-TAQ:

Please complete the following questionnaire by placing a check mark [√] in the box that corresponds to your level of agreement from Strongly Disagree to Strongly Agree. Please answer every question and select only one response for each question. The questionnaire is anonymous, so please do not put your name or any other identifying information on the questionnaire.

[Optional]: On the last page you will find questions about your background and experience. Please provide your responses to each question in the space provided. Thank you for your participation.

Return to Contents