Chapter 7. Trending: Comparing Results Over Time

Many hospitals that have administered the hospital patient safety culture survey have indicated that they intend to readminister the survey on a regular basis to track changes in patient safety culture over time. Some of the hospitals that previously administered the survey and submitted data for the 2008 report also submitted data for the 2009 report based on a followup survey of their staff. While the overall indicators presented earlier in this report reflect only the most recent survey data from all 622 participating hospitals, we have data from two or more administrations of the survey for 204 hospitals, allowing us to examine trends over time for these hospitals. This chapter presents the results from trend analyses comparing patient safety culture survey results for these 204 hospitals since their previous administration. Changes in scores of 5 percent or greater are highlighted.

Highlights For the 204 hospitals with trending data, the average time between previous and most recent survey administrations was 16 months (range: 7 months to 35 months). The average change in percent positive scores between administrations on the patient safety culture composites was a slight increase of 2 percent (ranging from 1 to 3 percent change). Thirty-seven percent of trending hospitals increased by 5 percent or more on Overall Perceptions of Patient Safety (Chart 7-1). Twenty-two percent of hospitals decreased in percent positive scores by 5 percent or more on Organizational Learning—Continuous Improvement (Chart 7-1). There were no noticeable differences in changes the percentage of respondents who gave their work area/unit a patient safety grade of "A-Excellent" and "B-Very Good" (average percentage increased by 4 percent). There were no noticeable differences in the number of events reported by respondents in the past 12 months (the average percentage of respondents reporting one or more events increased by only 2 percent).

When reviewing the results in this chapter, keep in mind that the trending results from these 204 hospitals represent approximately one-third of the total number of database hospitals. Therefore, the trending data should be viewed as preliminary. In addition, survey scores might change, or not change, over time for a number of complex reasons. Important factors to consider are whether the hospital implemented patient safety initiatives between survey administrations and the length of time between administrations. Survey methodology issues can also play a big role in score changes. Low survey response rates for the previous or most recent administration, changes in the number of staff asked to complete the survey, or changes in the types of staff asked to complete the survey, will make it difficult to interpret changes in scores over time. We provide descriptive information about some of the factors that may have affected changes in scores where possible.

Return to Contents

Characteristics of the 204 Trending Hospitals

Table 7-1 displays summary statistics from the previous and most recent survey administrations for the 204 trending hospitals. As shown in the table, the average number of completed surveys increased in the most recent survey administration (from an average of 320 to 341 respondents). Average response rates were similar between the previous and most recent administrations. Additional characteristics of the 204 hospitals are below:

• Most of the 204 trending hospitals (74 percent) administered the survey to the same types of staff in their previous and most recent administrations.
• The average change in response rate from the previous administration was 2 percent (range: one hospital had a 90 percent decrease in response rate and one had a 79 percent increase).
• The average time between the previous and most recent survey administrations was 16 months (range: 7 months to 35 months).

As shown in Table 7-2, the distribution of trending hospitals by bed size is similar to the distribution of AHA-registered U.S. hospitals, as well as the distribution of database hospitals. Similar to the AHA-registered U.S. hospitals, the largest group of trending hospitals (42 hospitals or 21 percent) fall in the bed size category of 25 to 49 beds. Most of the trending hospitals (132 hospitals, or 65 percent) have fewer than 200 beds, which is similar to the percentage of AHA-registered U.S. hospitals with fewer than 200 beds (74 percent). The trending hospitals, however, disproportionately represent a larger percentage of large hospitals (500 ore more beds), with more than twice the percentage of hospitals compared with the AHA-registered U.S. hospitals (12 percent versus 5 percent).

 Tables 7-3 and 7-4 show that most of the 204 trending hospitals were nonteaching (71 percent) and non-government owned and controlled (69 percent). Again, these distributions vary compared with the 2009 database overall (69 percent nonteaching and 22 percent government owned) and when compared with AHA hospitals (77 percent nonteaching and 26 percent government owned). Therefore, the trending hospitals disproportionately represent a larger percentage of non-teaching hospitals and a larger percentage of government-owned hospitals.

Return to Contents

Description of Trending Statistics

Before presenting results on the changes in survey scores over time, we provide an explanation of the trending statistics that are presented. Table 7-5a shows examples of the statistics shown in this chapter. The tables show the average percentage of respondents who answered positively in the most recent survey administration (left column) and the previous administration (middle column) for the 204 trending hospitals only. The change over time [Most Recent score minus (-) Previous score] is shown in the right column. the change is a negative number if the most recent administration showed a decline and a positive number if the most recent administration showed an increase.

Table 7-5b shows additional trending statistics that are provided. The maximum increase and maximum decrease show the scores for the hospitals with the largest average percent positive score increase and the hospitals with the largest decrease. The average increase and decrease of percent positive scores across the 204 trending hospitals is also shown. The average increase was calculated by only including hospitals that had an increase in their most recent score; hospitals that showed no change or decreased were not included when calculating the average increase. Similarly, the average decrease was calculated by only including hospitals that had a decrease in their most recent score; hospitals that showed no change or increased were not included when calculating the average decrease.

The pie charts in Charts 7-1, 7-2, and 7-3 show the percentage of hospitals that increased or decreased 5 percent or more on the composites, patient safety grades, and events reported, respectively. The percentage of hospitals that increased or decreased less than 5 percent are represented as "Did not change."

Return to Contents

Composite and Item-Level Trending Results

Table 7-6 presents trending results showing average percent positive scores on each of the 12 patient safety culture composites from the 204 trending hospitals. The table shows percent positive scores for the hospitals' most recent and previous data administration/submission. The table also shows the percentage of change over time, the hospital scores with the maximum increase and maximum decrease, and the average increase and decrease over time across the 204 hospitals. Table 7-6 also shows a slight overall increase in the average change in percent positive scores over time on the patient safety culture composites (average 2 percent, ranging from 1 to 3 percent change). For hospitals with increases in scores over time, average increases ranged from 5 to 8 percent. For hospitals with decreases in scores, average decreases ranged from -4 to -6 percent.

The item-level trending results in Table 7-7 show that the average change in item-level percent positive scores over time on the patient safety culture items ranged from a 1 percent increase to a 4 percent increase. For hospitals with increases in item scores over time, average increases ranged from 6 to 10 percent. For hospitals with decreases in item scores, average decreases ranged from -4 to -9 percent.

Trending results from the item that asks respondents to give their hospital work area/unit an overall grade on patient safety are shown in Table 7-8. The average percentage of respondents giving their work area/unit a patient safety grade of "A-Excellent" and "B-Very Good" increased over time by 4 percent.

Trending results from the item that asked respondents to indicate the number of events they had reported over the past 12 months are shown in Table 7-9. The average percentage of respondents reporting one or more events increased slightly over time by 2 percent.

Return to Contents

Pie Charts of Trending Results

The pie charts in Chart 7-1 show the percentage of hospitals that increased, decreased, or did not change by 5 percent or more on the 12 patient safety culture composites.  These charts show that:

• The composite with the largest percentage of hospitals that increased 5 percent or more was Overall Perceptions of Patient Safety (37 percent of trending hospitals increased by at least 5 percent).
• The composite with the largest percentage of hospitals that decreased 5 percent or more was Organizational Learning-Continuous Improvement (22 percent of trending hospitals decreased by at least 5 percent).

Chart 7-2 displays results for the percentage of hospitals that increased, decreased, or did not change by 5 percent or more on the percentage of respondents providing patient safety grades of "A-Excellent" or "B-Very Good" and shows that:

• 38 percent of hospitals increased by 5 percent or more;
• 41 percent of hospitals had changes of less than 5 percent; and
• 21 percent of hospitals decreased by 5 percent or more.

Chart 7-3 displays results for the percentage of hospitals that increased, decreased, or did not change by 5 percent or more on the percentage of respondents reporting one or more events and shows that:

• 32 percent of hospitals increased by 5 percent or more;
• 46 percent of hospitals had changes of less than 5 percent; and
• 23 percent of hospitals decreased by 5 percent or more.

Return to Contents

Additional Trending Analyses

The following sections present quantitative and qualitative data on changes in patient safety culture over time. The quantitative data include questionnaire data on actions taken by the trending hospitals to improve their patient safety culture, as well as correlations between improvement efforts and hospital survey scores. The qualitative data consist of findings from nine interviews conducted with staff at trending hospitals and suggest explanations for increases and decreases in hospitals' hospital survey scores.

Actions Taken by the Trending Hospitals

About 81 percent (165) of the 204 trending hospitals (hospitals that administered the patient safety culture survey and submitted data more than once) provided basic information about the types of patient safety actions they had taken in between their previous and most recent survey administrations.

Most of the trending hospitals that provided information about improvement efforts (153 hospitals, or 93 percent) reported that they had shared their previous survey results with hospital administrators. In addition, 76 percent (125 hospitals) reported they had also shared their previous survey results with hospital staff, but fewer had shared the results with their Board of Directors (100 hospitals or 61 percent) or with physicians (100 hospitals or 61 percent). Table 7-10 shows the percentages of trending hospitals that reported they had implemented various types of actions. The action most frequently taken was implementing the Situation-Background-Assessment-Recommendation (SBAR) technique (95 hospitals or 58 percent). About 10 percent (17 hospitals) indicated they had developed action plans but had not implemented them yet.

Most of the trending hospitals providing information on improvement efforts (151 hospitals, or 92 percent) indicated they had implemented more than one action (Table 7-10). Hospitals described the types of "other" actions implemented, such as:

• Patient Safety Champion/Representative programs.
• Color-coded wristbands.
• Hand hygiene programs.
• Electronic medical records.
• Medication error reduction strategies.

Given that the average time between survey administrations was 16 months, it appears that the trending hospitals were able to begin implementing these activities within a relatively short time after their previous survey administration.

Correlational Analyses

To explore potential reasons that some hospitals had increases in their patient safety culture scores over time, we examined the relationship between hospital characteristics—such as bed size, ownership, and teaching status—and changes in patient safety culture scores over time. We examined relationships by calculating correlations between hospital characteristics and the number of composites increasing by 5 percent or more per hospital. In addition, hospital characteristics were correlated with the percentage change in respondents giving their hospital a patient safety grade of Excellent or Very Good and the percentage change in respondents reporting one or more events. Correlations (r) are a type of statistic that convey the extent to which two variables have a linear relationship. Correlations range from a low of 0 to a high of 1.00 and can be either positive or negative. The closer the correlation is to 1.00 (or -1.00), the greater the degree of association between the variables.  A correlation is considered statistically significant or not due to chance when the p-value is less than .05 (p <.05).

The following relationships were found between hospital characteristics and changes in patient safety culture scores. These findings should be considered preliminary, as they are based on a relatively small sample of 204 trending hospitals.

• The smaller the hospital bed size, the greater the number of patient safety culture composites that increased by at least 5 percent (correlation: r = -.21, p < .05) and the greater the increase in respondents reporting one or more events (r = -.19, p < .05).
• Non-teaching hospitals tended to increase by 5 percent or more on the composites more than teaching hospitals (r = .15, p < .05) and tended to have greater increases in respondents reporting one or more events than teaching hospitals (r = .17, p < .05).
• Government hospitals tended to have greater increases in respondents giving their hospital a patient safety grade of Excellent or Very Good (r = .17, p < .05) and respondents reporting one or more events (r = .20, p < .05) than non-government hospitals.

We also examined whether hospitals that improved on Nonpunitive Response to Error also had increases in the number of respondents who reported at least one event in the past 12 months. This finding was supported; hospitals that increased their percent positive score on Nonpunitive Response to Error also tended to have an increase in the number of respondents who reported at least one event in the past 12 months (correlation: r = .14, p < .05).

Interview Findings

To gain a better understanding of changes in patient safety culture and patient care practices over time, HSOPS project team members conducted hour-long telephone interviews with staff from nine hospitals that administered the HSOPS more than once. Six of the hospitals experienced notable increases in their scores, and three hospitals experienced notable decreases. Most interview participants were quality/risk managers, and one was a chief executive officer. The nine hospitals varied with respect to system affiliation, bed size, teaching status, ownership, and geographic region.

Explanations for notable increases in HSOPS scores. During the interviews, participants were asked why their hospitals' HSOPS scores increased. Some participants mentioned specific actions including implementing the SBAR communication tool for unit-to-unit transfers, hiring a consultant group to work with department directors on targeted patient safety problems, addressing staffing requirements such as filling nursing vacancies and improving patient/staff ratios, and using and displaying scorecards to monitor progress on hospital initiatives. Generally, various themes emerged from their responses. These themes are shared here, along with participants' comments about actions taken by their hospitals to improve patient safety culture and safe patient care practices. Four main themes emerged from those hospitals with notable increases in their HSOPS scores.

Theme 1: Hospitals improved their communication between management and staff on patient safety.

Sample Actions and Illustrative Quotes

• Conducted walkarounds to learn about staff concerns about patient safety
• Focused on patient safety during staff meetings
• • One participant attributed her hospital's improvement to "the engagement of our department heads and nursing coordinators in making sure patient safety culture is on everyone's mind."
• Started conducting monthly staff meetings
• Implemented Open Book Management and participated in biweekly "huddles" to review the hospital budget, financial statements, and patient safety issues and concerns
• • "Open Book Management has had the biggest impact of all their initiatives...affected everything we do...employees are much more aware."

Theme 2: Hospitals focused on improving error reporting systems, responding appropriately to reports, and applying nonpunitive "Just Culture" principles.

Sample Actions and Illustrative Quotes

• Educated hospital leaders on making error reporting anonymous, easy, and convenient
• • "When we went from a paper system to an electronic system, our reporting increased about 40 percent—part of it was education, because we had to do a lot of education as we rolled out the electronic system—part of it...is because it's very easy."
• Set up a hotline for reporting errors and developed anonymous reporting forms for medical errors
• • "We got management to buy into that it was okay for a staff person to not provide their name, so they wouldn't be afraid to report."
• Trained staff to use the new reporting systems
• Provided training on "Just Culture" and taught managers to use an algorithm when examining patient safety error incidents
• • "The algorithm helps management more than anything else."

Theme 3: Hospitals engaged staff in developing solutions to patient safety problems.

Sample Actions and Illustrative Quotes

• Directly involved staff in designing solutions to handoff problems
• Started an employee engagement committee that includes senior leaders
• Instituted nursing peer review to promote open communication
• • "I personally think it is a combination of the employee engagement committee where employees have a voice. I think it's the peer review...having peers to go to, to voice your concerns."
• Assigned staff to a scheduling team to accommodate staff preferences
• Allocated resources for safety needs identified by staff—for example, buying safer beds

Theme 4: Hospitals developed, implemented, and monitored action plans, in some cases focusing on specific survey items.

Sample Action

• Charged department managers with developing and implementing an annual action plan and held them accountable

Explanations for notable decreases in hospital survey scores. Hospital participants provided the following explanations as possible reasons for decreases in their HSOPS scores in their most recent administration of the survey.

• Experienced issues among staff with specific managers and management styles, especially regarding managers' response to incident reports and lack of followup on staff feedback
• • "They felt like the managers really didn't act on them [incident reports] or hear them or do anything about them..."
• Had contracting issues and high turnover for managers and frontline staff—staff have had to get used to new unit managers; some new managers not familiar with hospital policies on "Just Culture"
• Needed to temporarily shut down hospital services because contract and financial constraints led to a large shortage of professional providers
• • "The staffing issue came up as part of contract problems. We're in a fairly isolated area, and we have a vacancy rate in the professional provider staff of about 40%. During this time frame we also changed financial management systems. We're not able to hire contractors with the speed that we had in the past. We ended up running very short and ended up closing beds and shutting services down for about an 18-month period."
• Drilled down in the survey data and observed that scores were lower for larger than smaller units—attributed the lower scores to less frequent and personal communications, weaker sense of accountability to coworkers
• Were in the middle of union negotiations and staff were feeling hostile
• Struggled with organizational learning and how much information could be fed back to staff given confidentiality requirements and concerns
• • "As we run into significant adverse events for patients, how much do we feed the information back to frontline staff? Where's that line of keeping it confidential yet sharing our learnings with staff?"

Return to Contents

Appendixes C and D: Trending Results by Hospital and Respondent Characteristics

Part III of the report contains Appendixes C and D that show trends over time for the 204 hospitals that administered the survey and submitted data more than once. Average percent positive scores from the most recent and previous administrations are shown on the survey composites and items, broken down by the following hospital and respondent characteristics:

• Appendix C: Trending Results by Hospital Characteristics
  • Bed size
  • Teaching status
  • Ownership and control

• Appendix D: Trending Results by Respondent Characteristics
  • Work area/unit
  • Staff position
  • Interaction with patients

Because there are many breakout tables, they are included in Appendixes C and D. Highlights of the findings from the breakout tables in these appendixes are provided on the following pages.

Note: Because there were fewer than 20 trending hospitals in several hospital region breakout categories, trending results are not shown by hospital region to ensure hospital confidentiality.