Chapter 6. Comparing Your Results

2007 Comparative Database Report

To compare your hospital's survey results to the results from the database hospitals, you will need to calculate your hospital's percent positive response on the survey's 42 items and 12 composites (refer to Chapter 5 and the Notes section for a description of how to calculate these percent positive scores). You will then be able to compare your hospital's results against the database averages, and examine the percentile scores to place your hospital's results relative to the distribution of database hospitals.

When comparing your hospital's results against results from the database, keep in mind that the database only provides relative comparisons. Even though your hospital's survey results may be better than the database statistics, you may still believe there is room for improvement in a particular area within your hospital in an absolute sense. As you will notice from the database results, there are some patient safety composites that even the highest-scoring hospitals could improve upon. Therefore, the comparative data provided in this report should be used to supplement your hospital's own efforts toward identifying areas of strength and areas on which to focus patient safety culture improvement efforts.

Highlights

  • When examining differences in percent positive scores across hospitals, there was considerable variability in the range of scores comparing the lowest and highest-scoring hospitals.
    • As an indicator of this variability in scores, the average difference between the percent positive scores of the lowest and highest-scoring hospitals was 69 percent across the 12 patient safety composites, and 76 percent across the 42 survey items.
  • There was a wide range of response in patient safety grades, from at least one hospital where none of the respondents (0 percent) provided their unit with a patient safety grade of "A-Excellent," to a hospital where 63 percent did.
  • There was also a wide range of response in the number of events reported, from a hospital where 96 percent of respondents had not reported a single event over the past 12 months, to a hospital where only 5 percent had not reported an event.

Return to Contents

Description of Comparative Statistics

In addition to the average percent positive scores presented in the charts in the previous chapter (Chapter 5), a number of additional statistics are provided in this report to facilitate comparisons against the database hospitals. A description of each statistic shown in the comparative results tables in this chapter is provided next.

Average Percent Positive and Standard Deviation

The average percent positive scores for each of the 12 patient safety culture composites and for the survey's 42 items are provided in the comparative results tables in this chapter (these statistics were also displayed in the previous chapter in Charts 5-1 and 5-2). These average percent positive scores were calculated by averaging composite-level percent positive scores across all hospitals in the database, as well as averaging item-level percent positive scores across hospitals. Since the percent positive is displayed as an overall average, scores from each hospital are weighted equally in their contribution to the calculation of the average.4

In addition, the standard deviation (SD), a measure of the spread or variability of hospital scores around the average, is also displayed. The standard deviation tells you the extent to which hospitals' scores vary from the average:

  • If scores from all hospitals were exactly the same, then the average would represent all their scores perfectly and the standard deviation would be zero.
  • If scores from all hospitals were very close to the average, then the standard deviation would be small, and close to zero.
  • If scores from many hospitals were very different from the average, then the standard deviation would be a large number.

When the distribution of hospital scores follows a normal, bell-shaped curve (where most of the scores fall in the middle of the distribution, with fewer scores at the lower and higher ends of the distribution), the average, plus or minus the standard deviation, will include about 68 percent of all hospital scores. For example, if an average percent positive score across the database hospitals was 70 percent with a standard deviation of 10 percent (and scores were normally distributed), then about 68 percent of all the database hospitals would have scores between 60 percent and 80 percent.

Statistically significant differences between scores. You may be interested in determining the statistical significance of differences between your scores and the averages in the database, or between scores in various breakout categories (differences in scores by hospital bed size, teaching status, etc). Statistical significance is greatly influenced by samples sizes, so that as the number of observations in comparison groups gets larger, small differences in scores will end up being statistically significant. While a 1 percent difference between percent positive scores might be statistically significant (that is, not due to chance), the difference is not likely to be meaningful or significant in practice. Keep in mind that statistically significant differences are not always important, and nonsignificant differences are not always trivial. Therefore, we recommend the following guideline:

  • Use a 5 percent difference as a rule of thumb when comparing your hospital's results to the database averages. Your hospital's percent positive score should be at least 5 percent higher than the database average to be considered "better," and should be at least 5 percent lower to be considered "lower" than the database average. A 5 percent difference is likely to be statistically significant for most hospitals, given the number of responses per hospital, and is also a meaningful difference to consider.

 4 As noted in Chapter 5, an alternative method would be to report a straight percent of positive response across all respondents, but this method would give greater weight to respondents from larger hospitals since they account for almost twice as many responses as those from smaller hospitals.


Minimum and Maximum Scores

The minimum (lowest) and maximum (highest) percent positive scores are presented for each composite and item. These scores provide information about the range of percent positive scores obtained by hospitals in the database and are actual scores from the lowest and highest-scoring hospitals. When comparing against the minimum and maximum scores, keep in mind that these scores may represent hospitals that are extreme outliers (indicated by large differences between the minimum and the 10th percentile score, or between the 90th percentile score and the maximum).

Percentiles

 The 10th, 25th, 50th (or median), 75th and 90th percentile scores are displayed for the survey composites and items. Percentiles provide information about the distribution of hospital scores. To calculate percentile scores, all hospital percent positive scores were ranked in order from low to high. A specific percentile score shows the percent of hospitals that scored at or below a particular score. For example, the 50th percentile, or median, is the percent positive score where 50 percent of the hospitals scored the same or lower, and 50 percent of the hospitals scored higher. When the distribution of hospital scores follows a normal, bell-shaped curve (where most of the scores fall in the middle of the distribution with fewer scores at the lower and higher ends of the distribution), the 50th percentile, or median, will be very similar to the average score. Interpret the percentile scores as shown in Table 6-1.

To compare against the database percentiles, compare your hospital's percent positive scores against the percentile scores for each composite and item. Look for the highest percentile where your hospital's score is higher than that percentile.

For example: On a survey item, the 75th percentile score is 49 percent positive, and the 90th percentile score is 62 percent positive.

  • If your hospital's score on the survey item is 55 percent positive, it falls above the 75th percentile (but below the 90th), meaning that your hospital scored higher than at least 75 percent of the hospitals in the database.
  • If your hospital's score on the survey item is 65 percent positive, it falls above the 90th percentile, meaning your hospital scored higher than at least 90 percent of the hospitals in the database.

Return to Contents

Composite and Item-level Comparative Tables

 Table 6-2 presents comparative statistics (average percent positive and standard deviation, minimum and maximum scores, and percentiles) for each of the 12 patient safety culture composites. The patient safety culture composites are shown in order from the highest average percent positive response to the lowest.

 Table 6-3 presents comparative statistics for each of the 42 survey items. The survey items are grouped by the patient safety culture composite they are intended to measure, and within each composite the items are presented in the order in which they appear in the survey.

The comparative results in Tables 6-2 and 6-3 show considerable variability in the range of hospital scores (lowest to highest) across the 12 patient safety culture composites. There was a 69 percent average difference between the percent positive scores of the lowest and highest hospitals for the composites, and a 76 percent average difference for the items. The standard deviation around the average percent positive scores ranged from 6.89 percent to 11.73 percent on the composites, and ranged from 8.42 percent to 14.09 percent on the items.

 Patient safety grades shown in Table 6-4 had a wide range of response, from at least one hospital where none of the respondents (0 percent) provided their unit with a patient safety grade of "A-Excellent," to a hospital where 63 percent did.

 Number of events reported also had a wide range of response as shown in Table 6-5, from a hospital where 96 percent of respondents had not reported a single event over the past 12 months, to a hospital where only 5 percent had not reported an event.

Return to Contents

Highlights of Results in Part II—Appendixes A & B: Results by Hospital and Respondent Characteristics

In addition to the overall results on the database hospitals already presented, the report presents data tables in Appendixes A and B that show average percent positive scores on the survey composites and items across database hospitals, broken down by the following hospital and respondent characteristics:

Since there are many breakout tables, they are included in Appendixes A and B. Highlights of the findings from the breakout tables in these appendixes follow.

Return to Contents

Highlights from Appendix A: Results by Hospital Characteristics

Bed Size (Table A-1)

  • Smaller hospitals (49 beds or fewer) had the highest average positive response on all 12 patient safety culture composites.
  • The largest difference across hospitals by bed size was on Handoffs & Transitions where the smallest hospitals (6-24 beds) scored 20 percentage points+ higher than the largest hospitals (400+ beds-56 percent positive compared to 36 percent positive).
  • The smallest difference across hospitals by bed size (4 percentage points) was on Feedback & communication About Error; all other composite differences were 5 percentage points or greater.

Teaching Status, and Ownership and Control (Table A-5)

  • The largest difference across hospitals based on teaching status was on Teamwork Across Units, where nonteaching hospitals were 5 percentage points more positive than teaching hospitals (58 percent positive compared to 53 percent positive).
  • Government-owned hospitals were more positive than nongovernment owned hospitals on Staffing (6 percentage points more positive), Handoffs & Transitions (6 percent more positive), and Teamwork Across Units (5 percentage points more positive).

Region (Table A-9)

  • East South Central, West North Central, and West South Central hospitals scored highest across the 12 patient safety culture composites; Mid-Atlantic/New England, East North Central, and Pacific hospitals scored lowest.
  • The largest difference by region was on Staffing where West North Central hospitals were 15 percentage points more positive than Mid Atlantic/New England hospitals (61 percent positive compared to 46 percent positive).

Patient Safety Grade (Tables A-3, A-7, A-11)

  • Large hospitals (400+ beds) and hospitals in the Mountain region scored lowest on the percent of respondents who gave their work area/unit a patient safety grade of "Excellent" or "Very good" (64 percent for 400+ beds in Table A-3 and 60 percent for the Mountain region in Table A-11).
  • There were no noticeable differences on patient safety grade based on teaching status or ownership and control (all differences were 4 percentage points or less).

Number of Events Reported (Tables A-4, A-8, A-12)

  • There were no noticeable differences on number of events reported based on bed size, teaching status or ownership and control (all differences were 2 percentage points or less).
  • Hospitals in the Pacific region had the highest percent of respondents who had reported one or more events in the past year (54 percent); the lowest percent of respondents reporting events was 42 percent in the East South Central and West South Central regions.

Highlights from Appendix B: Results by Respondent Characteristics

Respondent Work Area/Unit (Table B-1)

  • Respondents in Rehabilitation had the highest average positive response on 9 of the 12 patient safety culture composites.
  • The largest differences (23 percent) by work area/unit were on Overall Perceptions of Patient Safety (Rehabilitation was 76 percent positive; Medicine was 53 percent positive) and Nonpunitive Response to Error (Rehabilitation was 59 percent positive; Emergency was 36 percent positive).

Respondent Staff Position (Table B-5)

  • Respondents in Administration/Management had the highest average positive response on 11 of the 12 patient safety culture composites.
  • The largest difference (27 percent) by staff position was on Nonpunitive Response to Error; Administration/Management was 60 percent positive and Patient Care Assistants Aides/Care Partners were 33 percent positive.

Respondent Interaction With Patients (Table B-9)

  • Respondents with direct patient interaction were 8 percent more positive on Handoffs & Transitions compared to those without direct patient interaction (46 percent positive compared to 38 percent positive).
  • Respondents without direct patient interaction were 7 percent more positive about Management Support for Patient Safety than those with direct patient interaction (75 percent positive compared to 68 percent positive).
  • All other composite differences were 4 percent or less.

Patient Safety Grade (Tables B-3, B-7, B-11)

  • Rehabilitation had the highest percent of respondents who gave their work area/unit a patient safety grade of "Excellent" or "Very good" (81 percent); Medicine had the lowest percent (58 percent).
  • Administration/Management had the highest percent of respondents who gave their work area/unit a patient safety grade of "Excellent" or "Very good" (79 percent); Registered Nurse/LVN/LPN had the lowest percent (64 percent).
  • There were no noticeable differences in patient safety grade based on respondent interaction with patients (differences were 2 percent or less).

Number of Events Reported (Tables B-4, B-8, B-12)

  • ICU (any type) had the highest percent of respondents reporting one or more events in the past year (69 percent); the lowest percent reporting was Anesthesiology (41 percent).
  • Pharmacists had the highest percent of respondents reporting one or more events in the past year (76 percent); the lowest percent reporting were Unit Assistants/Clerks/Secretaries (21 percent).
  • More respondents with direct patient interaction reported one or more events in the past year (52 percent) compared to those without direct patient interaction (32 percent).
Return to Contents
Proceed to Next Section
Current as of December 2012
Internet Citation: Chapter 6. Comparing Your Results: 2007 Comparative Database Report. December 2012. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/professionals/quality-patient-safety/patientsafetyculture/hospital/2007/hospdbch6.html