Chapter 6. Comparing Your Results

2011 User Comparative Database Report

To compare your nursing home's survey results with the results from the database, you will need to calculate your nursing home's percent positive response on the survey's 42 items and 12 composites (plus the two questions on resident safety grade and number of events reported). Refer to the Notes section at the end of this report for a description of how to calculate these percent positive scores. You will then be able to compare your nursing home's results with the database averages and examine the percentile scores to place your nursing home's results relative to the distribution of database nursing homes.

When comparing your nursing home's results with results from the database, keep in mind that the database provides only relative comparisons. Even though your nursing home's survey results may be better than the database statistics, you may still believe there is room for improvement in a particular area within your nursing home in an absolute sense. As you will notice from the database results, there are some patient safety composites that even the highest scoring nursing homes could improve on. Therefore, the comparative data provided in this report should be used to supplement your nursing home's own efforts toward identifying areas of strength and areas on which to focus patient safety culture improvement efforts.

Highlights

  • There was considerable variability in the range of nursing home scores (lowest to highest) across the 12 patient safety culture composites.
  • Willingness to recommend one's nursing home also had a wide range of response. In one nursing home, 28 percent of respondents indicated that they were willing to recommend their nursing home, yet at another nursing home, 100 percent did.
  • Overall rating on resident safety showed a wide range of response as well. In at least one nursing home, none of the respondents (0 percent) provided their unit with a rating of "Excellent," and at another nursing home, 82 percent did.

Description of Comparative Statistics

In addition to the average percent positive scores presented in Chapter 5, a number of other statistics are provided to facilitate comparisons with the database nursing homes. A description of each statistic shown in this chapter is provided next.

Average Percent Positive

The average percent positive scores for each of the 12 patient safety culture composites and for the survey's 42 items (plus the two questions on recommending this nursing home and overall rating on resident safety) are provided in the comparative results tables in this chapter. These average percent positive scores were calculated by averaging composite-level percent positive scores across all nursing homes in the database, as well as averaging item-level percent positive scores across nursing homes. Since the percent positive is displayed as an overall average, scores from each nursing home are weighted equally in their contribution to the calculation of the average.v

Standard Deviation

The standard deviation (s.d.), a measure of the spread or variability of nursing home scores around the average, is also displayed. The standard deviation tells you the extent to which nursing homes' scores differ from the average:

  • If scores from all nursing homes were exactly the same, then the average would represent all their scores perfectly and the standard deviation would be zero.
  • If scores from all nursing homes were very close to the average, then the standard deviation would be small and close to zero.
  • If scores from many nursing homes were very different from the average, then the standard deviation would be a large number.

When the distribution of nursing home scores follows a normal bell-shaped curve (where most of the scores fall in the middle of the distribution, with fewer scores at the lower and higher ends of the distribution), the average, plus or minus the standard deviation, will include about 68 percent of all nursing home scores. For example, if an average percent positive score across the database nursing homes were 70 percent with a standard deviation of 10 percent (and scores were normally distributed), then about 68 percent of all the database nursing homes would have scores between 60 and 80 percent.

Statistically "significant" differences between scores. You may be interested in determining the statistical significance of differences between your scores and the averages in the database, or between scores in various breakout categories (nursing home bed size, ownership, etc.). Statistical significance is greatly influenced by sample size, so as the number of observations in comparison groups gets larger, small differences in scores will be statistically significant. While a 1 percentage point difference between percent positive scores might be "statistically" significant (that is, not due to chance), the difference is not likely to be meaningful or "practically" significant.

Keep in mind that statistically significant differences are not always important, and non significant differences are not always trivial. Therefore, we recommend the following guideline:

  • Use a 5 percentage point difference as a rule of thumb when comparing your nursing home's results with the database averages. Your nursing home's percent positive score should be at least 5 percentage points greater than the database average to be considered "better" and should be at least 5 percentage points less to be considered "lower" than the database average. A 5 percentage point difference is likely to be statistically significant for most nursing homes given the number of responses per nursing home and is also a meaningful difference to consider.

Minimum and Maximum Scores

The minimum (lowest) and maximum (highest) percent positive scores are presented for each composite and item. These scores provide information about the range of percent positive scores obtained by nursing homes in the database and are actual scores from the lowest and highest scoring nursing homes. When comparing with the minimum and maximum scores, keep in mind that these scores may represent nursing homes that are extreme outliers (indicated by large differences between the minimum score and the 10th percentile score, or between the 90th percentile score and the maximum score).

Percentiles

 

The 10th, 25th, 50th (or median), 75th, and 90th percentile scores are displayed for the survey composites and items. Percentiles provide information about the distribution of nursing home scores. To calculate percentile scores, all nursing home percent positive scores were ranked in order from low to high. A specific percentile score shows the percentage of nursing homes that scored at or below a particular score. For example, the 50th percentile, or median, is the percent positive score where 50 percent of the nursing homes scored the same or lower and 50 percent of the nursing homes scored higher. When the distribution of nursing home scores follows a normal bell-shaped curve (where most of the scores fall in the middle of the distribution, with fewer scores at the lower and higher ends of the distribution), the 50th percentile, or median, will be very similar to the average score. Interpret the percentile scores as shown in Table 6-1.

To compare with the database percentiles, compare your nursing home's percent positive scores with the percentile scores for each composite and item. Look for the highest percentile where your nursing home's score is higher than that percentile.

For example: On survey item 1 in Table 6-2, the 75th percentile score is 49 percent positive, and the 90th percentile score is 62 percent positive.

  • If your nursing home's score is 55 percent positive, it falls above the 75th percentile (but below the 90th), meaning that your nursing home scored higher than at least 75 percent of the nursing homes in the database.
  • If your nursing home's score is 65 percent positive, it falls above the 90th percentile, meaning your nursing home scored higher than at least 90 percent of the nursing homes in the database.

Composite and Item-Level Comparative Tables

Table 6-3 presents comparative statistics (average percent positive and standard deviation, minimum and maximum scores, and percentiles) for each of the 12 patient safety culture composites. The patient safety culture composites are shown in order from the highest average percent positive response to the lowest.

Table 6-4 presents comparative statistics for each of the 42 survey items. The survey items are grouped by the patient safety culture composite they are intended to measure. Within each composite, the items are presented in the order in which they appear in the survey.

The comparative results in Tables 6-3 and 6-4 show considerable variability in the range of hospital scores (lowest to highest) across the 12 patient safety culture composites. The standard deviation around the average percent positive scores ranged from 5.84 percent to 11.08 percent on the composites and ranged from 5.62 percent to 12.65 percent on the items.

Willingness to recommend one's nursing home, shown in Table 6-5, had a wide range of response. In one nursing home, 28 percent of respondents indicated that they were willing to recommend their nursing home, yet at another nursing home, 100 percent did.

Overall ratings on resident safety also had a wide range of response, as shown in Table 6-6, from at least one nursing home where none of the respondents (0 percent) provided their unit with a rating of "Excellent" to a nursing home where 82 percent did.

Appendixes A and B: Overall Results by Nursing Home and Respondent Characteristics

In addition to the overall results on the database nursing homes presented, Part II of the report presents data tables showing average percent positive scores on the survey composites and items across database nursing homes, broken down by the following nursing home and respondent characteristics:

Appendix A: Results by Nursing Home Characteristics

  • Bed size.
  • Ownership.

Appendix B: Results by Respondent Characteristics

  • Job title.
  • Work area.
  • Interaction with residents.
  • Shift worked most often.

The breakout tables are included as appendixes because there are a large number of them. Highlights of the findings from the breakout tables in these appendixes are provided on the following pages. The appendixes are available on the Web at: http://www.ahrq.gov/professionals/quality-patient-safety/patientsafetyculture/nursing-home/2011/index.html.

Highlights from Appendix A: Overall Results by Nursing Home Characteristics

Bed Size (Tables A-1, A-3, A-4)

  • Small nursing homes (49 or fewer beds) had the highest average percent positive response on 10 of the 12 patient safety culture composites.
  • Small nursing homes (49 or fewer beds) had the highest percentage of respondents who indicated they would tell their friends that this is a safe nursing home for their family (88 percent for 49 or fewer beds versus 74 percent for 100-199 beds).
  • Small nursing homes (49 or fewer beds) had the highest percentage of respondents who gave their nursing home an overall rating on resident safety of "Excellent" or "Very Good" (77 percent for 49 or fewer beds versus 59 percent for 100-199 beds and 200 beds or more).

Ownership (Tables A-5, A-7, A-8)

  • Nonprofit/government nursing homes had a higher average percent positive response than for profit nursing homes on all 12 patient safety culture composites.
  • Nonprofit/government nursing homes had a higher percentage of respondents who indicated they would tell their friends that this is a safe nursing home for their family (80 percent) than for profit nursing homes (72 percent).
  • Nonprofit/government nursing homes had a higher percentage of respondents who gave their nursing home an overall rating on resident safety of "Excellent" or "Very Good" (66 percent) than for profit nursing homes (57 percent).

Highlights from Appendix B: Overall Results by Respondent Characteristics

Job Title (Tables B-1, B-3, B-4)

  • Administrators/Managers and Physicians had the highest average percent positive response across the patient safety culture composites (79 percent); Nursing Assistants/Aides had the lowest (63 percent).
  • Administrators/Managers had the highest percentage of respondents who indicated they would tell their friends that this is a safe nursing home for their family (93 percent); Nursing Assistants/Aides had the lowest (72 percent).
  • Administrators/Managers had the highest percentage of respondents who gave their nursing home an overall rating on resident safety of "Excellent" or "Very Good" (81 percent); Other Providers had the lowest (56 percent).

Work Area (Tables B-5, B-7, B-8)

  • The results for the patient safety culture composites and willingness to recommend were quite similar across work areas.
  • Respondents who indicated they worked in Many different areas in this nursing home/No specific area or unit had the highest percentage of respondents who gave their nursing home an overall rating on resident safety of "Excellent" or "Very Good" (63 percent); Alzheimer’s/Dementia Unit and Rehab Unit had the lowest (58 percent).

Interaction With Residents (Tables B-9, B-11, B-12)

  • Respondents without direct interaction with residents were more positive on 11 of the 12 patient safety composites than those with direct interaction with residents. The average response across all 12 composites was 73 percent positive for respondents without direct interaction with residents and 66 percent positive for respondents with direct interaction with residents.
  • Respondents without direct interaction with residents had a higher percentage of respondents who indicated they would tell their friends that this is a safe nursing home for their family (81 percent) than respondents with direct interaction with residents (75 percent).
  • Respondents without direct interaction with residents had a higher percentage of respondents who gave their nursing home an overall rating on resident safety of "Excellent" or "Very Good" (69 percent) than respondents with direct interaction with residents (60 percent).

Shift Worked Most Often (Tables B-13, B-15, B-16)

  • Respondents working day shifts had the highest average percent positive response on 11 of the 12 patient safety culture composites. The average response across all 12 composites was 69 percent positive for respondents working days versus 62 percent positive for respondents working nights.
  • Respondents working day shifts had the highest percentage who indicated they would tell their friends that this is a safe nursing home for their family (79 percent for respondents working days versus 74 percent for respondents working evenings and 70 percent for respondents working nights).
  • Respondents working day shifts had the highest percentage who gave their nursing home an overall rating on resident safety of "Excellent" or "Very Good" (65 percent for respondents working days versus 60 percent for respondents working evenings and 51 percent for respondents working nights).

v As described in the Notes section, an alternative method would be to report a straight percentage of positive response across all respondents, but this method would give greater weight to respondents from larger nursing homes since they account for more than twice as many responses as those from smaller nursing homes.

Page last reviewed August 2011
Internet Citation: Chapter 6. Comparing Your Results: 2011 User Comparative Database Report. August 2011. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/professionals/quality-patient-safety/patientsafetyculture/nursing-home/2011/nhsurv11ch6.html