Notes: Description of Data Cleaning and Calculations

2011 User Comparative Database Report

This notes section provides additional detail regarding how various statistics presented in this report were calculated.

Data Cleaning

Each participating nursing home was asked to submit cleaned individual-level survey data. As an additional check, once the data were submitted, response frequencies were automatically run on each nursing home's data to look for out-of-range values, missing variables, or other data anomalies. When data problems were found, data submitters were required to make corrections and resubmit their data. All submitters were shown a copy of their data frequencies to verify that the dataset received was correct.

Response Rates

As part of the data submission process, nursing homes were asked to provide their response rate numerator and denominator. Response rates were calculated using the formula below.

Response Rate = Number of complete, returned surveys
                          Number of surveys distributed − Ineligibles

Numerator = Number of complete, returned surveys. The numerator equals the number of individual survey records submitted to the database. It should exclude surveys that were returned blank on all nondemographic survey items but include surveys where at least one nondemographic survey item was answered.

Denominator = The total number of surveys distributed minus ineligibles. Ineligibles include deceased individuals and those who were not employed at the nursing home during data collection.

As a data cleaning step, we examined whether any individual survey records submitted to the database were missing responses on all of the nondemographic survey items (indicating the respondent did not answer any of the main survey questions). Records where all nondemographic survey items were missing were excluded from the nursing home's numerator. Nursing homes were included in the database only if they had a numerator of at least 10 after this data cleaning step.

Calculation of Percent Positive Scores

Most of the survey's items ask respondents to answer using 5-point response categories in terms of agreement (Strongly agree, Agree, Neither, Disagree, Strongly disagree) or frequency (Always, Most of the time, Sometimes, Rarely, Never). Three of the 12 patient safety culture composites use the frequency response option (Handoffs, Feedback and Communication About Incidents, and Communication Openness), while the other 9 composites use the agreement response option.

Item-Level Percent Positive Response

Both positively worded items (such as "Staff support one another in this nursing home") and negatively worded items (such as "Staff use shortcuts to get their work done faster") are included in the survey. Calculating the percent positive response on an item is different for positively and negatively worded items:

  • For positively worded items, percent positive response is the combined percentage of respondents within a nursing home who answered Strongly agree or Agree, or Always or Most of the time, depending on the response categories used for the item.

    For example, for the item "Staff support one another in this nursing home," if 50 percent of respondents within a nursing home Strongly agree and 25 percent Agree, the item-level percent positive response for that nursing home would be 50% + 25% = 75% positive.

  • For negatively worded items, percent positive response is the combined percentage of respondents within a nursing home who answered Strongly disagree or Disagree, or Never or Rarely, because a negative answer on a negatively worded item indicates a positive response.

    For example, for the item "Staff use shortcuts to get their work done faster," if 60 percent of respondents within a nursing home Strongly disagree and 20 percent Disagree, the item-level percent positive response would be 80 percent positive (i.e., 80 percent of respondents do not believe staff use shortcuts to get their work done faster).

Composite-Level Percent Positive Response

The survey's 42 items measure 12 areas, or composites, of patient safety culture. Each of the 12 patient safety culture composites includes 3 or 4 survey items. Composite scores were calculated for each nursing home by averaging the percent positive response on the items within a composite. For example, for a three-item composite, if the item-level percent positive responses were 50 percent, 55 percent, and 60 percent, the nursing home's composite-level percent positive response would be the average of these three percentages, or 55% positive. 

Item and Composite Percent Positive Scores

To calculate your nursing home's composite score, simply average the percentage of positive response to each item in the composite. Here is an example of computing a composite score for Nonpunitive Response to Mistakes:

  1. There are four items in this composite—two are positively worded (items A15 and A18) and two are negatively worded (items A10 and A12). Keep in mind that disagreeing with a negatively worded item indicates a positive response.
  2. Calculate the percentage of positive responses at the item level. (Refer to the example in Table 1.)

In this example, there were four items with percent positive response scores of 46 percent, 52 percent, 46 percent, and 56 percent. Averaging these item-level percent positive scores results in a composite score of .50, or 50 percent, on Nonpunitive Response to Mistakes. In this example, an average of about 50 percent of the respondents responded positively to the survey items in this composite.

Once you calculate your nursing home's percent positive response for each of the 12 patient safety culture composites, you can compare your results with the composite-level results from the 226 database nursing homes.

Percentiles 

Percentiles were computed using the SAS® Software default method. The first step in this procedure is to rank order the percent positive scores from all the participating nursing homes from lowest to highest. The next step is to multiply the number of nursing homes (n) by the percentile of interest (p), which in our case would be the 10th, 25th, 50th, 75th, or 90th percentile.

For example, to calculate the 10th percentile, one would multiply 226 (the total number of nursing homes) by .10 (10th percentile). The product of n x p is equal to j + g, where j is the integer and g is the number after the decimal. If g equals 0, the percentile is equal to the percent positive value of the nursing home in the jth position plus the percent positive value of the nursing home in the jth +1 position, divided by 2 [(X(j) + X(j+1))/2]. If g is not equal to 0, the percentile is equal to the percent positive value of the nursing home in the jth +1 position.

The following examples show how the 10th and 50th percentiles would be computed using a sample of percent positive scores from 12 nursing homes (using fake data shown in Table 2). First, the percent positive scores are sorted from low to high on Composite "A."

10th percentile

  1. For the 10th percentile, we would first multiply the number of nursing homes by .10:

    (n x p = 12 x .10 = 1.2).

  2. The product of n x p = 1.2, where j = 1 and g = 2. Since g is not equal to 0, the 10th percentile score is equal to the percent positive value of the nursing home in the jth +1 position:
    1. j equals 1.
    2. The 10th percentile equals the value for the nursing home in the 2nd position = 48%.

50th percentile

  1. For the 50th percentile, we would first multiply the number of nursing homes by .50:

    (n x p = 12 x .50 = 6.0).

  2. The product of n x p = 6.0, where j = 6 and g = 0. Since g = 0, the 50th percentile score is equal to the percent positive value of the nursing home in the jth position plus the percent positive value of the nursing home in the jth +1 position, divided by 2:
    1. j equals 6.
    2. The 50th percentile equals the average of the nursing homes in the 6th and 7th positions (64%+66%)/2 = 65%.
Current as of August 2011
Internet Citation: Notes: Description of Data Cleaning and Calculations: 2011 User Comparative Database Report. August 2011. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/professionals/quality-patient-safety/patientsafetyculture/nursing-home/2011/nhsurv11notes.html