2014 User Comparative Database Report
Notes: Description of Data Cleaning and Calculations
This notes section provides additional detail regarding how various statistics presented in this report were calculated.
Each participating hospital submitted individual-level survey data. Once the data were submitted, response frequencies were run on each hospital's data to look for out-of-range values, missing variables, or other data anomalies. When data problems were found, hospitals were contacted and asked to make corrections and resubmit their data. In addition, each participating hospital was sent a copy of its data frequencies to verify that the dataset received was correct.
Records of respondents who supplied the same answers for sections A, B, C, D, and F or who answered only demographic items (i.e., straight-lined) were deleted before any analyses.
As part of the data submission process, hospitals were asked to provide their response rate numerator and denominator. Response rates were calculated using the formula below.
Response Rate = Number of complete, returned surveys
Number of surveys distributed − Ineligibles
Numerator = Number of complete, returned surveys. The numerator equals the number of individual survey records submitted to the database. It should exclude surveys that were returned blank on all nondemographic survey items but include surveys where at least one nondemographic survey item was answered.
Denominator = The total number of surveys distributed minus ineligibles. Ineligibles include deceased individuals and those who were not employed at the hospital during data collection.
Calculation of Percent Positive Scores
Most of the survey's items ask respondents to answer using 5-point response categories in terms of agreement (Strongly agree, Agree, Neither, Disagree, Strongly disagree) or frequency (Always, Most of the time, Sometimes, Rarely, Never). Three of the 12 patient safety culture composites use the frequency response option (Feedback and Communication About Error, Communication Openness, and Frequency of Events Reported), while the other 9 composites use the agreement response option.
Item-Level Percent Positive Response
Both positively worded items (such as "People support one another in this unit") and negatively worded items (such as "We have patient safety problems in this unit") are included in the survey. Calculating the percent positive response on an item is different for positively and negatively worded items:
- For positively worded items, percent positive response is the combined percentage of respondents within a hospital who answered "Strongly agree" or "Agree," or "Always" or "Most of the time," depending on the response categories used for the item.
For example, for the item "People support one another in this unit," if 50 percent of respondents within a hospital Strongly agree and 25 percent Agree, the item-level percent positive response for that hospital would be 50% + 25% = 75% positive.
- For negatively worded items, percent positive response is the combined percentage of respondents within a hospital who answered "Strongly disagree" or "Disagree," or "Never" or "Rarely," because a negative answer on a negatively worded item indicates a positive response.
For example, for the item "We have patient safety problems in this unit," if 60 percent of respondents within a hospital Strongly disagree and 20 percent Disagree, the item-level percent positive response would be 80 percent positive (i.e., 80 percent of respondents do not believe they have patient safety problems in their work area).
Composite-Level Percent Positive Response
The survey's 42 items measure 12 areas, or composites, of patient safety culture. Each of the 12 patient safety culture composites includes 3 or 4 survey items. Composite scores were calculated for each hospital by averaging the percent positive response on the items within a composite. For example, for a three-item composite, if the item-level percent positive responses were 50 percent, 55 percent, and 60 percent, the hospital's composite-level percent positive response would be the average of these three percentages, or 55 percent positive.xi
- There are four items in this composite—two are positively worded (items A15 and A18) and two are negatively worded (items A10 and A17). Keep in mind that disagreeing with a negatively worded item indicates a positive response.
- Calculate the percentage of positive responses at the item level. (Go to example in Table 1.)
In this example, there were four items with percent positive response scores of 46 percent, 52 percent, 46 percent, and 56 percent. Averaging these item-level percent positive scores results in a composite score of .50, or 50 percent, on Overall Perceptions of Patient Safety. In this example, an average of about 50 percent of the respondents responded positively to the survey items in this composite.
Table 2 shows how to calculate the percent positive response for Overall Patient Safety Grade (item E1) and Number of Events Reported (item G1).
In this example, the Overall Patient Safety Grade (item E1) percent positive response is calculated by combining the percentage of respondents who answered "Excellent" and "Very Good." The Number of Events Reported (item G1) percent positive response is calculated by combining the percentage of respondents who answered that they reported one or more events in the past 12 months.
Once you calculate your hospital's percent positive response for each of the 12 safety culture composites, Overall Patient Safety Grade, and Number of Events Reported, you can compare your results with the composite-level results from the database hospitals.
Minimum Number of Responses
Beginning with the 2010 database report, we enacted several new rules regarding a minimum number of responses for calculating the percent positive scores. First, we calculated percent positive scores only for hospitals that had at least 10 completed surveys. Second, item-level results were calculated only when there were at least three responses to the item. If a hospital had fewer than three responses to a survey item, the hospital's score for that item was set to missing. Starting with the 2014 Comparative Database, if a hospital had at least five respondents in a breakout category (e.g., work area/unit, staff position, direct interaction with patients), statistics were calculated for that breakout category.
Percentiles were computed using the SAS® software default method. The first step in this procedure is to rank order the percent positive scores from all the participating hospitals, from lowest to highest. The next step is to multiply the number of hospitals (n) by the percentile of interest (p), which in our case would be the 10th, 25th, 50th, 75th, or 90th percentile.
For example, to calculate the 10th percentile, one would multiply 653 (the total number of hospitals) by .10 (10th percentile). The product of n x p is equal to j + g, where j is the integer and g is the number after the decimal. If g equals 0, the percentile is equal to the percent positive value of the hospital in the jth position plus the percent positive value of the hospital in the jth + 1 position, divided by 2 [(X(j) + X(j+1))/2]. If g is not equal to 0, the percentile is equal to the percent positive value of the hospital in the jth + 1 position.
The following examples show how the 10th and 50th percentiles would be computed using a sample of percent positive scores from 12 hospitals (using fake data shown in Table 3). First, the percent positive scores are sorted from low to high on Composite "A."
For the 10th percentile, we would first multiply the number of hospitals by .10:
(n x p = 12 x .10 = 1.2).
- The product of n x p = 1.2, where j = 1 and g = 2. Since g is not equal to 0, the 10th percentile score is equal to the percent positive value of the hospital in the jth +1 position:
- j equals 1.
- The 10th percentile equals the value for the hospital in the 2nd position = 48%.
For the 50th percentile, we would first multiply the number of hospitals by .50:
(n x p = 12 x .50 = 6.0).
- The product of n x p = 6.0, where j = 6 and g = 0. Since g = 0, the 50th percentile score is equal to the percent positive value of the hospital in the jth position plus the percent positive value of the hospital in the jth +1 position, divided by 2:
- j equals 6.
- The 50th percentile equals the average of the hospitals in the 6th and 7th positions (64% + 66%)/2 = 65%.
xi. This method for calculating composite scores differs slightly from the method described in the September 2004 Survey User's Guide that is part of the original survey toolkit materials on the AHRQ Web site. The guide advises computing composites by calculating the overall percent positive across all the items within a composite. The updated recommendation included in this report is to compute item percent positive scores first, and then average the item percent positive scores to obtain the composite score, which gives equal weight to each item in a composite.