Establishing Data Collection Procedures

Once you have decided how you want the surveys distributed and returned, and have established at least one main hospital point-of-contact, you need to make several decisions regarding your data collection procedures. This section describes strategies for maximizing your response rate and outlines methods for tracking responses and collecting data.

Maximize Your Response Rate

The response rate is the total number of complete returned surveys divided by the total number of eligible staff sampled. Achieving a high response rate is very important for making valid generalizations about your hospital, based on your survey data collection effort. Surveys are used to infer something about a particular population. There must be enough survey respondents to accurately represent the hospital or larger population, before you can legitimately present your survey results as a reflection of your hospital's safety culture.

If your response rate is low, there is a danger that the large number of staff who did not respond to the survey would have answered very differently from those who did respond. Therefore, an overall response rate of 50 percent or more should be your minimal goal. The higher the response rate, the more confident you can be that you have an adequate representation of the staff's views. To achieve high response rates, we recommend a basic data collection approach that involves sending a paper survey and the following items, in the order presented:

1. Prenotification letter

   Before administering the survey, create a letter signed by your hospital's CEO or president on hospital letterhead. The letter will inform all the staff in your sample that they will be receiving a survey and that hospital administration is in full support of the survey effort. If an outside vendor is handling the data collection duties, use the letter as an opportunity to introduce the vendor.

2. First survey

   About 1 week later, send the survey to all staff in your sample group. Include a supporting cover letter similar in content to the prenotification letter and instructions for completing and returning the survey. Include preaddressed postage-paid envelopes to make it easy for respondents to return their surveys.

   In the cover letter, or on the survey form, ask staff to complete the survey within 7 days, but do not print an actual deadline date on the letter or survey. Sometimes data collection schedules get delayed, and you do not want to reprint letters or surveys because they are outdated. In addition, sometimes people will not complete a survey if they notice that it is beyond the deadline date.

3. First reminder postcard or letter

   Approximately 2 weeks after sending the survey, send a reminder postcard or letter to the sample group thanking those who have already responded and reminding others to please respond. The reminders can be sent to everyone, or only to those who have not responded.

4. Second survey

   Two weeks after sending the first reminder, send a second survey to nonrespondents, including a cover letter thanking those who have already responded and reminding others to please complete the second survey. If you are not using identifiers to track responses, it may be necessary to send a second survey to everyone in your sample.

5. Second reminder postcard or letter (optional)

   Approximately 1 week after sending the followup survey, you may choose to send a second and final reminder.

Additional Ways To Maximize Response Rates Publicize the Survey. Announce the survey in hospital newsletters, on message boards, via flyers posted throughout the hospital, and through staff E-mail. Publicizing the survey both prior to and during survey mailout will help to legitimize the effort and increase your response rate. Use Incentives. Offering incentives can be a good way to increase responses to a survey because respondents often ask, "What's in it for me?" You may want to offer individual incentives, such as a raffle for cash prizes or gift certificates, or you can offer group incentives, such as catered lunches for units with at least a 75-percent response rate. Be creative and think about what would motivate your staff to complete the survey.

Return to Contents

Track Responses With or Without Identifiers

To ensure confidentiality, respondents are asked not to provide their names on the completed survey forms. It is sometimes helpful, however, to include a number or code known as an identifier, on your surveys. Identifiers typically are used to track whether individuals have responded to the survey and/or to track the particular unit or hospital associated with a completed survey. The advantage of using identifiers is that they allow you to track responses so you:

• Send reminders and followup materials only to nonrespondents, saving on costs.
• Eliminate the possibility of someone completing more than one survey.
• Calculate response rates at the unit or hospital level (hospital-level response rates are important when administering the survey in several hospitals at the same time).

On the other hand, there are a number of disadvantages to using identifiers. Some respondents will be so concerned about the confidentiality of their responses that they will de-identify their own surveys by removing or marking out their identification number or code. Respondents also may refuse to complete the survey if they are concerned that their response will be tracked, especially if the data will be collected and analyzed within the hospital (rather than by an outside vendor). Furthermore, the inclusion of any type of identifier on surveys mandates a very strict adherence to procedures protecting the confidentiality of the information linking individual staff to the identification numbers or codes.

Guidelines for Using Identifiers Following careful procedures for using identifiers is critical to maintaining trust that survey responses are confidential and answers will not be linked back to individual staff. If you decide to use identifiers, you must ensure that only key project personnel have access to information linking individual names or groups to the identification numbers or codes. Do not use group identifiers (e.g., for a particular unit or staffing category) if there are fewer than 10 staff in a group because individual responses are more identifiable in a small group. Do not use obvious identifiers (e.g., do not use "East3"). At the conclusion of data collection, information linking names to identifiers should be destroyed.

Reply Postcards with Identifiers

An alternative to using identifiers printed on surveys is to include in the survey materials a postage-paid reply postcard that has an identifying number or code (with no identifiers on the actual surveys). In the sample reply postcard, the number "155" is one respondent's individual identification number. When respondents return their completed surveys, they are instructed to return the reply postcard separately, which notifies you that the staff member with the particular individual identification number has returned the survey and therefore does not need to be sent reminder materials. Using a separate postcard ensures the anonymity of survey responses because there is no way to link any completed survey answers to a particular individual. The main obstacle to this approach is that it is not an exact means of tracking responses, because there may be people who send in their surveys but not their postcard, and vice versa.

Sample Reply Postcard with an Identifier:

When you complete and return your survey, please return this postcard separately to let us know you have responded. Thank you very much for your time and participation. I am mailing this postcard to let you know that I have returned my survey in a separate envelope. 155

No Identifiers

If you decide it is best not to use any identifiers, reminder letters and followup surveys must be sent to all staff with instructions to disregard the second survey if the first survey was completed and mailed. You may receive phone calls from respondents who completed and returned their survey, wondering why they received followup materials, but you can instruct them to disregard the materials and remove their names from further followup mailings.

Return to Contents

Assemble Survey Materials

The following materials will need to be assembled in preparation for the survey mailing. To improve response rates, it is advantageous to personalize outer envelopes and letters (e.g., addressed to "Dear John Doe"). Care should be taken, however, to prevent names from appearing on the actual survey forms.

• Publicity materials (optional) Depending on how extensively you survey your staff, you may want to post informational flyers or send E-mail notices publicizing the survey.
• Unit-level point-of-contact letter (optional). You may want to send a letter to unit-level contact persons describing the purposes of the survey and explaining their role in the survey effort. The letter should be printed on official hospital letterhead, dated with month/year, signed by the hospital CEO, and should provide background information and instructions regarding the survey.
• Prenotification letter. The prenotification letter should describe the purposes of the survey and it should contain the completion instructions. This letter also should be on official hospital letterhead, signed by the hospital CEO or president.
• Cover letter. The cover letter should be on official hospital letterhead and is to be included with the first packet of survey materials. Include the following points:
  • Why the hospital is conducting the survey and how staff responses will be used.
  • Which hospital staff were selected to be surveyed (e.g., all staff, nursing staff, all clinical staff, a random sample of staff, etc.).
  • How much time is needed to complete the survey.
  • Confidentiality or anonymity assurances.
  • Suggested reply timeframe and how to return completed surveys.
  • Incentives for which staff will be eligible, if they respond (Optional).
  • Contact information for the main hospital point-of-contact.

  These points can be summarized in a few short paragraphs. For example:

  "The enclosed survey is part of our hospital's efforts to better address patient safety. The survey is being distributed to (sample description). It will take about 10 to 15 minutes to complete and your individual responses will be kept confidential. Only group statistics will be prepared from the survey results.

  Please complete your survey and return it within the next 7 days. (Do not provide a specific date) When you have completed your survey, please (provide return instructions). (Optional incentive) As a way of thanking staff members for their participation, respondents will receive (describe incentive).

  Please contact [contact name and job position] if you have any questions [provide phone number and E-mail address]. Thank you in advance for your participation in this important effort."

• Reminder postcards or letters. A reminder postcard or letter is sent to nonrespondents after the first survey administration, asking them to please complete and return their survey.
• Surveys. If you are not tracking responses and plan to send second surveys to everyone in your sample, print at least twice the number of surveys as staff in your sample. If you are tracking responses and will send only second surveys to nonrespondents, you may print fewer surveys. For example, if your hospital's survey response history typically results in a 20 percent response to the first survey, you could print 80 percent more surveys than were distributed initially, to prepare for the followup survey mailing—800 staff multiplied by .80 equals 640, for a total estimate of 1,440 printed surveys needed.
• Labels. You will need labels for the outside of each survey mailing envelope, addressed either to the home address or internal hospital mailing address of each staff member in your sample. Return address labels may be used on return envelopes. Labels also may be used to place identifiers onto surveys.
• Envelopes. You will need a set of outer envelopes to send the surveys and a set of return envelopes for the return of completed surveys. Preprint the return address on the return envelopes (or use labels). To make sure that the cover letter, survey, and return envelope fit without folding or bending, use slightly larger outer envelopes. Calculate the number of envelopes based on the number of initial and followup surveys to be sent.
• Postage. If surveys are to be sent through the mail, weigh the outgoing packet of survey materials to ensure you have adequate postage. If surveys are to be returned through the mail, weigh the survey and the return envelope to ensure you have adequate postage on the return envelopes. Calculate the total amount of postage based on the number of initial and followup surveys to be mailed.

Return to Contents

Track Responses and Response Rates

You, or your vendor, will need to follow survey response rates by tracking completed surveys as they are returned. Tracking returned surveys can be done very simply with a spreadsheet software program. If you are planning to use survey identifiers, create a separate row for each individual identifier. Create columns across the top of your spreadsheet for the date the initial survey is distributed, the date the returned survey is received (so respondents can be excluded from followup reminders), as well as the distribution dates for any first reminders, second surveys, or second reminders. Compile response rates for each round of followup contacts—at the time of the first reminder, the second survey, and the final reminder—to track your response progress.

Closing Out Data Collection

To ensure you receive as many responses as possible, plan to hold open data collection for at least 2 weeks after the second survey or second followup reminder is sent. Referring to the project timeline, allow 8 weeks or more from the prenotification letter to the your data collection period closeout. There always will be a few respondents who return the survey very late, so you may want to take this into consideration and hold the data collection period open longer. Once the established cutoff date arrives, close out data collection and begin preparing the data for analysis as described in the next section.

Calculating Your Response Rate

To calculate your survey response rate, divide the number of completed and returned surveys (numerator) by the number of surveys sent (denominator). This equation often needs adjusting, however. The number of surveys "returned" depends on the criteria you use to define a "completed" survey. The number of surveys "sent" depends on how many staff actually receive their survey. If a survey is returned due to a bad address or because a selected staff member no longer works at the hospital, the case is ineligible for inclusion and would be subtracted from the denominator. We recommend using the following formula for an adjusted response rate:

Number of complete, returned surveys

---

Number of surveys distributed - [minus] (ineligibles + [plus] incomplete surveys)

Return to Contents
Proceed to Next Section