AHRQ's Public Report Surveys

What Types of Surveys Are Available?

1. The AHRQ Hospital-Public Report (H-PR) Survey is tailored for Web-based public reports of hospital performance (  Microsoft Word version - 43.98 KB ).

What Types of Information Do the Surveys Gather?

The surveys are designed to first assess the category of respondent: consumer (including patient or friend or family member of patient), health care professional, employer, insurer, member of the media, researcher, patient advocate, foundation staff, lawyer, or government staff/elected official. Next, the survey branches to questions and answer options that correspond with the specific category of respondent.

Each survey includes questions that cover the following types of information:

• Purpose of the respondent's visit.
• Topics or types of information of interest to the respondent.
• Likelihood the respondent will use the information to choose a provider or change providers.
• Respondent's suggestions for improving the report.
• Demographics of the respondent.

When Do Respondents Complete the Survey? How Long Does it Take To Complete the Survey?

The assumption is that Web site visitors will complete the survey after visiting and interacting with the Web site (go to "What are the survey invitation presentation options?" for different methods of inviting site visitors to complete the survey). Each survey takes about 2 to 4 minutes to complete.

How Do the Surveys Differ?

While the two surveys are parallel in many ways, as their names imply, the H-PR survey is designed to assess respondents' perceptions specific to public reports of hospital performance, and the P-PR survey assesses respondents' perceptions of public reports of physician performance. One important design difference is that the P-PR survey allows respondents to choose questions corresponding with (1) an individual physician, (2) a clinic, or (3) a medical group. Given this feature of the P-PR survey, and as noted below, the survey software choices for the P-PR survey might be more restricted than software choices for the H-PR survey.

What Are the Survey Invitation Presentation Options?

You may present the survey to visitors in a number of ways:

• An embedded link on a page with other content.
• An embedded link on a specific "Feedback" page.
• A feedback "button," such as, a large button prominently displayed on the home page that includes text such as "Take our Survey" or "We want your feedback!"
• A modal dialog box, or "pop up" invitation that interrupts the visitor experience, triggered by the user's arrival on the home page or another specific page. The invitation language included in the surveys is written for use with a modal dialog box.

Technical support requirements for the survey invitation are greater for the modal dialog box than the other three options. Many survey software packages provide some code to insert a link to a survey invitation, but fewer provide programming for displaying a pop-up invitation. But the expected response rate is probably highest with the pop-up invitation, because it briefly interrupts visitors to invite them to give feedback, rather than relying on them to notice and access a static link or feedback "button" on a page.

Where Might the Survey Invitation or Link Be Placed on the Web site, and Which Is the Best Option?

The surveys can launch from any page of your Web site that you choose. The survey invitation and surveys were designed and written for the survey to be taken at the end of the Web site visit. Sometimes this is referred to as an "exit survey." Since this timing is explained in the survey invitation as well as the first page of the survey, the invitation or embedded link may appear on any page of the report.

In order for the visitor to be invited from any page of the report and still be presented with the survey only at the end of the visit, the pop-up invitation can be programmed as follows: When the visitor agrees to take the survey, the survey opens up in another window on the computer, but behind the Web site of interest, for the respondent to complete at the end of the visit. In this way, the user's visit is minimally interrupted and the survey may be taken after the visitor has spent time interacting with the Web site. When the visitor declines to take the survey, the invitation window closes and the visitor can return to the Web site visit.

The decision of which page or pages to launch from (whether appearing as a pop-up invitation or simply appearing as an embedded link) depends on the kind of information you are hoping to collect. For example, if you would like to know the breadth of consumers who are coming to your Web site, you might consider putting the invitation on the home page. Alternatively, if you'd like to collect information more specifically from the segment of users looking for diabetes information, you might consider putting the survey invitation on the pages with quality information on diabetes.

If you would like to program the pop-up invitation to appear on more than one page of the Web site, it may be programmed to appear only once for any user, even if the user visits more than one page where the invitation is programmed to launch.

What Is the Timeframe for Use of the Surveys?

The surveys can be used on an ongoing basis to collect responses. The surveys also can be used intermittently, to capture responses to changes in Web site design or to new data releases. If used intermittently, the duration will be dictated in part by the length of time it takes to collect responses from the type of respondents (e.g., consumers) you are targeting. Most survey experts would recommend continuing to collect data until you have at least 30 responses for a particular type of respondent.

If I Already Use Web Analytics, Does Adding a Survey Make Sense?

Yes. Web analytics and surveys measure different things, but they work together well to form a more complete basis of information for site planners.

AHRQ's Public Report Surveys are one approach to evaluating your Web site. Another approach, Web analytics (also known by two subtypes, logfile analysis and page tagging), involve the collection of data about visitors' "searching and browsing behavior" related to your Web site. Examples of information collected might include the number of visitors to your Web site, how they get there (the search engines and search terms they used, if any, or what links they clicked to get to your Web site), how long they spend there, and how many pages they view.

Web analytics can be thought of as complementary to the surveys, rather than as substitutes. You might use Web analytics primarily to learn about the number of people who come to your site via which pathway (for example, through online search engines such as Google or Yahoo, or through online newspaper articles with embedded links to your site). These data may inform strategies to attract more visitors to your site.

In contrast, the surveys are designed to get more detail about who is coming to your Web site (e.g., patients, health care professionals, employers) and why. Surveys assess user experiences once they get to the site. (Did they accomplish their purpose? How do they think the Web site could be improved? Will they come back?). Surveys also address topics for which no data are available from Web analytics.

What Is Needed To Implement the Surveys?

To use these surveys, you will need survey software. Several types are publicly available and can be found online, with varying price structures. Some software requires very little additional technical support and some requires more. You also will need a data server to store the data. The data server often comes with the survey software, in which case the software vendor also runs the data server that houses your survey responses. However, if you are using a freestanding survey software package, you will need to arrange for the data server as well. Since this solution probably would be difficult for a small organization to implement, for smaller groups we recommend finding a software company that stores the data on your behalf.

The software features that the survey requires depend on which survey (the H-PR or P-PR) you are using. Both surveys are designed to be used with survey software that will display the questions and route responses back to the server where the data (responses) are kept. Each of the two surveys requires survey software that can do branching and skip patterns. The P-PR survey has an additional requirement: survey software that can carry forward the respondent's selection (individual physician, clinic, or medical group) into the text of another question (sometimes called "piping in").

One optional software feature for both surveys is a randomization feature that allows for random display of subgroups of questions. The purpose of randomly displaying questions is to decrease the overall burden to any one respondent so that he or she does not have to answer all the questions corresponding with a respondent category (e.g., "Patient"). Since the longest lists of questions correspond with the "Patient" and "Friend or Family Member of a Patient" respondent categories, questions targeted to these two groups would be well suited for randomization. If you would like to randomize questions, you will need survey software with this capability and the implementation may require more technical support.

What Was the Survey Development Process?

The surveys were developed by Naomi S. Bardach and R. Adams Dudley from the University of California, San Francisco, and Judith Hibbard from the University of Oregon. After assembling and analyzing a sample of existing online surveys from public reporting Web sites, the survey team drafted two surveys and vetted them with a group of 12 report developers (a subset of community quality collaboratives affiliated with AHRQ called Chartered Value Exchanges). Subsequently, we conducted cognitive interviews with 11 consumers, providers, and health care executives to assess and improve interpretability of questions and response options.

Note: We expect these surveys to be updated periodically, so please check this Web page for the latest versions of the surveys. Survey versions (e.g., 1.0) will be prominently displayed at the top of each survey.