Executive Summary

States, community quality collaboratives, and others are investing millions of dollars in the sponsorship and development of public reports on the quality of hospital care. The hope is that these reports will stimulate quality improvement, increase accountability, and improve consumer choices. Although the major method of distributing these reports is through Web sites, it is not known who visits these public reporting Web sites or how Web site visitors use the data presented. This evidence gap leaves report sponsors with minimal guidance on how to construct and implement a report that will successfully engage consumers and providers.

Two key indicators of success for a public reporting Web site are the number of people who access it and the experiences people have when using it. However, there is little publicly available information on these topics. In order to capture these two indicators, we worked with 16 hospital reporting sites affiliated with the AHRQ Chartered Value Exchange program over a 3-month period starting in February 2011. We used two tools: Web analytics to measure and analyze Web site usage patterns for all visitors to the 16 sites, and an online pop-up survey on each Web site to gather information through direct interaction with a subset of visitors. The resulting data were augmented with expert review of the Web sites.

From the survey responses, we focused on consumers (patients and friends or family members) and health care professional respondents (physicians, nurses, hospital executives, etc.) because they are important target audiences for the public report sponsors and also accounted for most of the survey respondents.

We identified 12 key takeaways.

Getting people to come to your Web site:

1. Although more than 80,000 unique visitors came to the participating Web sites during the study, the individual sites vary dramatically in the amount of traffic they receive. In addition, some sites are attracting primarily providers and some are attracting primarily consumers.
2. There is a "most common user" profile among current consumer visitors to the 16 participating Web sites. More than 80 percent of consumer respondents were 45 years old or older, 90 percent were White, and 64 percent had at least a 4-year college degree. Strategies to address the needs of these individuals would serve the current users.
3. Conversely, the Web sites appear to be little used by important vulnerable populations. For example, although Medicaid covers over 15 percent of all citizens in the United States, less than one-half of one percent of survey respondents were Medicaid beneficiaries. Individuals with less than college education and from racial and ethnic minorities were also underrepresented. To reach these individuals, it will be important for Web site sponsors to identify methods of outreach to these populations.
4. The participating Web sites were also very different in the amount and share of traffic garnered through the three primary sources: 1) search, or typing keywords into a search engine; 2) referral, which means arriving through a link on another Web site, such as a media page or blog; and 3) direct, which includes typing the Web site address into the bar at the top of the browser or clicking on the Web site name in an E-mail or word processed document. For instance, sites with the most search traffic often lagged far behind other sites in referral traffic. No Web site excelled at generating traffic through all three sources.
5. There is opportunity for each of the participating Web sites to increase traffic via one or more of the following strategies: search engine optimization (techniques to generate more search traffic), marketing campaigns to consumers or health care professionals to increase direct traffic, and placement of links to the public report on affiliated Web sites to drive referral traffic. To date, improving search traffic has been the most effective method of generating visits.
6. Few visitors arrive at the participating Web sites through a search for a specific medical condition or a search for 'high quality hospitals' or 'hospital performance.' Many visitors arriving through search used the specific Web site names or specific hospital names in their search. In addition, there is limited competition for individual hospital names in search engines, compared to medical condition searches. Therefore, these may be good search terms on which to focus search engine optimization activities.

Engaging them once they arrive:

1. While 'high quality hospitals' or 'hospital performance' were not commonly used as search terms among visitors to the participating sites, the majority of visitors indeed came to the sites to look at quality information (choose or compare hospitals or confirm a choice already made). A substantial percentage of visitors are looking for other types of information (e.g., practical information such as location and phone number) that is often not available. A strategic response for Web site hosts could be to add content that addresses these unmet needs.
2. Some consumers are using the quality data for hospital selection. Across all 16 Web sites, almost half of consumer visitors reported that they were "likely" or "very likely" to use the data to choose a hospital or change hospitals.
3. Consumer visitors to the participating sites indicated they would like to see information more specific to their decisionmaking needs—specific conditions or surgeries (such as cancer or joint replacement surgery) and performance of individual doctors practicing at the hospital.
4. Health care professionals rated the participating sites as easier to use—and had higher overall satisfaction ratings—than consumers, but even some health care professionals found them difficult to use. Across the participating Web sites, there are opportunities to make it easier for each category of user to find the best hospital.
5. In addition, and in contrast to consumers, health care professionals often expressed interest in having more details about quality measures or measurement methodology. This is the first empirical, survey evidence that it is difficult to serve both audiences with a single Web site.
6. An expert review of the participating sites identified areas for consideration for redesign that would make it easier for users to find the best hospital. For example, few of the participating sites use rating or measure display strategies that have been shown to help users understand comparative information and use it to make a choice; few sites use composite measures or interpretive labels (e.g., "better," "average," or "worse") to display provider performance. Only one uses a conceptual framework to help visitors understand the larger concept of quality. Nearly all sites used technical language (for example, including abbreviations such as ARB for angiotensin receptor blockers, but even the latter is not understood by most consumers). Many of the participating Web sites do not allow visitors to choose the hospitals of interest to them, many have visual displays of information that are difficult to understand, and some do not allow side-by-side comparisons of hospitals.

It is encouraging that some consumers and health care professionals are finding these Web sites and that some visitors are using the reports for hospital selection. Exciting opportunities remain for report sponsors: reaching consumer and health care professional markets more broadly as well as other markets that remain untapped and improving the power of their reports by focusing on relevant content and making it easier for individuals to find the hospitals that will provide them with the best care.