Appendix 2-F. AHRQ Linking Policy

Introduction

Hyperlinks allow users to move from concept to concept in a nonlinear fashion. Links permit associative references to other sections within Web documents, other documents and files on a Web site, and other Web sites and Web-based resources. Internal links point to materials contained on a single Web server and resident to that Web site. External links point to materials that are resident on other Web servers or applications that are maintained by outside entities.

This AHRQ policy is in addition to the Web standard of the Department of Health and Human Services (HHS) for external link icons and disclaimers at: http://www.hhs.gov/web/policies/webstandards/disclaimer.html

Return to Section Contents

Requirements

Internal links

Internal links do not create any liability issues because the materials are on the same server. However, these links serve as one method of navigation within a site and should facilitate use of a Web site, not confuse or disorient users so they become lost on the site.

As part of creating a navigation architecture that allows the user to maintain orientation, be consistent in the use of hypertext links in lists and the level of the target for these links to subcategories of information.

Hypertext links placed within content should direct users to more detailed information but clearly indicate where the target of the link is located with a brief description of what that link contains. For this reason, it is better to have links to other materials at the end of sections rather than buried in the middle of paragraphs.

External links

External links to other Web sites constitute an "implied endorsement" and create a business advantage for the linked sites. Therefore, the Office of Management and Budget (OMB) requires Federal agencies to perform risk assessments of external links from their sites.

For AHRQ-funded resources, potential links to external sources need to be assessed against HHS and AHRQ linking policies and criteria. If a site can make a case for deviating from these policies, then the specific review and selection criteria must be justified and posted on the Web site for full disclosure.

Before establishing links to external sites, check on the linking policies of those sites. Even if the sites do not require permission to establish a link, you should notify Web sites of your intention to establish a link as a courtesy because your links will drive traffic to the sites and create demands on their servers.

Outside Web resources may link to Agency resources, providing the link is not displayed in a way that would imply the Agency's endorsement of a specific commercial product or service. Each AHRQ-funded Web site should have a page that discusses the"Linking In" policy and provides a 25-word descriptor for the site with key words that other sites can use when establishing the link.

Return to Section Contents

AHRQ Policies and Criteria

The Agency takes a conservative approach to external linking because AHRQ cannot appear to recommend sites that are incompatible with its scope and mission. Principal requirements follow.

• Links should be limited to other Federal agencies (particularly other HHS agencies), non-profit organizations that partner with AHRQ for specific projects, and other selected non-commercial Web resources, such as State and local government resources or educational institutions.
• Links to other Web-based resources should be established only if they are specifically referenced in AHRQ Web documents and directly relate to the Agency mission and outputs. Even these need to be evaluated against several quality and risk-assessment criteria.
• Linked sites should not conflict with any Federal policies or regulations.
• Links to commercial sites that market products or services are generally not appropriate, nor are links to sites that charge for information. However, this does not exclude notices of the availability of publications by public agencies or nonprofit organizations that charge for the publications or notices of conferences and meetings that charge a registration fee, providing these are directly related to the Agency mission and initiatives.

The Office of Communications and Knowledge Transfer (OCKT) reviews links for AHRQ Web-based resources maintained internally and housed on AHRQ servers for compliance with the Agency policy and selection criteria. For contractor-maintained servers, the contractor is responsible for assessing links and posting only those links that meet selection and evaluation criteria.

External links must be clearly delineated, and a brief description should be provided about the content of each linked resource. The link URL can be transparent to the user, but keep in mind that providing the specific URL for the linked resource has greater utility for the user when the Web pages are printed and subsequently referenced. Be consistent in the conventions that you use to designate external links.

An HHS standard for external link icons and disclaimers is being finalized at press time. The standard will be at: http://www.hhs.gov/web/policies/standardscategory.html#links. Draft wording for the disclaimer notice follows:

External Link Disclaimer Notice

• This graphic notice () means that you are leaving an HHS Web site.
• This external link provides additional information that is consistent with the intended purpose of a Federal site.
• The Department of Health and Human Services cannot attest to the accuracy of information provided by this link.
• Linking to a non-Federal site does not constitute an endorsement by HHS or any of its employees of the sponsors or the information and products presented on the site.
• You will be subject to the destination site's privacy policy when you follow the link.

Return to the previous page to continue.

Once links are established, they need to be re-assessed periodically to ensure they are still valid and the linked resources continue to meet selection criteria.

These principles may not cover all possibilities and circumstances. Specific cases that present new issues must be evaluated on their merits in keeping with the goals and mission of AHRQ.

Additional portal links

The General Services Administration (GSA) requires that all federally funded sites in the Government domain provide a link to the GSA portal, USA.gov, from the home page of the funded site. These sites are, in turn, indexed through the USA.gov portal.

AHRQ also has reciprocal links established with two health portals funded under HHS auspices: healthfinder® and MedlinePlus®, both of which provide information on diseases, conditions, wellness issues, and other consumer health information and decision tools.

There may be other E-Government initiatives for portals where incoming or outgoing links will need to be established depending on the purpose and content of the Web-based resource.

Return to Section Contents

Tools and Resources

AHRQ Web site external linking criteria

The AHRQ Web Site External Linking Criteria policy summarizes selection and review criteria for external links from the AHRQ Web site.

Selection criteria

Active links to sites external to AHRQ Web resources can only be made to:

• HHS and other Government agency Web sites.
• Nonprofit organization Web resources that reflect the outputs of specific projects or conferences of AHRQ with official partners (specific URL, not home page).
• Noncommercial resources that are specifically referenced in AHRQ-generated Web documents (specific URL, not home page).

Review criteria

• External links should only be established to sites or Web-based resources that are directly related to AHRQ's mission and outputs.
• External links must not conflict with official Agency, HHS, or other Federal policies or regulations.
• Links to external resources cannot imply endorsement of a specific commercial product or service (Title 44 USC).
• Linked resources must be in compliance with the Section 508 of the Rehabilitation Act of 1998 as amended and other accessibility guidelines as directed by the Department of Justice.
• Linked resources should not contain inappropriate or questionable materials that jeopardize the Agency through associated liability, potential embarrassment, or political ramifications.
• Content is accurate, scientifically sound, balanced, and current (pages show updates).
• Sources of all content are specifically identified, and references are provided for health and scientific claims.
• The sponsoring organization(s), aims, and sources of support are clearly identified.
• Biases or conflicts of interest from advocacy positions, marketing, or sources of financial support are explicitly acknowledged.
• Privacy and confidentiality matters are clearly addressed, and registration is not required.
• Contact information and feedback exist for both content and technical issues.
• The design, reading level, and navigation tools are appropriate for the intended audience and do not present barriers to users.

External Link Evaluation Checklist

The External Link Evaluation Checklist is an evaluation document that is used to assess potential external links. Copies of these assessments should be kept on file for the life of a project to show that you performed due diligence in selecting and evaluating external links and to address any challenges to the linking policy of your Web site by outside entities.

For examples of "Linking In" policy pages that can serve as models, visit the following:

• AHRQ Web site:
  http://www.ahrq.gov/news/weblink.htm
• TalkingQuality Web site:
  https://www.talkingquality.ahrq.gov/default.aspx
• healthfinder® portal:
  http://www.healthfinder.gov/aboutus/linking.asp

For examples of "Selection Criteria" policy pages on Web portals, visit the following:

• healthfinder® portal:
  http://www.healthfinder.gov/aboutus/content_guidelines.aspx
• National Women's Health Information Center:
  http://www.4woman.gov/about/select-s/
• National Guideline Clearinghouse:
  http://www.guideline.gov/about/inclusion.aspx

Return to Section Contents

External Link Evaluation Checklist

Selection categories

Is the Web resource that of a Government or nonprofit sponsoring organization in any of the categories below covered under the External Linking Policy? _____Yes _____No

Please select as appropriate:

___ HHS agency Web site or resource.
___ Other U.S. Government agency Web site or resource.
___ Nonprofit partner of AHRQ.
___ Noncommercial Web resource specifically cited.
___ State or local government agency with information useful beyond its borders.
___ University or other educational institution.
___ Public, medical, or special library.
___ National voluntary, nonprofit, or professional organization.

If yes, skip to Nominating Criteria. If no, proceed to next selection category.

Is the Web resource that of an other than nonprofit sponsoring organization in any of the categories below covered under the External Linking Policy, which warrants consideration assuming evaluation criteria are met? ____Yes ____No

Please select as appropriate:

___ Foundation with a corporate sponsor.
___ For-profit organization involved in a public-private partnership, CRADA, other grant, or contract with AHRQ.
___ Patient support or advocacy group.
___ Commercial organization offering free Web resources as a public service.

If yes, skip to Nominating Criteria. If no, proceed to next selection category.

Is the Web resource that of a sponsoring organization that falls into any of the automatic exclusion categories listed below? ____Yes ____No

Please select as appropriate:

___ Partisan political orientation.
___ Marketing or advertising site of a company, product, or service.
___ Commercial search engine site.
___ Bias, agenda, or purpose contrary to the public good.
___ Undetermined sponsorship or affiliation.

If yes, stop here. This resource cannot be considered for an external link.

Nominating criteria

Enter the name of the sponsoring organization:

Provide a brief description of the nature of the organization, its stated purpose, and sources of support:

Enter the URL (and title) of the Web site or specific Web resource being considered:

Describe the nature of the information and services offered by the organization at this URL:

Describe its relevancy to an AHRQ Web resource, project, information collection, or constituency group:

Does the site or information resource provide a contact for Web site management and electronic policies? Please list name, title, and contact information (E-mail, phone, fax, and/or mailing address).

Evaluation criteria

Is the organization and its sources of information and funding clearly identified on the Web site or information resource? _____Yes _____No _____Undecided

Does the Web site clearly distinguish between information and services offered free and products and services marketed at a commercial rate? _____Yes _____No _____Undecided

Would the presentation or content of the Web site lead a reasonable user to infer endorsement of products or services by AHRQ? _____Yes _____No _____Undecided

Can you determine any originator qualifications or quality assurance mechanisms for the site? _____Yes _____No _____Undecided

Can you determine the following based on the material presented in the Web resource?

• Authority (author/publisher/credentials)? _____Yes _____No _____Undecided
• Accuracy (verifiable source/reviewed)? _____Yes _____No _____Undecided
• Objectivity (balanced/biases)? _____Yes _____No _____Undecided
• Currency (release date/update)? _____Yes _____No _____Undecided

Is there potential for political sensitivity or embarrassment to the Agency as listed below?

• Political point of view? _____Yes _____No _____Undecided
• Political commentary or satire? _____Yes _____No _____Undecided
• Advertising or fundraising? _____Yes _____No _____Undecided
• Controversial information? _____Yes _____No _____Undecided
• Unsubstantiated claims? _____Yes _____No _____Undecided
• Profanity or sexual content? _____Yes _____No _____Undecided

Are there any potential legal complications as listed below?

• Copyright protection? _____Yes _____No _____Undecided
• Licensing agreements? _____Yes _____No _____Undecided
• Registration for access? _____Yes _____No _____Undecided
• Tracking/user profiles? _____Yes _____No _____Undecided
• Privacy concerns? _____Yes _____No _____Undecided

Is the material presented technologically accessible to the majority of users (compatible with older Web browsers and text browsers used with assistive devices to accommodate disabled users)? _____Yes _____No _____Undecided

Are additional links, LISTSERV®, or chat options directly accessible from the proposed Web resource URL being linked to the AHRQ site? If so, please briefly describe below:

Would any of these accessible links present problems if evaluated in the same context as the direct link? _____Yes _____No _____Undecided

If yes, please explain below:

General comments

Recommendation: _____Include link _____Exclude link _____Further evaluate

Reviewer: Name _____________________________ Date ______________

Return to Section Contents

References and Authorities

AHRQ Web site external linking policy

The AHRQ Web Site External Linking Policy summarizes the issues, principles, and selection criteria that apply to the AHRQ Web site. Under its current policy, AHRQ links only to:

• Other HHS agencies.
• Other Federal agencies.
• Nonprofit organizations that are official partners on specific projects or conferences.
• Noncommercial Web resources that are specifically referenced in AHRQ-generated Web documents.

Issues

The following issues affect the external linking policy:

• Under basic guidance from OMB, Federal agencies are expected to conduct a risk assessment before providing external links from their Web sites.
• Links to other sites are viewed as both an implied endorsement and a conduit to traffic, providing a business advantage for selected sites.
• Given the ephemeral nature of the Web, content can frequently change on linked resources.
• Web sites with external links must be regularly monitored for dead links or changed links.
• Resources are required to determine potential links, review and evaluate them, establish agreements between sites, and maintain and update links.

Principles

The following principles apply to the external linking policy:

• Selection criteria as established are to be followed for recommending links.
• Review criteria as established are to be followed for evaluating links.
• Disclosure language that addresses the policy, selection, and review criteria must be posted on any Web site that has external links.

OCKT reviews recommendations and makes final determinations on appropriate links.

• OCKT negotiates linking arrangements, addresses requests from external sources, and responds to challenges on linking decisions.
• OCKT provides oversight and periodic re-evaluation of linked resources (at least on a quarterly basis).

Selection criteria

A number of general guidelines for evaluating the quality of health Web sites have been published, and AHRQ has been involved in several public-private collaborations that have addressed this issue.

Existing criteria that are broadly accepted include:

• Content is accurate, scientifically sound, balanced, and current (pages show updates).
• Sources of all content are specifically identified and references are provided for health and scientific claims.
• The sponsoring organization(s), aims, and sources of support are clearly identified.
• Biases or conflicts of interest from advocacy positions, marketing, or sources of financial support are explicitly acknowledged.
• Privacy and confidentiality matters are clearly addressed.
• Contact information and feedback exist for both content and technical issues.
• The design, reading level, and navigation tools are appropriate for the intended audience and do not present barriers to users.

Additional criteria that affect AHRQ as a Federal Web site and agency of HHS:

• External links should only be established to sites or Web-based resources that are related to AHRQ's mission.
• External links must not present conflicts with official Agency, HHS, or other Federal policies or regulations.
• Links to external resources cannot imply endorsement of a specific commercial product or service (Title 44 USC).
• Linked resources must be in compliance with the Americans With Disabilities Act and other accessibility guidelines as directed by the Department of Justice.
• Linked resources should not contain inappropriate or questionable materials that jeopardize the Agency through associated liability, potential embarrassment, or political ramifications.

Return to Section Contents

Criteria for HHS External Web Site Selection

This section provides guidelines for policy development and practical application of best practices for linking HHS-sponsored Web sites to external Web sites and resources. A number of general guidelines to evaluate the quality and reliability of health information Web sites have been published. These criteria can also be used to narrow the field of candidate sites for links to avoid information overload for users and reduce link maintenance.

Agencies should build upon these guidelines with examples specific to their own sites to develop operating procedures. Those procedures should recognize that no Web site is likely to meet all criteria. However, criteria provide appropriate guidance for staff to use in assessing the communication needs and the relative risks and benefits of linkages for their specific Web sites.

1. Online health information providers or sources should meet the following core criteria to be included in or linked to from an HHS Web site.
   1. Content is accurate, scientifically sound, balanced, and current.
   2. Sources of all content are specifically identified.
   3. References are provided for health and scientific claims.
   4. Site indicates that content is updated regularly (pages show dates).
   5. Qualifications of persons or organizations providing any medical or health advice are clearly presented; trained professionals or an advisory board oversees such activities.
   6. Information provided is designed to support, not replace, the patient/provider relationship and appropriate disclaimers are present.
   7. A privacy policy statement is prominently displayed. Minimum elements of a privacy policy are discussed below.
   8. Sponsoring organization(s), aims, and sources of support are clearly identified.
   9. Biases or conflicts of interest resulting from strong advocacy positions, marketing or advertising, or sources of support are explicitly acknowledged.
   10. Marketing information and advertising are presented in a manner that would allow an average user to clearly distinguish between commercial content and other information presented.
   11. Contact information and a feedback mechanism for both content and technical issues are available.
   12. Design, reading level, search tools, site navigation, and interactive components are appropriate for the intended audience and do not present barriers to users.
2. Privacy protections for personal information. A Web site's privacy policy should:
   1. Notify users of which information is collected about them while they are on the site.
   2. Explain how the information will be used, shared and protected.
   3. Include a requirement to ask users for explicit permission to collect, track, aggregate, or share personally identifiable information.
3. Beyond core selection criteria, HHS-specific considerations include the following. HHS Web sites should link to or recommend only those information sources that:
   1. Are directly relevant to HHS programs, activities, communication goals, or target populations.
   2. Comply with Section 508 of the Rehabilitation Act, and World Wide Web Consortium and other accessibility guidelines, as directed by the Department of Justice.
   3. Provide balanced treatment of topics addressed on the Web site.
4. HHS sites should be clear on the relationship between the HHS site and any linked sites. There are currently multiple means to indicate to users that they are accessing content on a non-HHS site, including disclaimers, exit notices and pop-up boxes. Technology is constantly changing, however, and mandating a specific technical solution does not always provide the highest standard of user protection. The technical solution to clarify the relationship between HHS and non-HHS content, therefore, will be left to individual HHS agencies and Web sites.
5. In the case of online discussion forums, HHS Web sites should link to or recommend only those forums that use qualified people to moderate or regularly review the discussion for inaccurate content.
6. HHS Web sites should not link to or recommend information sources that:
   1. Conflict with official agency, HHS, or other Federal policies or regulations.
   2. Include invalid or unsupported health claims or invalid or unsupported science.
   3. Violate Federal or State laws and regulations governing the practice of medicine or the dispensing of pharmaceuticals across State lines or national borders.
   4. Imply endorsement of products or services by the Department.
   5. Allow targeted advertising by topic (for example, search on diabetes and get a specific drug ad).
   6. Imply endorsement of advocacy efforts targeting Federal laws, regulations, or policies.
   7. Do not disclose the nature of the partnerships and affiliations of contributors to the Web site and related projects.
   8. Provide secondary linkages that lead to inappropriate content.
   9. Require registration (anonymous use should be possible) or charge fees for basic information.

External guidelines used as sources for core criteria

• Health on the Net Code of Conduct:
  http://www.hon.ch/HONcode/Conduct.html
• healthfinder® Selection Guidelines:
  http://www.healthfinder.gov/aboutus/content_guidelines.aspx
• Evaluating Health-Related Web Sites, Emory University:
  http://www.sph.emory.edu/WELLNESS/instrument.html
• IHC Application Checklist, Science Panel on Interactive Communication and Health (SciPICH):
  http://www.health.gov/scipich/IHC/checklist.htm

Existing guidelines used as sources for HHS-specific considerations

• Office of Management and Budget, Policies for Federal Government Public Web Sites:
  http://www.usa.gov/webcontent/reqs_bestpractices/omb_policies/linking.shtml
• U.S. General Services Administration, USA.gov:
  http://www.usa.gov/About/Linking_Policy.shtml

Return to Section Contents
Return to Guidelines Contents
Proceed to Next Section