IV. Findings (continued)

Evaluation of the Use of AHRQ and Other Quality Indicators (PDF)

1.2 Process for Adopting the USPSTF Recommendations

Each health plan has its own unique process for adopting clinical preventive services recommendations from the USPSTF and other sources. To learn more about the process and practice of adopting the USPSTF recommendations, and clinical preventive services recommendations, more generally, we asked respondents two key questions:

  • "Can you describe the process that [your health plan] uses to review clinical preventive services recommendations, from the Task Force and from other sources, when deciding whether or not those recommendations should be adopted and integrated into your system?"
  • "If you don't implement all of the recommendations, what are the reasons or criteria [your health plan] uses to select which recommendations will be integrated?"

The sections below synthesize the responses we received and elaborate on several key findings. Exhibit 8 depicts these findings, highlighting distinctions across plans.

Process and Practice of Adopting the USPSTF Recommendations. As depicted below, the closed-panel plan and governmental plan comply with recommendations for clinical preventive services issued by their respective systems-level or national headquarters. However, the degree of compliance with these nationally-determined recommendations – and relative autonomy to deviate – differs dramatically. In the open-panel and hybrid plans, the process of adopting the USPSTF recommendations occurs at the plan level only; various committees provide a place for organizational dialogue on prevention issues and to develop their own plan guidelines. The importance of provider involvement in adopting clinical preventive services recommendations is also a key finding across all four plans.

 

Exhibit 8: Process for Adopting CPS Recommendations

Key ElementsOpen-PanelClosed-PanelHybridGovernment
CPS recs are determined at the national or systems-level x  
Plan reviews and adopts CPS recs locallyxxx 
Expert opinion plays a rolexxx*
Consensus-based decision-making plays a rolexxx*
Plan consults the USPSTF recs during the processxxxx
Plan consults other organizations that issue CPS recommendationsxxxx

*This asterisk depicts that there is not enough information to make a classification.

The closed-panel plan has a great degree of local autonomy with respect to adopting recommendations for clinical preventive services. A national set of guidelines for clinical preventive services is created at the plan's systems-level headquarters and distributed to affiliated plans in regions throughout the country. The plan has a committee at the regional level that reviews the recommendations from the national group and approves them for adoption. Analysts examine evidence from the Task Force and other sources, essentially conducting a second screen of the supporting evidence for each clinical preventive service recommendation. The information is presented to the committee, approximately every two years (for every plan guideline). Physicians, nurses, and health plan staff participate in a meeting to review, and modify (if necessary) the national guidelines. Historically, recommendations were reviewed and adopted at the regional level only. The Director of Quality Improvement explained that the process changed over the past few years in order to set national standards for clinical preventive services. The new process enables the plan to retrain local control over its own guidelines. For example, while the closed-panel plan's national group recommended that eligible men and women be screened for colorectal cancer every 5-10 years, the regional plan modified the recommendation to mandate screening every 5 years.

In contrast to the closed-panel plan, the governmental plan has the least degree of local autonomy with respect to adopting the recommendations for clinical preventive services. The government plan must integrate the clinical preventive services guidelines adopted by a national group at the plan's headquarters office. Respondents from the government plan indicated that the local facilities have very little control over which recommendations are selected for integration and implementation. One Clinical Advisor from the plan indicated that it would be desirable to have a guideline review process at the local level. Another Clinical Advisor described that leadership from the local level of the governmental plan participates in the national level process of reviewing and adopting recommendations. According to a high-ranking director at the national headquarters office, the governmental system reviews recommendations from professional groups and specialty societies to develop system-wide guidelines.

The hybrid plan, while also part of a larger health system, reviews and adopts clinical preventive services recommendations at the plan level. The process incorporates the input of physicians that are plan-affiliated employees as well as physicians that are contractors. A quality committee and several subgroups at the plan level meet regularly to review and adopt recommendations for clinical preventive services. The hybrid plan's Clinical Advisors discussed that the plan looks for the best evidence, selecting recommendations that are ranked or rated in a manner similar to the Task Force. The plan consults a range of recommendations from various specialty societies, sometimes using the Task Force's recommendations and other times adopting recommendations from other organizations such as the American Cancer Society or the American Radiological Society.

Finally, the open-panel plan's process for reviewing and adopting clinical preventive services recommendations occurs at the plan level only. The Director of Quality Improvement indicated that the plan's medical policy department researches recommendations for approximately 450-500 "medical policies," which include those for clinical preventive services. An evidence-based approach is employed, whereby recommendations must meet five basic criteria: (1) the service must have final approval from the appropriate governmental regulatory body; (2) scientific evidence must permit conclusions concerning the effect of the service on health outcomes; (3) the service must improve net health outcomes; (4) the service must be as beneficial as any established alternative; and (5) improvement must be attainable outside the investigational setting. The medical policies are reviewed at a regional advisory board meeting by panels of contracted clinical staff. Then, the medical policies are reviewed and adopted by the medical policy review committee which includes staff from all major departments of the health plan (e.g., reimbursement, information systems, networking). The plan's medical policies are reevaluated every three years, or earlier if scientific evidence merits a change.

The Role of Expert Opinion and Consensus-Based Decision-Making. We asked respondents to elaborate on the reasons or criteria used to select which recommendations will be integrated. While all of the plans used a highly evidence-based methodology where various committee structures reviewed clinical preventive services recommendations from the Task Force and other sources, we found that expert opinion and group consensus decision-making are two other influences that impact the selection of recommendations.

The closed-panel plan is striving to employ a more evidence-based approach to its review and adoption process for clinical preventive services. However, according to respondents, the process is still heavily influenced by practice patterns and expert opinion. For example, in the closed-panel plan, "Clinical Thought Leaders"57 – primary care physicians and clinicians who work for the health plan and have a particular area of clinical expertise – have significant influence over which CPS recommendations are adopted for delivery at the plan. For example, if organizations are not in agreement on a recommendation for a particular preventive service, the Clinical Thought Leaders are primarily responsible for making the decision about adoption and integration. Respondents from the plan indicated that these physicians and clinicians play a large role in the adoption of the recommendations.

In the open-panel plan, while the process is primarily evidence-based, the plan has incorporated experts into its process for policies that are early in development. A Director of Quality Improvement indicated that the medical policy and research departments have involved external expert consultants or academic physicians to review new policy issues. In the past, the open-panel plan has also used a consensus-based process to decide whether or not to adopt a clinical preventive service recommendation as a medical policy. A Director of Quality Improvement indicated that while the plan has tried a consensus process in the past, the process was "fairly expensive" and has only been done "on occasion."

The hybrid plan's Clinical Advisors indicated that the plan rarely incorporates expert opinion into its process for selecting clinical preventive services recommendations. The respondents described that the plan tries to only incorporate recommendations that have "level one" evidence, describing the quality of these recommendations as similar to the Task Force's "A" and "B" recommendations. However, the Clinical Advisors described that the plan will implement recommendations based on expert opinion if "it's appropriate for patients in their situations."

Respondents from the governmental plan indicated that they have very little control over the adoption of recommendations for clinical preventive services, as this process is purely orchestrated at their system headquarters. The Director of Quality Improvement for the governmental plan indicated that at the local level "we don't have a lot of control over [the process used to review and adopt recommendations]. This is a national determination, and then [the recommendations] roll down to the facility level. I'm not sure of what process they would use to adopt or reject or table certain types of recommendations." As a result, it is unclear how large of a role expert opinion and consensus-based decision-making play in the process used by the governmental plan.

The Role of the USPSTF Recommendations. The USPSTF recommendations play an important role in the process that health plans use to develop and adopt their own health plan recommendations/guidelines for clinical preventive services. For some plans, the Task Force recommendations are a primary source of information. A Director of Quality Improvement respondent for the open-panel plan indicated that the USPSTF recommendations are "the biggest source" of recommendations used for clinical preventive services.

A Director of Quality Improvement at the hybrid plan discussed that the Task Force recommendations are important to the plan because they are "purely evidence-based" and "unbiased," in comparison to other sources. A respondent from the closed-panel plan indicated that the process for adopting CPS recommendations from the Task Force is "easier" than from other specialty societies because the closed-panel plan uses a similar grading methodology to integrate its recommendations. The Director of Quality Improvement at the closed-panel plan also explained that if the national group does not have a guideline for a particular clinical preventive service, then the USPSTF recommendation is considered for adoption.

One Task Force recommendation in particular that has had a large impact on practice patterns at the closed-panel and hybrid plans: the "I" recommendation for protein specific antigen (PSA) testing for prostate cancer. The closed-panel, governmental, and hybrid plans stopped routinely delivering the PSA tests for prostate cancer as a result of the USPSTF "I" recommendation.58 The Task Force concluded that the evidence was insufficient to recommend for or against routinely providing the PSA test to screen for prostate cancer. A Director of Quality Improvement from the hybrid plan indicated that "the PSA test is one that we've actually sided with the USPSTF as opposed to the American Urological Association and the American Cancer Society when we put out recommendations to members. We tell members that you really need to talk to your doctor about Clinical Advisor respondent from the closed-panel plan also indicated that the plan stopped its protein specific antigen (PSA) testing on a regular basis for prostate cancer. However, this respondent did not indicate that this decision was due to the USPSTF's "I" recommendation.

Respondents from the closed-panel plan also suggested that the plan has decreased its frequency of cervical cytology because of the USPSTF recommendation.d The plan now screens patients every three years (after two or three cytologically normal tests), in accordance with the Task Force's "A" recommendation.

It should be noted that while the Task Force recommendations do play a key role in the process that plans use to review and adopt recommendations for clinical preventive services, the health plans also consult a wide range of other sources, such as the National Cancer Society and other specialty societies and associations (to be described in more detail in Section 2.6 "Competing Recommendations"). One Clinical Advisor from the closed-panel plan indicated that "we do not review [the USPSTF] information and say, "the Task Force recommendation is this, we're going to go implement it." The Task Force is one source—albeit a powerful source – among sources we review. None of our recommendations are purely informed by Task Force [recommendations]. It's just one source of information." Thus, while the USPSTF recommendations play an important role in the process that health plans use to develop their own guidelines, respondents suggested that other factors also affect the adoption and integration of the recommendations.

1.3. Accessing the USPSTF Recommendations

Our conversations with respondents suggest that few health plan staff regularly access the USPSTF recommendations for updates. Less than one quarter of respondents indicated that they regularly check for updated evidence and recommendations from the Task Force. Directors of Quality Improvement were most likely to check for updated evidence and recommendations. Conversely, almost 52% of participants indicated that they do not check for updated evidence from the USPSTF (22 out of 42). Clinical Advisors discussed that they do not usually check for updated evidence and recommendations from the Task Force because they are overburdened and face competing priorities. Given onerous patient care responsibilities, one Clinical Advisor indicated that it is difficult to find the time to "see what the newest thing is [in terms of preventive health recommendations]." Another Clinical Advisor indicated that he/she does not check for updated evidence from the Task Force because it is "outside [his/her] role."

While few health plan staff regularly access the USPSTF recommendations, it appears that people may be strategically consulting the USPSTF recommendations on several occasions. Clinical Advisors described that they may check for updated evidence from the Task Force when (1) the health plan is reviewing or updating its own CPS recommendations or (2) the health plan is launching a quality improvement program or other initiative that focuses on improving the delivery of one particular recommendation. Other respondents suggested that their health plan will consult the USPSTF when new recommendations for clinical preventive services are evolving or have been released (e.g. obesity). Additionally, respondents indicated that they may consult the USPSTF if a clinical preventive service becomes highly controversial, enters the public spotlight, or needs to be reevaluated in light of new scientific findings.

In order to learn more about the users of the USPSTF recommendations, we asked respondents to describe the primary target audience for the recommendations at their health plan. Specifically, we asked: "Who is the primary target audience within your health system for information about recommendations from the Task Force? In other words, when the Task Force disseminates new information about the recommendations, who should they contact at your system?" Health plan respondents identified a number of users of the USPSTF recommendations. A complete list is provided in Exhibit 9 below.

 

Exhibit 9: Users of the USPSTF Recommendations

  • Medical Directors and Chiefs of Staff
  • Directors of Quality Improvement and Performance Improvement
  • Quality Improvement Staff
  • Primary Care Physicians
  • Nurse Practitioners
  • Staff of CPS Recommendations Committees
  • Staff of Medical Policy Departments
  • Performance Improvement Committees
  • Plan Leadership (at the systems-level or central office headquarters)
  • Members

Overall, we found that the Director of Quality Improvement and Quality Improvement Department were cited most often as the primary target audience of the USPSTF recommendations. Medical directors and plan leadership at the systems-level or central office headquarters were also mentioned frequently. The latter was particularly relevant to the governmental plan, which is a local facility that operates within a larger governmental system.

Interestingly, while many of the products and tools produced by the Task Force are intended for use by clinicians and other providers of health care, we found that only 9 out of the 41 respondents (22%) mentioned clinicians as a primary target audience within their health plan for information about recommendations from the Task Force.e Interestingly, few respondents perceived clinicians as an important target audience for the USPSTF recommendations, suggesting a need for further dissemination of the recommendations to this group.


dThe USPSTF found no direct evidence that annual screening achieves better outcomes than screening every 3 years.
eThe sample size for this question is 41 respondents rather than 42. An expert interview was conducted with a senior-level manager at the governmental plan's system headquarters. The respondent was not asked this question.


Return to Contents
Proceed to Next Section

Page last reviewed December 2007
Internet Citation: IV. Findings (continued): Evaluation of the Use of AHRQ and Other Quality Indicators (PDF). December 2007. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/research/findings/final-reports/uspstf/4a.html