IV. Findings (continued)

Evaluation of the Use of AHRQ and Other Quality Indicators (PDF)

2.6. Perceptions of the USPSTF Recommendations

Health plan leadership and staff provided positive feedback on the USPSTF recommendations, describing them as "objective," "unbiased," "purely evidence-based," "less influenced by emotion [than other organizations]," "user-friendly," "broad-based," "balanced," "very thorough," "packaged well," and "neutral." The next section begins with our findings on respondents' perceptions of the packaging of the USPSTF recommendations and the competing recommendations used by health plans. Finally, we move on to a discussion of the alignment of the Task Force's prevention priorities with other systems-level variables and state and Federal initiatives.

Packaging of the Recommendations. Respondents were asked to comment on the packaging of the USPSTF recommendations. Specifically, respondents were asked: "Are the USPSTF recommendations packaged in a user-friendly way to make review, adoption, and integration as straightforward for you as possible?" The majority of respondents found that the USPSTF recommendations are packaged in a user-friendly way. Respondents described that the recommendations are "thorough," "easy to read," and "very easy to follow." Clinical Advisors expressed that the USPSTF recommendations are accessible because they are on the Internet. One Clinical Advisor noted that "since the recommendations are available on the web, it's very easy for people when they are researching a subject." A Clinical Advisor from the closed-panel plan explained that the recommendations are packaged in a highly organized and usable fashion: "There's a recommendation, a letter that goes with it, and there's background. And then there is more background and references. From my standpoint, it's sort of the best packaging possible." Given that the interviews were conducted prior to the release of the USPSTF's 2006 Guide to Clinical Preventive Services, this feedback reflects the USPSTF's 2005 Guide to Clinical Preventive Services.

Several interviews with hybrid plan participants were conducted in late November 2006, after the release of the 2006 recommendations. While we did not ask respondents to comment specifically on the 2006 Guide to Clinical Preventive Services, a Director of Quality Improvement respondent indicated that the 2006 Guide is organized in a more user-friendly format than previous editions: "The recommendations are packaged in a much more user-friendly way than they were before on the Web site." The respondent indicated that it is easier to identify when recommendations were last updated.

Several respondents felt that AHRQ could improve upon the packaging of the USPSTF recommendations. A Medical Director respondent from the open-panel plan described that the electronic format of the recommendations is not optimal because the clinician has to "click several times and then you lose your audience." This respondent's recommendation was to reduce the number of times the user has to click through the Web site to access the recommendations. A Medical Director respondent from the government plan also described the main document as too long and cumbersome to read given his/her clinical responsibilities. The respondent commented: "Here is a 100 page booklet of new recommendations. When do you want me to review that? What day are you going to give me off from my clinical responsibilities to review this information?"

One interesting finding is that respondents from the closed-panel and governmental plan indicated that the packaging of the USPSTF recommendations does not affect whether the recommendations are integrated and delivered at their systems. According to several respondents, the USPSTF recommendations are not distributed to individual physicians, but rather used in combination with recommendations from other sources to inform the development of plan guidelines at the systems-level. A Director of Quality Improvement at the closed-panel plan indicated: "It probably wouldn't matter how [the USPSTF recommendations] are packaged because our process would occur regardless. It's not like we take the Task Force documents and send them out to our physicians."

Since the adoption of clinical preventive services recommendations occurs at the systems-level, many clinicians are not reviewing the recommendations from the Task Force. A Clinical Advisor respondent from the closed-panel plan explained that "[clinicians] are not held accountable to the Task Force—we're held accountable to our leadership. [AHRQ and the Task Force] need to communicate the recommendations to the leadership."

While the USPSTF recommendations target clinical audiences, we found that two Clinical Advisors and Quality Improvement staff for the government plan who indicated that they are not familiar with the packaging of the USPSTF recommendations.

Competing Recommendations. In order to assess the issue of competing recommendations, respondents were asked: "Other than the Task Force, what are your other sources of information for clinical preventive services?" Over 45 different organizations were cited as other sources consulted. The Medical Director at the open-plan discussed that specialty societies, such as the American Academy of Family Practice, are a major source for the plan. Exhibit 11 presents a sample of organizations cited by plan respondents and referenced in prevention materials as sources of clinical preventive services recommendations.

Sources cited by respondents from the open-panel plan include the Centers for Medicare and Medicaid Services, American Health Quality Association, Ambulatory Care Quality Alliance, American Diabetes Association, American Heart Association, American Pediatric Society, and the Centers for Disease Control and Prevention. The closed-panel plan referenced these sources as well as the Canadian Task Force.

The hybrid plan uses recommendations from specialty societies and organizations such as the American Cancer Society for its colorectal cancer screening recommendations. The hybrid plan also uses the American Academy of Pediatrics for recommendations on immunizations and the American Diabetes Association for recommendations on diabetes. Respondents from the hybrid plan mentioned that they consult the HEDIS measures when developing their recommendations for clinical preventive services.

Respondents from the government plan did not have a strong sense of which sources are consulted for clinical preventive services recommendations—potentially because the recommendations are developed exclusively by the headquarters office. However, the Director of Quality Improvement for the governmental plan indicated that some sources were the USPSTF, Institute for Healthcare Improvement, the Patient Safety Foundation, the Centers for Disease Control and Prevention, and HEDIS.

When asked "how do [other organization's] recommendations compare to the Task Force recommendations in terms of ease of adoption and integration," several respondents indicated that the recommendations from the Task Force are easier to use. The USPSTF recommendations were described as "more balanced" and "neutral" and "more strictly evidence-based" than recommendations from specialty societies, which "carry some bias" because they "represent the financial interests of physicians." Other respondents, such as a Director of Quality Improvement from the open-panel plan finds that recommendations from specialty societies like the American Heart Association are relatively comparable to those produced by the Task Force.

 

Exhibit 11: Sources of Recommendations for Clinical Preventive Services

Alphabetical Listing of Organizations
  1. Agency for Healthcare Research and Quality
  2. Ambulatory Care Quality Alliance
  3. American Academy of Family Practice
  4. American Association of Clinical Endocrinologists
  5. American Cancer Society
  6. American College of Cardiology
  7. American College of Gastroenterology
  8. American College of Radiology
  9. American College of Rheumatology
  10. American Diabetes Association
  11. American Gastroenterological Association
  12. American Gastrointestinal Endoscopic Surgeons
  13. American Health Care Quality Association
  14. American Heart Association
  15. American Hospital Association
  16. American Medical Association
  17. American Pediatric Society
  18. American Society for Gastrointestinal Endoscopy
  19. American Society of Colon and Rectal Surgeons
  20. American Thoracic Society
  21. American Urological Association
  22. Canadian Society of Colon and Rectal Surgeons
  23. Canadian Task Force
  1. Centers for Disease Control and Prevention
  2. Centers for Medicare and Medicaid Services
  3. Competing health plans
  4. Gastroenterological Association
  5. Gastrointestinal Consortium
  6. Institute for Clinical Systems Integration
  7. Institute for Healthcare Improvement
  8. Institute of Mental Heath
  9. International Diabetes Center
  10. Medicare
  11. National Asthma Education and Prevention Program
  12. National Business Coalition on Health
  13. National Cholesterol Education Program
  14. National Committee for Quality Assurance, HEDIS
  15. National Institute Of Health
  16. National Institute of Health, National Heart, Lung, and Blood Institute
  17. National Institute on Alcohol Abuse and Alcoholism
  18. National Kidney Foundation
  19. National Osteoporosis Foundation
  20. Patient Safety Foundation
  21. State Medical Society
  22. Surgeon General's Report

Alignment with Systems-Level Variables. Directors of Quality Improvement, Quality Improvement Staff, and Medical Directors were asked "Are the Task Force's prevention priorities aligned with other systems-level variables—such as payer expectations, industry quality indicators, and consumer demand—that you have to manage?" Overall, there was a mix of responses regarding whether the prevention priorities were aligned with systems-level variables. Several respondents indicated that the Task Force's prevention priorities are aligned well or fairly well with other systems-level variables, but did not elaborate on specific variables. For example, a Medical Director respondent at the governmental plan discussed that the Task Force's prevention priorities are closely aligned with systems-level variables because priorities are not overly strict, lax, or too liberal. A few respondents commented that prevention priorities are aligned with payer expectations and quality indicators; however, responses varied on the degree of alignment with these variables.

Respondents across plans described that the prevention priorities are aligned moderately well with quality indicators, like the HEDIS measures. Respondents confirmed the importance of HEDIS performance measures in their health plans, suggesting that the USPSTF recommendations associated with HEDIS measures are evaluated and tracked more frequently than USPSTF recommendations that are not associated with HEDIS measures. Our interviews suggested that many respondents considered the USPSTF's prevention priorities to be very closely aligned with HEDIS measures—so close in fact that many respondents confused the USPSTF recommendations and HEDIS measures or believed that the former are identical to the latter.

When asked to provide specific examples where the recommendations are and are not aligned with other factors, a Director of Quality Improvement for the closed-panel plan indicated that the Task Force's prevention priorities are aligned with quality indicators on colorectal cancer screening, breast cancer screening, and specifically, mammography for breast cancer. Another respondent from the open-panel plan remarked that the prevention priorities are moderately to significantly aligned with quality indicators, poorly aligned with reimbursement, and poorly aligned with consumer demand. A Director of Quality Improvement respondent from the governmental plan also suggested that the Task Force's prevention priorities are not aligned with "reimbursement." However, the respondent commented that it would be difficult to align the Task Force's prevention priorities with the government plan's reimbursement structure, given that the plan provides services to its population regardless of the associated costs.

Our strongest finding is that many respondents believe that the USPSTF's prevention priorities are not aligned well with consumer demand. Several respondents were particularly interested in offering their thoughts on the alignment of the Task Force's prevention priorities with consumer demand. A Medical Director from the open-plan remarked that consumers often have priorities and agendas that are not necessarily aligned with prevention, in general. The respondent went on to say that consumers often want a tangible solution to the problem at hand (e.g., prescription medication or medical procedure) rather than counseling or other medical advice: "Any time a recommendation is going to purport [the provider] doing nothing or doing little or just giving the patient advice to go home, rest, and drink lots of fluids—any recommendation that doesn't support giving patients a prescription—will fly in the face of what the consumer wants."

Other respondents took a slightly different perspective. A Director of Quality Improvement at the government plan remarked that the only time the prevention priorities do not align with consumer demand are for flu immunizations, where the demand often outstretches supply. Given the plan's population is predominantly over 50 years old, the government plan has some difficulty meeting the high demand for the flu vaccine. Another example worth noting is that a Director of Quality Improvement from the hybrid plan suggested that the prostate-specific antigen (PSA) test is not necessarily in alignment with consumer demand, though further detail was not provided.

Our question regarding the alignment with systems-level variables such as consumer demand led to a more global discussion about prevention priorities. One respondent indicated that the Task Force's prevention priorities were not aligned with consumer demand because our society does not value preventive health, evidenced by the obesity epidemic, high prevalence of certain preventable diseases, and rampant prescription drug usage. On a similar note, the Director of Quality Improvement for the open-panel plan cited that prevention and wellness are the last priorities for large purchasers of health care, such as employers. The respondent, who consequently was also clinically trained, expressed frustration that purchasers are more concerned with cost than preventive health care: "What I want to say to some of these purchasers is 'don't you want your employees at work rather than at a hospital or a sick bed?'" As a result, according to this participant, the real goal should be to "push" preventive health care to consumers, and ideally give consumers the tools to manage their own health care.

The theme of personal responsibility reemerged in our interview with a Medical Director for the open-panel plan. According to this respondent, consumers have to be able to advocate for their own health care and ask providers for health screenings and examinations. This respondent told us that consumer demand is poorly aligned with the Task Force's prevention priorities because there is an "unspoken assumption" behind the recommendations that patients are taking some form of responsibility for their care when, in reality, personal responsibility for health care has slipped away from medicine.

Several other respondents remarked that consumers do not have the knowledge and/or tools to request screenings from their providers. Consumers were described as relatively unknowledgeable about the importance of prevention and incapable of starting conversations with their health care providers about preventive health care. For example, one respondent from the closed-panel plan commented that the average person does not understand that preventive health care will help him or her to stay healthy and more active in the long run.

Consumer education about the importance of preventive health was highlighted as a key priority for AHRQ. When asked "do you have suggestions for what the Task Force can do to better align its recommendations with other factors that you need to weigh when deciding whether or not to adopt a recommendation," one respondent remarked that AHRQ could play an important role in educating consumers. A Medical Director from the open-panel plan indicated that: "consumer demand presumes consumers know what they want and what quality is, and I think that has been a very weak area. I think AHRQ could do a lot more to educate consumers about what quality is." Several people suggested that AHRQ should reach out "direct-to-consumer" about preventive health in order to better align the Task Force's prevention priorities with consumer demand. Rather than reaching out to clinicians and the health care community, AHRQ should "get their name out—not with doctors, insurers, and hospitals, but with Joe public." Others recommended education for consumers, in order to "get consumers aligned with the USPSTF rather than the other way around." Specifically, a Director of Quality Improvement at the open-panel plan cited smoking cessation as an example of a similar "direct-to-consumer" movement and suggested that "the more we can push to the consumer to help them to interact with the health care provider, the better."

Another suggestion from a Director of Quality Improvement respondent from the closed-panel plan was to align the Task Force's recommendations with those of other larger specialty groups. The respondent suggested that the USPSTF invite comment or participation from larger specialty groups in developing the recommendations, in order to potentially align recommendations with those of other groups: "It would be fantastic if the American Cancer Society had the same recommendations as the Task Force."

Alignment with Federal and State Initiatives. Similar to the previous section on alignment with systems-level variables, respondents were also asked to comment on whether the Task Force's prevention priorities are aligned with other Federal or state initiatives that their health plans follow. We asked respondents: "Are the Task Force's prevention priorities aligned with Federal initiatives, such as the National Health Quality or Disparities reports, and/or state initiatives that you are required to, or choose, to follow?" Overall, of the 17 responses we received to this question, six respondents indicated that the prevention priorities are aligned with Federal and/or state initiatives. Respondents were also asked "can you provide examples where the recommendations are and are not aligned with other factors." Several interesting responses are provided below as examples:

  • A Medical Director respondent from the open-panel health plan explained that the Task Force's recommendations for screening, such as mammography for breast cancer, are aligned now with Federal and state initiatives, although they were not in alignment ten years ago.
  • A Director of Quality Improvement from the open-panel plan indicated that the Task Force's prevention priorities are aligned with state initiatives, citing that the open-panel plan collaborated with the state as well as pediatricians and family physicians to adopt a set of recommendations for children, based primarily on recommendations from the American Academy of Pediatrics and the Task Force.
  • A Quality Improvement Staff respondent from the closed-panel plan indicated that the Task Force's prevention priorities are aligned with state and Federal initiatives. In particular, the Task Force's prevention priorities for cancer screenings are aligned with state and Federal programs related to tobacco cessation.
  • A Quality Improvement Staff member from the hybrid plan commented that the prevention priorities are aligned with state requirements. However, the respondent did qualify this statement with "in our state, there are not a lot of requirements, other than to be NCQA certified." When asked how the Task Force could better align itself with other Federal and/or state initiatives that the plan follows, the respondent indicated that the hybrid plan is "not really driven by [state or Federal] initiatives."

While we did not plan to ask Directors of Health Information Technology this question, discussions with one particular respondent from the open-plan led to a discussion about alignment with Federal and state initiatives. The respondent commented that the Task Force's prevention priorities related to primary care and family care are aligned with state and Federal initiatives, but not as aligned with state initiatives related to other clinical specialties.

The majority of respondents indicated that they did not know whether the Task Force's prevention priorities are aligned with other state or Federal initiatives because they did not have a strong sense of the national and state priorities. Other respondents chose not to answer this question.

Return to Contents
Proceed to Next Section

Current as of December 2007
Internet Citation: IV. Findings (continued): Evaluation of the Use of AHRQ and Other Quality Indicators (PDF). December 2007. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/research/findings/final-reports/uspstf/4d.html