V. Thematic Overviews

Evaluation of the Use of AHRQ and Other Quality Indicators (PDF)

 

Section V takes the evaluation findings a step further by exploring several of the most interesting and important cross-cutting themes. Next, we present five thematic overviews that analyze themes related to the adoption, integration and delivery, and dissemination of the USPSTF recommendations. Each thematic overview explores a single theme, relying on published and unpublished literature and the interviews to present top-level conclusions and to suggest areas for further research.

Our thematic overviews follow in this order:

  1. The Impact of Pay-for-Performance on the Delivery of the USPSTF Recommendations (Overview 1).
  2. The Role of Health IT in the Integration and Delivery of the USPSTF Recommendations (Overview 2).
  3. Systems-Level Changes to Encourage the Delivery of the USPSTF Recommendations (Overview 3).
  4. Delivering the USPSTF Recommendations in a Rural Health Care Setting (Overview 4).
  5. The Impact of Health Plan Structures on the Delivery and Integration of the USPSTF Recommendations (Overview 5).

The Impact of Pay-for-Performance on the Delivery of the USPSTF Recommendations


Overview. Health plans are using pay-for-performance in an attempt to increase the delivery of clinical preventive services. We explore the use of pay-for-performance in four different types of managed care plans.

Key findings include:

  • Some health plans reward employees with annual bonuses, group meals, and/or recognition awards based on their delivery of certain clinical preventive services.
  • Health plan leaders express mixed support for pay-for-performance, emphasizing the importance of proper implementation of pay-for-performance initiatives and the need for better integration with other quality improvement activities.
  • Future research should explore ways in which pay-for-performance initiatives affect patient outcomes via the delivery of clinical preventive services.

Pay-for-performance, the idea of aligning financial incentives with high quality health care, has become a popular mechanism for fostering quality improvement in health care systems. A small albeit growing body of research explores the potential for pay-for-performance programs to change health care system behavior. Studies have specifically examined the impact of pay-for-performance initiatives on the delivery of a range of clinical preventive services such as cancer screenings, pediatric immunizations, and testing for diabetic patients.1-8 However, little research has explored the impact of pay-for-performance programs on the delivery of the clinical preventive services specifically recommended by the U.S. Preventive Task Force (USPSTF).9 Furthermore, no studies have examined the use of pay-for-performance programs that reward the delivery of the USPSTF recommendations in different types of managed health care plans.

As part of a larger evaluation of the USPSTF recommendations for clinical preventive services, NORC at the University Chicago explored the role of health plan reimbursement structures on the delivery of clinical preventive services. NORC studied the integration and delivery of the USPSTF recommendations in four different types of health plans: a closed-panel health plan, open-panel health plan, hybrid health plan having both open and closed-panel characteristics, and governmental health plan. Structured interviews were conducted with over 40 health plan staff members, including Medical Directors, Directors of Quality Improvement, Directors of Health Information Technology (IT), Quality Improvement and Health IT staff, and Clinical Advisors (clinicians who also serve in a leadership or broader prevention role). Respondents were asked five questions regarding the reimbursement structure at their health plans.10 The following overview synthesizes key findings with respect to the role of the reimbursement structure as a means for increasing the delivery of clinical preventive services in four different managed care plans.

Rewarding the Delivery of Clinical Preventive Services

The interviews confirmed that the closed-panel, governmental, and hybrid plans have begun to tie reimbursement to quality for clinical preventive services. While staff physicians of all these plans are salaried, there are additional opportunities to earn performance-based financial bonuses. The closed-panel plan has the most sophisticated pay-for-performance mechanism, which involves financial incentives, public recognition, and other in-kind perks. Physicians receive bonuses for performance on clinical quality elements which include one USPSTF recommendation: mammography for breast cancer screening. The financial pay-for-performance mechanism is combined with team-based incentives for performing well; incentives at the provider group level included public recognition and group meals.

The hybrid plan also recently incorporated pay-for-performance mechanisms into its physician reimbursement structure for those physicians employed by the plan. Physicians are awarded bonuses for delivery of preventive services such as mammography for breast cancer screening, Papanicolaou smear for cervical cancer screening, and childhood and adolescent immunizations. A Director of Quality Improvement for the hybrid plan discussed that it has a generous pay-for-performance program which could conceivably add as much as 20% to a primary care physician's income. When asked how long it would take to see a change in clinician's delivery of the service, one Director of Quality Improvement from the hybrid plan indicated that the immediacy of the effect depends upon the size of the financial incentive: "If the [financial change] is small, maybe never."

The open-panel plan was the only plan whose reimbursement structure does not reward the delivery of clinical preventive services. Physicians are reimbursed for services rendered regardless of quality; and there are no additional incentives or disincentives to either deliver or withhold health care services. Respondents from the open-plan cited several barriers to implementing a pay-for-performance initiative. From a clinical perspective, open-panel plan providers are concerned about the pay-for-performance movement, in general. The Director of Health IT indicated that "in terms of pay-for-performance, providers are asking 'What are we being paid for today if you want to pay us for quality tomorrow?'" The plan's Medical Director also described the budgetary implications associated with pay-for-performance programs as a deterrent: "At the end of the day, no matter what you do in regard to paying incentives to physicians, it has to be budget neutral. The money must come out of someone else's pocket.it typically comes out of another provider's pocket in the form of less high-tech imaging tests, fewer surgical procedures, or maybe fewer admissions." A Director of Quality Improvement commented that the plan is developing a broader pay-for-performance effort that will include a focus on preventive services.

The governmental plan recently implemented a pay-for-performance initiative for its senior management in fiscal year 2006. A Director of Quality Improvement described the pay-for-performance mechanism generally as "providers who score 85% or better get 10% on top of their baseline salary at the end of the year." According to respondents, given the little time elapsed since its implementation, the program's impact will likely not be ascertained until the end of 2007.

In terms of pay-for-performance, providers are asking 'What are we being paid for today if you want to pay us for quality tomorrow?'
—Director of Health IT, Open-Panel Plan

Is the Reimbursement Structure an Effective Means for Increasing Service Delivery?

Many respondents across plans indicated that the reimbursement structure is an effective means for increasing the delivery of clinical preventive services. A Director of Health IT from the open-panel plan indicated that the reimbursement structure may impact compliance with the USPSTF recommendations: "Whatever you pay [providers] for, they're going to do. If you structure [reimbursement systems] right, yes, it will improve compliance with the [USPSTF] recommendations." Other respondents across plans commented that the reimbursement structure is an effective means for increasing the delivery of clinical preventive services, but only one piece of a larger puzzle. A Medical Director of the open-panel plan indicated that pay-for-performance programs should be tailored to work in accordance with other quality improvement initiatives such as health plan profiling tools that allow physicians to examine one another's performance. According to the respondent, pay-for-performance initiatives are only effective in combination with other strategies that encourage transparency: "You need to look at a network strategy. All of these things work in synergy. [Pay-for-performance] is effective but has to be part of a whole program that looks at performance."

Respondents cited a few disincentives to utilizing the reimbursement structure to improve the delivery of clinical preventive services. Staff from the closed-panel plan highlighted the concern that providers will learn how to "game" the system to manipulate a pay-for-performance program to increase payment. A Director of Quality Improvement described this problem as "a constant struggle to get people to do the right thing for their patients, and not for themselves." Respondents at the governmental plan indicated that while the reimbursement structure is designed to reward providers, performance of certain services may be directly attributable to non-clinical staff members who also contribute to the delivery of clinical preventive services. A Clinical Advisor indicated that "a lot of people who work toward [delivering a clinical preventive service] don't get any benefit or bonus. And the providers don't turn around and say 'thank you.'"

"You need to look at a network strategy. All of these things work in synergy. [Pay-for-performance] is effective but has to be part of a whole program that looks at performance."
—Medical Director, Open-Panel Plan

Conclusions and Further Exploration

Overall, our findings suggest that different types of health plans are incorporating unique pay-for-performance mechanisms that specifically reward the delivery of clinical preventive services. In addition, the majority of respondents across different types of health plans indicated that the reimbursement structure is an effective means for increasing the appropriate delivery of clinical preventive services. Future research should continue to explore the impact of health plan reimbursement structures and pay-for-performance mechanisms on preventive health outcomes. On a separate note, there is also a need for studies that examine the role of clinical preventive recommendations, such as those produced by the USPSTF, in developing reimbursable outcomes. Major purchasers of health care, such as the Pacific Business Group on Health, have integrated the USPSTF recommendations into their reimbursement structures.11 Future research should explore the potential for the USPSTF recommendations to be integrated directly into health plan reimbursement structures, and should document the impact of integration on the delivery of clinical preventive services and related patient health outcomes.

References

1. Petersen LA, Woddard LD, Urech T, Daw C, and Sookanan S. Does pay-for-performance improve the quality of health care? Annals of Internal Medicine 2006; 145:265-72.
2. Hillman AL, Ripley K, Goldfarb N, Nuamah I, Weiner J, Lusk E. Physician financial incentives and feedback: failure to increase cancer screening in Medicaid managed care. American Journal of Public Health 1998; 88:1699-701.
3. Rosenthal MB, Frank RG, Li Z, Epstein AM. Early experience with pay-for-performance: from concept to practice. Journal of the American Medical Association 2005; 294:1788-93.
4. Kouides RW, Bennett NM, Lewis B, Cappuccio JD, Barker WH, LaForce FM. Performance-based physician reimbursement and influenza immunization rates in the elderly. The Primary-Care Physicians of Monroe County. American Journal of Preventive Medicine 1998; 14:89-95.
5. Hillman AL, Ripley K, Goldfarb N, Weiner J, Nuamah I, Lusk E. The use of physician financial incentives and feedback to improve pediatric preventive care in Medicaid managed care. Pediatrics 1999; 104:931-5.
6. Roski J, Jeddeloh R, An L, Lando H, Hannan P, Hall C, et al. The impact of financial incentives and a patient registry on preventive care quality: increasing provider adherence to evidence-based smoking cessation practice guidelines. Preventive Medicine 2003; 36:291-9.
7. Grady KE, Lemkau JP, Lee NR, Caddell C. Enhancing mammography referral in primary care. Preventive Medicine 1997; 26:791-800.
8. Fairbrother G, Hanson KL, Friedman S, Butts GC. The impact of physician bonuses, enhanced fees, and feedback on childhood immunization coverage rates. American Journal of Public Health 1999; 89:171-5.
9. In 1984, the U.S. Public Health Service created the USPSTF as an independent panel of experts in primary care and prevention that systematically review the evidence of effectiveness and develop recommendations for clinical preventive services. The Agency for Healthcare Research and Quality (AHRQ) began sponsoring USPSTF activities in 1998 and includes the USPSTF recommendations as part of their diverse Prevention Portfolio. The USPSTF recommendations are developed by a team of clinicians and are based on a thorough review of evidence including individual studies such as randomized controlled trials, costs, the negotiations of benefits and harms, and the evidence as a whole.
10. Respondents were asked a series of five questions about their perspectives on the reimbursement structure at their health plans: (1) Does the reimbursement structure at [health plan] reward the appropriate delivery of clinical preventive services? (2) Can you provide examples of Task Force recommendations that are integrated into the reimbursement structure? (3) We understand that certain Task Force recommendations are more difficult to incorporate into the reimbursement structure. Do you think that there are any disincentives to the appropriate delivery of clinical preventive services in the reimbursement structure? (4) When [health plan] makes a change in the reimbursement structure to reward the appropriate delivery of clinical preventive services, how long does it generally take to see changes in clinicians' delivery of the service? (5) Do you believe the reimbursement structure is an effective means for increasing the appropriate delivery of clinical preventive services?
11. Schauffler HH and Rodriguez T. Exercising purchasing power for preventive care. Health Affairs 1996; 15:73-85.

Return to Contents

 

The Role of Health IT in the Integration and Delivery of the USPSTF Recommendations


Overview. Health plans are using health IT tools to support the integration and delivery of the USPSTF recommendations for clinical preventive services. We explore the role of health IT in four different types of managed care plans.

Key findings include:

  • Electronic medical records, community health records, clinical decision support tools, and the Internet are used to integrate and deliver the USPSTF recommendations.
  • Two key challenges of using health IT to integrate and deliver the USPSTF recommendations are inconsistent integration and inadequate information exchange.
  • Future research should explore new ways to use health IT in concert with the USPSTF recommendations to track patient outcomes. over the long run.

As the use of health information technology (IT) increases in health care, it is apparent that health IT has enormous potential to improve the integration and delivery of recommended clinical preventive services. A growing body of research supports the cost-effectiveness of health IT tools such as electronic medical record systems (EMRs), computerized physician order entry (CPOE), and clinical decision support systems.1 Chaudhry et al. (2006) determined that despite the costs of implementation, health IT can generate significant cost savings when used in preventive medicine settings and facilitate the delivery of preventive services. The study also concluded that the major effect of health IT on quality of health care was to enhance adherence to recommendations, particularly those focused on preventive care.

Other researchers have concluded that health IT is effective in improving outcomes related to vaccine and screening recommendations. Hillestad et al. (2005) found that the use of EMR systems to provide reminders for screenings for colorectal cancer could prevent between 17,000 and 38,000 deaths per year.2 Health IT tools facilitate the delivery of clinical preventive services in multiple settings, including primary care settings serving disadvantaged populations3 and urban health department clinics.4 However, little research has been conducted to explore the role of health IT in the integration and delivery of the clinical preventive services specifically recommended by the U.S. Preventive Services Task Force (USPSTF).5 Furthermore, no studies have examined the use of health IT tools to facilitate integration and delivery of the USPSTF recommendations in different types of managed care plans.

As part of a larger evaluation of the USPSTF recommendations for clinical preventive services, NORC at the University of Chicago explored the role of health IT tools in the integration and delivery of the USPSTF recommendations in four different types of health plans: a closed-panel health plan, open-panel health plan, hybrid health plan having both open and closed-panel characteristics, and governmental health plan. Structured interviews were conducted with over 40 health plan staff members, including Medical Directors, Directors of Quality Improvement and Health IT, Quality Improvement and Health IT staff, and Clinical Advisors (clinicians who also serve in a leadership or broader prevention role). Respondents were asked to provide examples of how the USPSTF recommendations are integrated into their health plan's IT system. This overview synthesizes key findings with respect to the role of the health IT in the integration and delivery of clinical preventive services in four different managed care plans.

Health IT Fosters Integration and Delivery of USPSTF Recommendations

Health plan staff respondents were asked whether health IT tools are utilized to integrate clinical preventive services recommendations from the USPSTF and other sources. The closed-panel plan, hybrid plan, and governmental plan utilize EMRs to integrate the USPSTF recommendations and foster information sharing among providers and patients about preventive services. While the open-panel plan did not utilize an EMR to integrate the USPSTF recommendations at the time interviews were conducted, it is in the process of developing another type of electronic health tool called a patient-centered community health record (CHR). This tool will provide an electronic reserve of health care information for all members, and the information will be viewable by all providers, fostering improvements in care coordination for patients who access care through multiple providers.

Health plans reported that clinical decision support and other health IT tools were also used to integrate the USPSTF recommendations. Order sets are used by the hybrid plan to identify patients in need of preventive services for particular conditions. The closed-panel and governmental plans utilize clinical reminders to assist medical management decision-making by prompting providers to deliver the USPSTF recommendations. The open-panel plan also uses the Internet as a hub for disseminating the USPSTF recommendations, reaching both providers and members.

Barriers to Utilizing Health IT: Inconsistent Integration and Inadequate Information Exchange

The interviews confirmed that the health plans face challenges in using health IT tools to integrate and deliver the USPSTF recommendations. One of the greatest challenges, described by respondents from the open-panel plan and the hybrid plan, is the issue of inconsistent integration of health IT tools such as the EMR. The hybrid plan operates within a larger health system that integrates the USPSTF recommendations into its EMR. However, since only approximately half of plan members access services through the health system, integration is limited. Contracted providers, who provide services to members of multiple health plans, are provided access to, but often do not use, the EMR. The end result is incomplete patient records as the EMR only captures data on patient visits to system-employed providers.

The open-panel plan faces a similar challenge—inadequate information exchange. This issue is somewhat inherent to open-panel plans where patients see multiple providers for preventive health services. A Director of Health IT from the open-panel plan described that "[health IT tools] are being used to try to bring together a fragmented care system. So if [a patient] goes to a specialist, [the provider] can identify what other treatments they've received. In a preferred provider organization environment, a lot of people will go [to other providers] without telling their primary care provider." This issue may be rectified by the plan's upcoming implementation of a CHR. However, the open-panel plan may experience additional problems related to coordinating a plan-wide CHR with affiliated providers' existing EMRs.

"[Health IT tools] are being used to try to bring together a fragmented system."
—Director of Health IT, Open-Panel Plan

Conclusions and Further Exploration

Health plans have differing capacities to use health IT to integrate and deliver the USPSTF recommendations for clinical preventive services. The governmental and closed-panel systems, where providers are employees of the plans, had the greatest integration of Task Force recommendations using health IT, followed by the hybrid system. Finally, the open-panel system, where contracted providers may provide services under several health plans, had the least integration of the Task Force recommendations using health IT, though this may change in the future.

In the future, health plans should examine new opportunities for utilizing health IT tools to integrate and deliver the USPSTF recommendations. The health plans in this study do not currently utilize their health IT tools to monitor the delivery of the USPSTF recommendations in a systematic matter or to track long-term patient outcomes. Health IT tools have an enormous untapped potential to improve patient outcomes by increasing the delivery of clinical preventive services. Health plans would benefit from investing time and funding in developing new methods to utilize health IT in concert with the USPSTF recommendations. Next steps might include using health IT to compare long-term patient outcomes to provider delivery of the USPSTF recommendations. Future research should measure the impact of health IT on the delivery of clinical preventive services in different types of health plans. Furthermore, studies should explore which health IT tools are most effective in improving the delivery of the USPSTF recommendations.

References

1. Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, Morton SC, Shekelle PG. Systematic review: impact of health information technology on quality, efficiency and costs of medical care. Annals of Internal Medicine 2006; 144:742-752.
2. Hillestad R, Bigelow J, Bower A, Girosi F, Meili R, Scoville R, Taylor R. Can electronic medical record systems transform health care? Potential health benefits, savings, and costs. Health Affairs 2005; 24:1103-1117.
3. Roetzheim RG, Christman LK, Jacobsen PB, Cantor AB, Schroeder J, Abdulla R, Hunter S, Chirikos TN and Krischer JP. A randomized controlled trial to increase cancer screening among attendees of community health centers. Annals of Family Medicine 2004; 2:294-300.
4. Burack RC, Gimotty PA. Mammography in inner-city settings: the sustained effectiveness of computerized reminders in a randomized controlled trial. Medical Care 1997; 35:921-931.
5. In 1984, the U.S. Public Health Service created the USPSTF as an independent panel of experts in primary care and prevention that systematically review the evidence of effectiveness and develop recommendations for clinical preventive services. The Agency for Healthcare Research and Quality (AHRQ) began sponsoring USPSTF activities in 1998 and includes the USPSTF recommendations as part of their diverse Prevention Portfolio. The USPSTF recommendations are developed by a team of clinicians and are based on a thorough review of evidence including individual studies such as randomized controlled trials, costs, the negotiations of benefits and harms, and the evidence as a whole.

Return to Contents
Proceed to Next Section

Current as of December 2007
Internet Citation: V. Thematic Overviews: Evaluation of the Use of AHRQ and Other Quality Indicators (PDF). December 2007. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/research/findings/final-reports/uspstf/5.html