Selecting Quality and Resource Use Measures: A Decision Guide for Community Quality Collaboratives
Foreword: Improving Quality One Community at a Time
The Agency for Healthcare Research and Quality (AHRQ) partnered with Patrick Romano, M.D., M.P.H., Peter Hussey, Ph.D., and Dominique Ritley, M.P.H., to develop Selecting Quality and Resource Use Measures: A Decision Guide for Community Quality Collaboratives. Our collective motivation was to meet the needs of local health care leaders seeking an evidence-based primer and decisionmaking framework to guide their strategic and operational planning related to performance measurement.
The Decision Guide tapped a panel of 10 community-based leaders—representing purchasers, plans, providers, and consumer organizations—over the yearlong development period. The panel was asked to identify questions that need to be addressed when considering or designing a measurement strategy, which were then used to form the outline for the Decision Guide. Responses to these questions, developed by the expert team led by Patrick Romano, summarize empirical evidence, when it exists, and incorporate expert advice, best practices, and real life case examples to illustrate the breadth of considerations and implementation options.
The resulting Decision Guide is organized into five sections:
- Introduction to performance data.
- Introduction to measures of quality.
- Introduction to resource use/efficiency measures.
- Selecting quality and resource use measures.
- Interpreting quality and resource use measures.
Partnering with local leadership to improve quality and value is not new to AHRQ. AHRQ actively supports community-based quality collaboratives through a portfolio of initiatives, the centerpiece of which is a Learning Network for Chartered Value Exchanges (CVEs). The 24 community quality collaboratives that participate in the Learning Network include active participation from four key stakeholder groups—providers, private and public purchasers, health plans, and consumer organizations—in pursuit of a communitywide system of health care performance measurement, transparency, and improvement. CVEs are involved in a variety of different strategies and approaches to improving quality. But the measurement of quality and resource use, the focus of this Decision Guide, is the keystone activity that undergirds all others and is common to all the participating quality collaboratives.
The 24 community quality collaboratives that are working with AHRQ through the Learning Network provide a window into the broader pool of community collaboratives. Contrary to what might be hypothesized, collaborative implementation does not appear to be constrained to any particular setting or market condition. The 24 collaboratives illustrate a breadth of contexts in which quality collaboratives are being formed across the country. Some of the communities that host collaboratives have one or two dominant health plans, while others are in more competitive markets. Some are urban, while others include a large rural component.1
The 24 collaboratives also illustrate a variety of approaches to both operational policy and strategy. Most collaboratives govern by consensus, although they vary in terms of how often they meet.
Collaboratives' sources of operating revenue include dues from local members, sale of collaborative products (e.g., data) or services, and grant funding from foundations, governments, and local stakeholders. Most collaboratives rely at least in part on in-kind resources from their stakeholder members.2 Of the four categories of collaborative membership, consumer organizations tend to have the fewest representatives involved in the process.
Approaches to collecting data vary across the collaboratives and quality measures include a rich mix of structural, process, and outcome indicators. Collaborative models share some common features that affect decisionmaking related to data and measure selection (e.g., decisionmaking by consensus, reliance on in-kind contributions). Other elements vary widely, such as size of annual operating budget, menu of quality measures, and use of health information technology. Some collaboratives have overcome significant challenges to quality measurement and reporting. Although not commonplace, a smaller subset of collaborative pioneers is moving beyond public reporting by developing provider or consumer incentives to reinforce their respective quality agendas. This breadth of environmental contexts and range of design models suggests that collaboratives are adaptable and feasible to implement nationwide, but budgetary, political, and other challenges can temper the pace of progress.1
In addition to the Learning Network for Chartered Value Exchanges, AHRQ's library of past and current quality implementation partnerships includes, for example:
- The AHRQ Learning Network on Quality-based Purchasing, which provided a forum for employers and State Medicaid agencies to learn about pay-for-performance best practices from experts as well as each other.
- The AHRQ Quality Indicators Learning Institute, which provided a forum for discussing and facilitating the use of the AHRQ Quality Indicators (QIs) in statewide and regional programs that report hospital quality measures to the public.
- The AHRQ Medicaid Medical Directors Learning Network, which provides a venue for clinical leaders of State Medicaid programs to connect with other organizations interested in using evidence-based medicine to make policy decisions that affect Medicaid programs.
This Guide is the latest in a series of user-driven guides developed by AHRQ to distill and summarize evidence, expertise, and implementation considerations for an audience of local decisionmakers. Other guides are:
- Pay for Performance: A Decision Guide for Purchasers, an evidence summary organized around 20 questions that span 4 phases of purchaser decisionmaking related to provider incentives: contemplation, design, implementation, and evaluation.3
- Consumer Financial Incentives: A Decision Guide for Purchasers, an evidence summary organized around 21 questions that reviews the application of incentives to 5 types of consumer decisions: selecting a high-value provider, selecting a high-value health plan, deciding among treatment options, reducing health risks by seeking preventive care, and reducing health risks by decreasing or eliminating high-risk behavior.4
These and other evidence-based resources for community quality collaboratives can be ordered by e-mailing AHRQPubs@ahrq.hhs.gov, calling AHRQ's Publications Clearinghouse at 1-800-358-9295, or downloading from AHRQ's Web site at www.ahrq.gov/qual/value/localnetworks.htm.
While AHRQ has been working with scientists who develop the evidence base and decisionmakers in the field who apply the evidence base to guide their actions to make a positive impact on quality, we as a Nation still have a long way to go to achieve a 21st century health care system that serves all Americans well. Unfortunately, we know little about the long-term impact of various collaborative strategies on quality of care or on collaboratives' sustainability over time. More attention and resources are needed to build the evidence base about best practices in translating measurement into performance improvement (e.g., via public reporting and payment incentives) and to disseminate what we know to more decisionmakers who are in a position to act on the evidence. Much remains to be done to better understand how communities can most effectively engage purchasers, plans, providers, and consumer organizations in applying performance measures through reporting, pay-for-performance, consumer incentives, and HIT initiatives.
AHRQ expresses appreciation to the team of Patrick Romano, Peter Hussey, and Dominique Ritley and the interdisciplinary panel of reviewers that included community collaboratives, Federal Agency representatives, and other expert colleagues. In publishing this Decision Guide, we hope to contribute to and advance an ongoing local and national dialogue related to how community collaboratives and their component stakeholders can improve performance measurement and quality of care. We hope this Decision Guide informs their deliberations, and we welcome feedback.
The logic of aligning local health care leadership interests through a community collaborative to achieve a common quality improvement or value-enhancing agenda resonates. While national policies can support quality of care, real improvement requires hard work at the local level and is accomplished only one community at a time.
Senior Fellow, AHRQ
- McNamara P, De La Mare J, Moss D. Local, multi-stakeholder quality collaboratives: what can the U.S. experience tell us about their feasibility as part of a national effort to improve health care quality? Rockville, MD: Agency for Healthcare Research and Quality; May 2009. Working paper.
- McNamara, P. CVE sustainability—Overview. Presentation at AHRQ's Third National Meeting of Chartered Value Exchanges, Washington, DC, July 2009.
- Dudley RA, Rosenthal MB. Pay for performance: a decision guide for purchasers. (Final Contract Report) Rockville, MD: Agency for Healthcare Research and Quality; April 2006. AHRQ Pub. No. 06-0047. Available at: www.ahrq.gov/qual/p4pguide.htm.
- Dudley RA, Tseng CW, Bozic K, et al. Consumer financial incentives: a decision guide for purchasers (Final Contract Report). Rockville, MD: Agency for Healthcare Research and Quality; November 2007. AHRQ Pub. No. 07(08)-0059. Available at: