Opportunities for Advancing Delivery System Research
ARRA Grants Initiative
Improving the way that care is delivered is critical to the drive for health care reform. The public and private sectors are experimenting with promising new ways to deliver and pay for care. Still, rigorous research is needed to determine the most effective forms of system design and improvement approaches, under what circumstances they are effective and for whom, and what it would take to scale up the more promising new forms of delivery. AHRQ has a long history of support for this type of research. In 2009 AHRQ awarded 16 grants on delivery system research and the spread of evidence-based practices through delivery systems. These grants were funded by the American Recovery and Reinvestment Act (ARRA) (refer to Research Awards [2.2 and 2.3] and Dissemination, Translation, and Implementation Awards [2.1].
Grantee Presentations and Publications
These grants will end in September 2013. Early presentations and publications, which describe the projects and report early results, are available at http://www.ahrq.gov/professionals/systems/system/delivery-system-initiative/arragranteepubs.html.
Advanced Methods in Delivery System Research—Webinars on Planning, Executing, Analyzing, and Reporting Research on Delivery System Improvement
Series of five webinars sponsored by AHRQ's Delivery System Initiative in partnership with the AHRQ PCMH program. Topics include Fuzzy Set Analysis, Statistical Process Control, Logic Models, Formative Evaluation, and Mixed Methods.
AHRQ Expert Meeting on the Challenge and Promise of Delivery System Research: Meeting Summary
Advancing Delivery System Research. To advance research on delivery system design and improvement, AHRQ convened a meeting of its ARRA delivery system grantees, leading Federal officials and health policy experts, and nationally known health service researchers. The meeting was held February 16-17, 2011, in Sterling, VA.
Their research covers areas such as the primary care medical home, care coordination, payment, reporting, improvement of care quality and efficiency, and research methods. The meeting sought to advance the national agenda for delivery system research; identify key topics and methods deserving greater attention by researchers and funding agencies; and foster dialogue among delivery system researchers. The meeting was organized around workgroup discussions of three white papers:
- Jeffrey Alexander, University of Michigan, Methods and Metrics Issues in Delivery Systems Research.
- (subsequently published as Alexander JA, Hearld LR. Methods and metrics challenges of delivery-system research. Implement Sci 2012 Mar 12;7(1):15. Available at http://www.implementationscience.com/content/pdf/1748-5908-7-15.pdf).
- Lawrence Casalino, Weill Cornell, Identifying Key Areas for Delivery Systems Research.
- Joseph McCannon, Centers for Medicare & Medicaid Services Innovations Center, The Spread Problem.
This summary lists opportunities for advancing delivery system research and recommends activities to support this research. In addition, it provides messages for delivery system researchers.
Opportunities for Advancing Delivery System Research
- Encourage cutting-edge methodologies and research designs (for examples, go to "Messages for Delivery System Researchers").
- Call for mixed-method research.
- Encourage use of diverse research designs.
- Encourage research on implementation and practice.
- Increase support for process evaluations and periodic, iterative assessments of new and continuing demonstration projects.
- Fund studies of demonstrations and improvement initiatives over longer time periods to allow for meaningful evaluation of their impact and sustainability.
- Take steps to enhance uptake and spread of findings.
- As it becomes available, provide information on successful innovations and interventions.
- Test findings from successful pilots or demonstrations in additional settings (gradual scale-up).
- Support greater involvement of patients, clinicians, managers, and decisionmakers in planning and development of research.
- Focus research on best ways to solve problems of concern to stakeholders (e.g., hospital readmissions).
- Support studies that directly examine how dissemination and spread occur.
- Develop banks of key delivery-system concepts and measures to enhance consensus on definitions and operationalizaition of important concepts for delivery system research. Until people are using the same words to describe the same things (e.g., "care coordination") it will be difficult to establish the impact of particular practices or processes.
- Encourage adoption of standardized evaluation approaches across funders for more meaningful cross-comparisons and syntheses from diverse studies.
- Adopt common research frameworks (although these can vary for particular types of interventions, such as patient-centered medical home) and use them in developing funding announcements. Require or encourage researchers to address the frameworks in their proposals.
- Showcase completed studies that use new and diverse designs and methods.
- Bundle information on multiple, related innovations in issue briefs and other reports.
- Encourage journal editors and study sections to support research using mixed methods and diverse research designs.
- Support training and conferences on new research designs, qualitative research, and new analytic techniques.
Messages for Delivery System Researchers
Research Logic and Frameworks
- Provide more fully developed logic models (also known as program theory) for expected effects.
- Develop and apply common evaluation frameworks for examining similar change initiatives (e.g., patient-centered medical home, pay for performance).
- Increase focus on discovering how interventions operate in diverse settings, rather than seeking a "best intervention" or a "best organizational type." (Ask: "When does it work and how?"instead of "Does it work and what works best?").
- Expand examination of external forces affecting change in delivery organizations beyond payment and reporting. Consider effects of professional norms, regulations and policies, and influence of hospitals on ambulatory providers; roles of other external stakeholders; and availability of technical knowledge, tools, and resources.
- Carefully analyze how organizational features, such as culture, structure, resource base (e.g., safety net providers), and patient characteristics, affect organizations' responses to external incentives and forces.
- Examine interaction among providers and relations between medical care providers and providers of social, educational, and community services.
Research Design and Data Analysis
- Use a wider range of research designs, including interrupted time series; stepped wedge; ethnography; and statistical identification of positive deviants followed by qualitative analysis.
- Make more use of mixed methods and seek synergies across multiple data sources (e.g., qualitative and quantitative evaluation data).
- Model change and program developments over time (e.g., growth and decay of intervention effects; development and adaptation of interventions during stages of implementation).
Measurement
- Include (and, if needed, develop) more measures of care outcomes, rather than care processes.
- Include measures of care outcomes for patients receiving care from multiple providers and measures of how multiple care providers interact to affect care of these complex patients.
- Develop and refine measures of capability and readiness to change; sustainability; organizational contexts; broad mix of incentives and external influences (not just payment); and provider organizational culture.
- Develop measures that are aligned across multiple levels (e.g., individual practitioner, group, care system).
Reporting and Dissemination of Findings
- Design research with dissemination in mind; for example, consider target audiences in developing research plans.
- Make measures and findings more meaningful to those whom research seeks to affect (e.g., patients, policymakers), possibly by involving them in research development.
Next Steps
In keeping with the meeting participants' recommendations AHRQ is planning the following:
- Sessions at AHRQ Annual Meeting: Delivery System Research: Discovering Ways to Improve Care Value (open to all attendees); Research Updates and Challenges (participation limited to ARRA Delivery System Grantees).
- Electronic portal supporting communication among ARRA Delivery System grantees about research methods, common investigation topics, and related learning and funding opportunities.
- Issue briefs describing how ARRA grantee research results contribute to emerging findings on key delivery system issues (e.g., reducing rehospitalization, enhancing patient-centered care, public reporting on physician performance).
For additional information, contact:
Michael.Harrison@ahrq.hhs.gov