Delivery System Research Initiative

ARRA Grants Initiative

Findings from a set of 16 grants on improving delivery systems and on spreading evidence-based practices through delivery systems; recommendations and methods for advancing delivery system research.

AHRQ Expert Meeting on the Challenge and Promise of Delivery System Research: Meeting Summary

Advancing Delivery System Research. To advance research on delivery system design and improvement, AHRQ convened a meeting of its ARRA delivery system grantees, leading Federal officials and health policy experts, and nationally known health service researchers. The meeting was held February 16-17, 2011, in Sterling, VA.

Their research covers areas such as the primary care medical home, care coordination, payment, reporting, improvement of care quality and efficiency, and research methods. The meeting sought to advance the national agenda for delivery system research; identify key topics and methods deserving greater attention by researchers and funding agencies; and foster dialogue among delivery system researchers. The meeting was organized around workgroup discussions of three white papers:

This summary lists opportunities for advancing delivery system research and recommends activities to support this research. In addition, it provides messages for delivery system researchers.

Opportunities for Advancing Delivery System Research

  • Encourage cutting-edge methodologies and research designs (for examples, go to "Messages for Delivery System Researchers").
    • Call for mixed-method research.
    • Encourage use of diverse research designs.
  • Encourage research on implementation and practice.
    • Increase support for process evaluations and periodic, iterative assessments of new and continuing demonstration projects.
  • Fund studies of demonstrations and improvement initiatives over longer time periods to allow for meaningful evaluation of their impact and sustainability.
  • Take steps to enhance uptake and spread of findings.
    • As it becomes available, provide information on successful innovations and interventions.
    • Test findings from successful pilots or demonstrations in additional settings (gradual scale-up).
    • Support greater involvement of patients, clinicians, managers, and decisionmakers in planning and development of research.
    • Focus research on best ways to solve problems of concern to stakeholders (e.g., hospital readmissions).
    • Support studies that directly examine how dissemination and spread occur.
  • Develop banks of key delivery-system concepts and measures to enhance consensus on definitions and operationalizaition of important concepts for delivery system research. Until people are using the same words to describe the same things (e.g., "care coordination") it will be difficult to establish the impact of particular practices or processes.
  • Encourage adoption of standardized evaluation approaches across funders for more meaningful cross-comparisons and syntheses from diverse studies.
    • Adopt common research frameworks (although these can vary for particular types of interventions, such as patient-centered medical home) and use them in developing funding announcements. Require or encourage researchers to address the frameworks in their proposals.
  • Showcase completed studies that use new and diverse designs and methods.
  • Bundle information on multiple, related innovations in issue briefs and other reports.
  • Encourage journal editors and study sections to support research using mixed methods and diverse research designs.
  • Support training and conferences on new research designs, qualitative research, and new analytic techniques.

Messages for Delivery System Researchers

Research Logic and Frameworks

  • Provide more fully developed logic models (also known as program theory) for expected effects.
  • Develop and apply common evaluation frameworks for examining similar change initiatives (e.g., patient-centered medical home, pay for performance).
  • Increase focus on discovering how interventions operate in diverse settings, rather than seeking a "best intervention" or a "best organizational type." (Ask: "When does it work and how?"instead of "Does it work and what works best?").
  • Expand examination of external forces affecting change in delivery organizations beyond payment and reporting. Consider effects of professional norms, regulations and policies, and influence of hospitals on ambulatory providers; roles of other external stakeholders; and availability of technical knowledge, tools, and resources.
  • Carefully analyze how organizational features, such as culture, structure, resource base (e.g., safety net providers), and patient characteristics, affect organizations' responses to external incentives and forces.
  • Examine interaction among providers and relations between medical care providers and providers of social, educational, and community services.

Research Design and Data Analysis

  • Use a wider range of research designs, including interrupted time series; stepped wedge; ethnography; and statistical identification of positive deviants followed by qualitative analysis.
  • Make more use of mixed methods and seek synergies across multiple data sources (e.g., qualitative and quantitative evaluation data).
  • Model change and program developments over time (e.g., growth and decay of intervention effects; development and adaptation of interventions during stages of implementation).


  • Include (and, if needed, develop) more measures of care outcomes, rather than care processes.
  • Include measures of care outcomes for patients receiving care from multiple providers and measures of how multiple care providers interact to affect care of these complex patients.
  • Develop and refine measures of capability and readiness to change; sustainability; organizational contexts; broad mix of incentives and external influences (not just payment); and provider organizational culture.
  • Develop measures that are aligned across multiple levels (e.g., individual practitioner, group, care system).

Reporting and Dissemination of Findings

  • Design research with dissemination in mind; for example, consider target audiences in developing research plans.
  • Make measures and findings more meaningful to those whom research seeks to affect (e.g., patients, policymakers), possibly by involving them in research development.

Next Steps

In keeping with the meeting participants' recommendations AHRQ is planning the following:

  • Sessions at AHRQ Annual Meeting: Delivery System Research: Discovering Ways to Improve Care Value (open to all attendees); Research Updates and Challenges (participation limited to ARRA Delivery System Grantees).
  • Electronic portal supporting communication among ARRA Delivery System grantees about research methods, common investigation topics, and related learning and funding opportunities.
  • Issue briefs describing how ARRA grantee research results contribute to emerging findings on key delivery system issues (e.g., reducing rehospitalization, enhancing patient-centered care, public reporting on physician performance).
Page last reviewed February 2017
Page originally created September 2012
Internet Citation: Delivery System Research Initiative. Content last reviewed February 2017. Agency for Healthcare Research and Quality, Rockville, MD.