Identifying Key Areas for Delivery System Research

Appendix D: Four Suggested Key Areas

Note: The table includes references to empirical articles or reviews of empirical articles and is intended to present examples rather than a comprehensive list of publications. If no references are given, I have not been able to find useful empirical articles for that topic. However, even when references are given, the data available on the topic area are in all cases far from sufficient. AHRQ's recent American Recovery and Reinvestment Act (ARRA) Comparative Effectiveness Research Delivery System Research funded proposals are listed in italics in the "Research Studies/ARRA Grants" column. The grantee's name links to a description of the research.


Research Area Research Studies/ARRA Grants
1. Analyses of the Demographics of the Delivery System and Relationships Among the Components of the Model
Delivery system demographics, for example:
Percentage of physicians in groups of various sizes and specialty types Cunningham R. Professionalism reconsidered: physician payment in a small-practice environment. Health Aff (Millwood) 2004;23(6):36-47.

Hing E, Cherry DK, Woodwell DA. National Ambulatory Medical Care Survey: 2004 summary. Adv Data 2006 Jun 23;374:1-33. Available at: (933 KB) Accessed January 9, 2014.
Percentage of physicians, by specialty, employed by hospitals Casalino LP, November EA, Berenson RA, et al. Hospital-physician relations: two tracks and the decline of the voluntary medical staff model. Health Aff (Millwood) 2008;27(5):1305-14.
Percentage of physicians in patient-centered medical home practices Rittenhouse DR, Casalino LP, Gillies RR, et al. Measuring the medical home infrastructure in large medical groups. Health Aff (Millwood) 2008;27(5):1246-58.
Percentage of physicians in organizations that could function as accountable care organizations  
Number of integrated delivery systems and their characteristics  
Gold standard database of U.S. physician organizations  
Change over time in delivery system demographics, especially in relation to changes in external incentives, for example:
Is the percentage of physicians employed by hospitals changing? If so, why? Casalino LP, November EA, Berenson RA, et al. Hospital-physician relations: two tracks and the decline of the voluntary medical staff model. Health Aff (Millwood) 2008;27(5):1305-14.

Isaacs SL, Jellinek PS, Ray WL. The independent physician—going, going… New Engl J Med 2009;360(7):655-7.
Are physicians more likely to be employed by hospitals (and/or by large medical groups) in areas where pay for performance (P4P) is prevalent?  
Structure-process-outcome relationships among components of the conceptual model for different forms of organization, for example:
Which types of medical groups perform better? Solberg LI, Asche SE, Pawlson LG, et al. Practice systems are associated with high-quality care for diabetes. Am J Manag Care 2008;14(2):85-92.

ARRA Grant: Swigonski
Which types of organizations perform better: independent practice associations (IPAs) vs. medical groups of various sizes vs. integrated delivery systems vs. accountable care organizations? Rittenhouse DR, Casalino LP, Gillies RR, et al. Measuring the medical home infrastructure in large medical groups. Health Aff (Millwood) 2008;27(5):1246-58.

Friedberg MW, Coltin KL, Pearson SD, et al. Does affiliation of physician groups with one another produce higher quality primary care? J Gen Intern Med 2007;22(10):1385-92.

Mehrotra A, Epstein AM, Rosenthal MB. Do self-identified integrated medical groups provide higher quality medical care than self-identified IPAs? Ann Intern Med 2006;145(11):826-33.

Casalino L, Gillies RR, Shortell SM, et al. External incentives, information technology, and organized processes to improve health care quality for patients with chronic diseases. JAMA 2003;289(4):434-41.

Kerr EA, Gerzoff RB, Krein SL, et al. Diabetes quality care in the Veterans Affairs health care system and commercial managed care: the TRIAD study. Ann Intern Med 2004;141(4):316-8.

ARRA Grant: Holtrop
What is the relationship among structure-process-outcomes? Friedberg MW, Coltin KL, Safran DG, et al. Association between structural capabilities of primary care practices and performance on selected quality measures. Ann Intern Med 2009;151(7):456-63.

Hearld LR, Alexander JA, Fraser I, et al. Review: how do hospital organizational structure and processes affect quality of care? A critical review of research methods. Med Care Res Rev 2008;65(3):259-99.

Stolzmann KL, Meterko M, Shwartz M, et al. Accounting for variation in technical quality and patient satisfaction: the contribution of patient, provider, team, and medical center. Med Care 2010;48(8):678-82.

Landon BE, Normand SLT, Meara E, et al. The relationship between medical practice characteristics and quality of care for cardiovascular disease. Med Care Res Rev 2008;65(2):167-86.

Damberg CL, Shortell SM, Raube K, et al. Relationship between quality improvement processes and clinical performance. Am J Manag Care 2010;16(8):601-6.

ARRA Grant: Swigonski
Do organizations that have more external incentives to improve performance use more processes to do so, and do they actually perform better? Casalino L, Gillies RR, Shortell SM, et al. External incentives, information technology, and organized processes to improve health care quality for patients with chronic diseases. JAMA 2003;289(4):434-41.

Shortell SM, Gillies R, Siddique J, et al. Improving chronic illness care: a longitudinal study of large physician organizations. Med Care 2009;47(9):932-9.

Conrad DA, Perry L. Quality-based financial incentives in health care: can we improve quality by paying for it? Annu Rev Public Health 2009;30:357-71.

ARRA Grant: Dowd (only for individual physicians)
Bring theoretical concepts, research methods, and substantive findings from fields outside health service research to the study of the delivery system, for example, knowledge that has been gained in other fields about organizational culture, about leadership, and about change within organizations.
Change within organizations ARRA Grant: Rodriguez
2. How Can Incentives and Public and Private Policies Be Structured To Induce Change in the Demography of Delivery System Organizations (Toward the Types of Organizations That Research Indicates Provide Better Care) and To Induce These Organizations To Continually Try To Improve the Value of the Care They Provide?
Compare the effects of different payment methods—not only on the quality and costs of care, but also on the demography of the delivery system and on the extent of organizations' efforts to improve care, for example: ARRA Grant: Fischer
  • Capitation plus public reporting and/or quality bonuses
  • Diagnosis related groups plus "shared savings" plus quality bonuses
  • Bundled payment for services that are primarily inpatient
Committee on Research. Bundled payment: AHA research synthesis report. Chicago, IL: American Hospital Association; 2010.

ARRA Grant: Williams
  • Bundled payment for services that are primarily outpatient
Determine whether it is feasible and desirable to provide incentives at the individual physician level, or whether these incentives should be given at the level of the provider organization
Compare the effectiveness of P4P vs. public reporting vs. improvement collaboratives without P4P or public reporting vs. usual care Lindenauer PK, Remus D, Roman S, et al. Public reporting and pay for performance in hospital quality improvement. New Engl J Med 2007;356(5):486-96.

ARRA Grant: Dowd; Malouin
Include inquiry into unintended consequences of incentives, for example, do P4P and/or public reporting lead to:
Increased resource disparities between hospitals and medical groups in rich and poor areas Werner RM, Goldman LE, Dudley RA. Comparison of change in quality of care between safety-net and non-safety-net hospitals. JAMA 2008;299(18):2180-7.

Blustein J, Borden WB, Valentine M. Hospital performance, the local economy, and the local workforce: findings from a U.S. national longitudinal study. PLoS Medicine 2010 June;7(6): e1000297. Published online 2010 June 29. doi: 10.1371/journal.pmed.1000297.

Doran T, Fullwood C, Kontonpantelis E, et al. Effect of financial incentives on inequalities in the delivery of primary clinical care in England: analysis of clinical activity indicators for the quality and outcomes framework. Lancet 2008;372(9640):728-36.
"Crowding out" of important unmeasured quality by measured quality Campbell SM, Reeves D, Kontonpantelis E, et al. Effects of pay for performance on the quality of primary care in England. New Engl J Med 2009;361(4):368-78.
Avoiding of high-risk patients by provider organizations Werner RM, Asch DA, Polsky D. Racial profiling: the unintended consequences of coronary artery bypass graft report cards. Circulation 2005;111(10):1257-63.

Doran T, Fullwood C, Reeves D, et al. Exclusion of patients from pay-for-performance targets by English physicians. New Engl J Med 2008;359(3):274-84.
Possibly undesirable changes in the demography of provider organizations (e.g., disappearance of small practices)  
Seek to learn more about the effects of regulations (e.g., antitrust enforcement against physicians, hospitals, and health plans) on structure, process, and outcomes in the delivery system
What are the effects of the Federal Trade Commission antitrust guidelines for physicians on the organization of physician practice and on physician-hospital relations?  
3. Measurement of Performance
Careful thinking and research about what measures of important areas of value (quality and cost) can effectively be used, given considerations of:
  • Statistical reliability
Adams JL, Mehrotra A, Thomas JW, et al. Physician cost profiling - reliability and risk of misclassification. New Engl J Med 2010;362:1014-21.

Nyweide DJ, Weeks WB, Gottlieb DJ, et al. Relationship of primary care physician's patient caseload with measurement of quality and cost performance. JAMA 2009;302(22):2444-50.
  • Ability to change the delivery system to high-performing organizations that focus continually on improving value (not just on improving scores on a limited number of process measures)
ARRA GRANT: Swigonski
  • Possible unintended consequences
Research into using patient experience, including but not limited to patient-reported outcomes, to gain a more complete estimate of the value provided by an organization than can be gained by process and outcome measures A great deal of research funding is being devoted to this area at present.
Research into how electronic medical records can be used to better measure quality  
4. Analyses of Interprovider/Interorganizational Processes for Improving Care
Improving transitions in care, not just from inpatient to outpatient, or between nursing home and hospital, but also from outpatient to inpatient and referrals from physician to physician ARRA GRANT: Magill
Processes aimed at reducing readmissions Mor V, Besdine RW. Policy options to improve discharge planning and reduce rehospitalization. JAMA 2011;305(3):302-3.
Resource sharing to support implementation of value-improving processes among small practices  
Page last reviewed February 2014
Internet Citation: Appendix D: Four Suggested Key Areas. February 2014. Agency for Healthcare Research and Quality, Rockville, MD.