3. Breadth and Focus Areas of Federal Agency Research Portfolios in HSR and PCR
This chapter addresses the study’s first key question: What is the breadth and focus of federally funded HSR and PCR? There are two main components to this chapter. First, we present findings on the HSR and PCR portfolios and focus research areas of the federal agencies in the study’s scope, based on expert information from the TEPs and stakeholder interviews, as well as documentary sources. Second, we report results from the environmental scan and portfolio analysis, which enumerates the extramurally funded HSR and PCR projects of agencies by topic according to the study’s research domain framework (Figure 2.1 in the previous chapter).
Agency HSR/PCR Portfolios and Research Focus Areas
The breadth and scope of federally funded HSR and PCR are a function of the types of research sponsored by different federal agencies, which in turn are shaped by the congressional statutory authorizations and mission of each respective agency. Research is part of the core mission of three of the agencies we studied: AHRQ, ASPE, and NIH. Other agencies within the study’s scope generate HSR and PCR through research activities in support of their primary mission (e.g., CMS evaluates demonstrations to improve quality and cost of care for Medicare and Medicaid beneficiaries, and the VHA conducts research to improve delivery of care for military veterans). Over time, HHS agencies and the VHA have developed portfolios of HSR and PCR around particular focus areas that address the requirements of their individual missions and operational needs.
Distinct Characteristics of Agency HSR and PCR Portfolios
TEP and interview participants and federal agency points of contact identified eight agencies in the scope of the study with portfolios of research in HSR and PCR according to the definitions provided in Chapter 2: ACL, AHRQ, ASPE, CDC, CMS, HRSA, NIH, and VHA. Table 3.1 summarizes the missions, key characteristics differentiating research portfolios—namely, scope of the health care system examined, research objectives, main research audiences—and particular focus topics in HSR and PCR of each agency.
Table 3.1. Agency Missions, HSR and PCR Portfolio Characteristics, and Research Focus Topics
| Agency Mission | Scope of Health Care System Examined | Research Objectives | Main Research Audiences | HSR Focus Topics | PCR Focus Topics |
|---|---|---|---|---|---|
ACL
|
Services: Health care and supportive services Population: The elderly and disabled individuals |
|
|
|
|
AHRQ
|
Services: All services across the health care system Population: All individuals in the U.S. health care system |
|
|
|
|
ASPE
|
Services: Related to specific federal health programs Population: Individuals served or affected by specific federal health programs and initiatives |
|
|
|
|
CDC
|
Services: Prevention and health promotion, in community and health care settings Population: General public, including both ill and healthy individuals |
|
|
|
|
CMS
|
Services: Health care services paid for by CMS programs Population: Beneficiaries covered by Medicare, Medicaid, and other CMS programs |
|
|
|
|
HRSA
|
Services: Safety net systems Population: Individuals who are uninsured, isolated, or medically vulnerable |
|
|
|
|
NIH
|
Services: Clinical and other health-related services across the health care system Population: Individuals having or at risk for illness or disability, including specific populations (children, aged, and minorities) |
|
|
|
|
VHA
|
Services: All health care services in VHA system Population: U.S. military veterans |
|
|
|
|
We first provide an overview of the distinct characteristics of the HSR and PCR portfolios of these agencies. Later we describe in more detail the individual research portfolios of each agency and also indicate which agencies have no or negligible HSR or PCR in their research portfolios as defined by the study. Detailed descriptions of agency HSR and PCR portfolios are provided in Appendix C.
The particular focus, strengths, and expertise of federal agency HSR and PCR portfolios tend to be differentiated by three dimensions:
- The scope of the health care system studied (i.e., whether a specific health care setting or population, a broader range of health care settings or populations, or the intersection of health care with public health and community settings).
- The research objectives (i.e., whether focused on system processes and outcomes and/or disease-specific care and outcomes).
- The main audience(s) for the research.
Study participants across the range of federal and non-federal stakeholders noted that AHRQ, as the only agency authorized to generate HSR with a mission to do so across the U.S. healthcare system, has a unique focus within its research portfolio on systems-based outcomes and approaches to implementing improvements across health care settings and populations in the United States. It is also the agency authorized to serve as the home for federal PCR, and the only agency whose research primarily targets health care delivery leaders, as well as providers and researchers. As the leader of a state-level payer organization commented:
[T]o my mind, AHRQ is the driving entity looking at do we have the best possible understanding of the major processes, delivery systems, tools to make care meet those IOM six dimensions [safe, timely, effective, efficient, equitable and patient-centered], and are we comparing different tactics and strategies fairly and well? . . . that’s right at the center of AHRQ’s swim lane.
NIH similarly seeks to enhance the health of persons generally throughout the United States; its portfolio of HSR and PCR plays a fundamental role in funding research on the effectiveness and implementation of care practices and interventions typically focused on specific diseases, body systems, or populations, according to the individual missions of the agency’s component Institutes and Centers (ICs). The main audiences for this research tend to be the research community and health care providers.
A health policy interview participant described the important role of the type of HSR and PCR funded by NIH and how that compares to research on broader systems-level outcomes:
NIH funds health services research that is very specific to body part, disease endpoints and outcomes. . . . And so it’s a subset of health services research. . . . It’s absolutely important. But it is not the broad sort of system-level performance outcomes . . . in terms of the cost, quality, safety outcomes, endpoints.
A federal interview participant from NIH noted how those broader system outcomes, “because they’re not necessarily specific to a given disease or organ or population, have been a really natural place for AHRQ that NIH can’t do as much with.” A researcher interview participant also emphasized the conceptual contrast of a systems perspective:
AHRQ is more systems-focused looking at how services are delivered, who delivers it, from the workforce perspective, from the administrative organizational perspective. . . . How can we change health care systems to impact patient outcomes? . . . Of course, patients are at the center of AHRQ’s research projects. . . . But the way AHRQ conceptualizes it is different than the way the NIH conceptualize, in terms of looking from the system perspective.
The HSR and PCR portfolios of other agencies differ by scope of the health care system addressed and the main audiences for the research. CDC’s research, like that of NIH, is largely organized by diseases and conditions, but its HSR and PCR portfolios focus on broad population health outcomes measurement, surveillance, and analysis; and the linkages between prevention, population health, and the health care system. The main audiences for its research are public health officials and the broader public and communities. A federal interview participant from CDC explained how “our population is quite a bit broader than . . . even that of AHRQ. It’s not just those who cross the threshold of the hospital. It’s those who are healthy, as well as those who actually have a disease.” This interview participant also noted that there is an intersection between the health care and public health systems, which has increased over the past few decades:
The programs that we [CDC] are trying to influence are those that affect a broader public health audience. Sometimes clearly those audiences overlap with the health care system, because . . . the line between health care and public health or population health and prevention are beginning to blur a bit more, than they were two decades ago.
In contrast, ASPE, CMS, and the VHA are more narrowly focused on specific federal health policies and delivery systems, with their research targeted to informing policy and decisionmakers for these programs. ASPE’s HSR and PCR portfolios focus on health care policy priorities of HHS leadership and Congress, as well as requests from HHS operating divisions that have limited research capacity or that require unique research expertise and resources possessed by ASPE. A federal interview participant from ASPE explained that, while HSR and PCR from other agencies may inform policy, ASPE specifically operates at the “nexus” of research that supports policymakers in the time frame needed for decisionmaking:
[ASPE’s] work is really designed to inform policymakers. We work at the nexus of research and policy. AHRQ doesn’t work at that same nexus. It doesn’t mean their work may not inform policymaking. . . . I’d [also] say a lot of our timetables are often shorter than theirs might be, because we’re trying to inform real-time decision-making.
CMS generates HSR and PCR through evaluation to inform efforts to improve care and reduce costs for its beneficiaries, particularly those efforts related to quality measurement and payment and delivery innovations for populations and services covered by Medicare, Medicaid, and other CMS programs. As noted by a state policy interview participant, a major portion of this portfolio consists of the agency’s CMMI funding of “program evaluation . . . of their payment [demonstrations], their payment models, and by extension, CMS doing [evaluations of] like a Medicaid waiver, their health homes waiver.” The VHA conducts HSR and PCR intended to improve the health care and health of military veterans. A federal interview participant from the VHA explained that “within the field of HSR, we have priorities that are dictated by the particular challenges of veterans or the VA Health System,” such as care for pain, opioids, suicide, and aging, and priorities of the VHA system to address access to care and coordination with other delivery systems. Given the size and scope of the health policies and delivery systems for which these agencies are responsible, their research and evaluation activities typically contribute unique results and data applicable more generally to the fields of HSR and PCR.
ACL and HRSA sponsor HSR and PCR as part of their broader missions as funders of services for underserved and special needs populations. ACL’s HSR and PCR portfolio focuses on improving care to support community-living elderly and disabled individuals. HRSA sponsors HSR and PCR to improve services for safety net and other vulnerable populations, such as those who receive care at federally qualified health centers (FQHCs) supported by the agency’s Health Center program. HRSA also serves as a distinct HSR and PCR resource providing analysis and data on the distribution and development of the U.S. health care workforce.
Across agencies, interview stakeholders observed that the time horizons for the research tended to differ by type of audience—shorter term for policy and decisionmakers, medium term for health care delivery leaders and providers, and longer term for research audiences.
Research Portfolios and Focus Areas by Agency
Next, we describe the individual HSR and PCR portfolios of each agency, including the specific topical focus areas that agencies have developed to address their individual missions and operational needs.1
Administration for Community Living
Improving care to support health and wellness of community-living elderly and disabled individuals. As indicated in Table 3.1, ACL serves as the federal agency responsible for increasing access to community supports, while focusing attention and resources on the unique needs of older Americans and people with disabilities across the life span (Administration for Community Living, 2019a). Most of the ACL’s extramural funding consists of service grants that do not include research as defined by this study. However, the agency’s NIDILRR institute operates several programs that fund research projects with health care–related components (Administration for Community Living, 2019b). This portfolio includes Model Systems Programs that fund centers of excellence to provide and evaluate comprehensive, evidence-based models of care and support for individuals in communities who have spinal cord, burn, or traumatic brain injuries (Model Systems Knowledge Translation Center, undated). The NIDILRR portfolio also contains extramural funding of research on the implementation and effects of health policies on community living of people with disabilities of all ages, rehabilitation research centers that include studies of health services and financing, and grants on health technology that include research on telehealth to support people with disabilities living in community.
1 Detailed descriptions of agency HSR and PCR portfolios are provided in Appendix C.
Agency for Healthcare Research and Quality
Improving health care systems and outcomes across health care services and populations in the U.S. health care system. AHRQ is the only federal agency that has a congressional authorization to generate HSR with a mission to do so across the U.S. healthcare system, including research to make health care safer, higher quality, more accessible, equitable, and affordable (42 CFR 67.13). AHRQ also has a statutory charge to serve as the lead federal agency for primary care research (42 U.S.C. 299 et seq). Although AHRQ has not received targeted appropriations for this latter mission, the agency funds and disseminates research on primary care systems and innovations, including the nature of primary care as the usual source for addressing personal health care needs, the management of commonly occurring and undifferentiated clinical problems, and the continuity and coordination of health services. The agency also hosts the National Center of Excellence for Primary Care Research, which provides evidence, practical tools, and other resources to improve primary care (Agency for Healthcare Research and Quality, undated-i).
Interview and TEP participants noted that a hallmark of AHRQ-sponsored research is its focus on systems-based improvement across the spectrum of health care settings and populations in the United States. AHRQ’s research is organized around broad health care system inputs and outcomes, including quality of care, with an emphasis on patient safety; access and disparities; organization of delivery systems and markets; financing and costs; and HIT adoption and effective use. Patient safety became a particular emphasis after Congress directed AHRQ in 2001 to invest significant resources in this area (Senate Report 106-293, 2002), which has included research on the causes of and effective strategies to reduce medical errors and harms, such as healthcare-associated infections, adverse drug events, and preventable hospital readmissions (Agency for Healthcare Research and Quality, 2018). Interview and TEP participants further commented on AHRQ’s special emphasis on implementation tools, methods, and evaluation—particularly for understanding the implementation and wider scale-up of improvement practices—as well as its unique role in synthesizing, disseminating, and translating scientific evidence from HSR and PCR into practice.
In addition, AHRQ produces and maintains several large databases covering the breadth of the U.S. health care system for use by health services and primary care researchers. These databases include the Healthcare Cost and Utilization Project (Healthcare Cost and Utilization Project, 2019), which contains the largest collection of longitudinal encounter-level hospital care data; and the Medical Expenditure Panel, the most complete source of data on the cost and use of health care and health insurance coverage in the United States (Agency for Healthcare Research and Quality, undated-e).
Office of the Assistant Secretary for Planning and Evaluation
Informing policy and decisionmaking on federal health care priorities. ASPE advises the Secretary of HHS on policy development, coordinates the department’s evaluation, research, and demonstration activities, and manages cross-department planning (Assistant Secretary for Planning and Evaluation, undated). ASPE research and evaluation are intended to inform policies related to specific federal programs and initiatives at the department or operating division levels within HHS (U.S. Department of Health and Human Services, 2019). Its HSR and PCR portfolios focus on priorities of HHS leadership and Congress, requests from HHS operating divisions that have limited research capacity or require unique research expertise and resources possessed by ASPE, and self-initiated projects, including both short-turnaround and forward-thinking projects to address anticipated policy needs. Over time, ASPE has developed HSR portfolios on federal programs and policies related to access to care, health insurance coverage, costs and spending (including value-based care), behavioral health, and care for special populations served by federal programs. Its portfolios in PCR have focused on access to primary care, payment policy changes for primary care services in Medicare and Medicaid, and primary care improvement and transformation in the IHS system.
Centers for Disease Control and Prevention
Protecting public health from threats posed by specific diseases, conditions, and injuries. The mission of CDC is to protect the health of Americans by fighting disease and supporting communities and citizens in health promotion. As described above, CDC has a broad public and population health focus that includes all people in communities—healthy as well as sick individuals, and whether or not they are engaged in the health care system. To fight disease and support health promotion, CDC largely organizes its research around disease states and health conditions, including both those that are communicable and noncommunicable.
To accomplish this objective, CDC conducts critical science and provides health information to understand and address threats to public health. Although much of this research falls outside the scope of this study, federal and other stakeholders noted several areas in which the agency’s research on population health outcomes measurement, surveillance and analysis, and the evaluation of prevention services intersect with the health care system and HSR and PCR topics.
Its HSR-specific portfolio has concentrations on the use of CDC surveillance systems and health outcomes measurement to assess health care quality. For example, CDC’s National Healthcare Safety Network (NHSN) provides data on healthcare–associated infections (HAIs) in hospitals and other health care settings used in research on HAI-related patient safety improvement, as well as for CMS quality reporting programs. The agency’s PCR focuses on prevention services in primary care, including immunizations and various screening and health promotion activities, and the linkages between primary care and broader public health and community prevention resources.
Centers for Medicare and Medicaid Services
Improving health care access, quality, cost, and population health for Medicare, Medicaid, and other CMS beneficiaries. CMS generates HSR and PCR through the evaluation work it sponsors in support of its business objectives to improve care and reduce costs for Medicare, Medicaid, and other CMS programs. Thus, its HSR and PCR activities are focused on the populations and services covered by its programs (e.g., medical services for elderly in Medicare, medical, and long-term care services for low-income individuals in Medicaid) and on informing policy- and decisionmaking for the agency. The first portfolio of HSR within CMS focuses on the development of quality measures, particularly for hospital and other health care facilities, and evaluation of the impact of reporting quality measures for CMS programs on health care systems and providers. The agency’s second and larger portfolio consists of evaluations of demonstrations funded by CMMI of new payment and delivery models. Within CMMI-sponsored research, PCR-related evaluations have also focused on models promoting primary care transformation (e.g., patient-centered medical homes [PCMHs] in multipayer and safety-net practices), and on the role of primary care in new payment and delivery models (e.g., Accountable Care Organizations [ACOs]).
Health Resources and Services Administration
Improving services for safety net and other vulnerable populations and analyzing the distribution and development of the U.S. health care workforce. HRSA’s mission is to improve health and achieve health equity through access to quality services, a skilled health workforce and innovative programs (Health Resources and Services Administration, 2019). Study participants noted HRSA for its support of services for safety net and other vulnerable populations, as well as health care workforce development. Most of HRSA’s extramural funding provides grants for direct services and service infrastructure that do not include research. However, HRSA has long-established programs of research and evaluation related to its main functional areas, which the agency has more recently expanded in part to inform decisions about costs, interventions, and quality of care for the populations it serves (Dievler and Fisher, 2017).
The agency’s Bureau of Primary Health Care collects extensive data on the community health centers it funds nationwide for underserved populations, which it makes available to the wider research community, as well as contracts externally for analyses. Other HRSA research for specific populations and services include the Maternal and Child Health Bureau’s intramural research on issues such as health care, preventive and early intervention services for maternal and child populations; the Office of Rural Health’s funding of extramural Rural Health Research Centers; and the Ryan White HIV/AIDS Program, which conducts research on health care disparities, services, and innovative models of care for under- and uninsured individuals served by its funded HIV centers. More broadly, the National Center for Health Workforce Analysis (NCHWA) serves as a national resource for projections of health care workforce supply and demand, and analysis of the distribution and education of the nation’s health workforce (Health Resources and Services Administration, undated-b). As with other HRSA programs, NCHWA provides extensive data and information for use by the wider research and policy community.
National Institutes of Health
Enhancing health of the nation with a focus on specific diseases, body systems, and populations. NIH’s mission is to seek fundamental knowledge about the nature and behavior of living systems and the application of that knowledge to enhance health, lengthen life, and reduce illness and disability. Although the vast proportion of NIH’s research portfolio is focused on basic, clinical, and translational research, federal and research interview participants and TEP members noted that NIH ICs fund a considerable amount of HSR and PCR. These portfolios of research play a fundamental role in funding HSR and PCR on the effectiveness and implementation of care practices and interventions typically focused on specific diseases, body systems, or populations, according to the individual missions of the agency’s component ICs.2 PCR interview and TEP participants reported that NIH has been a significant source of funding and sponsor of important studies in the PCR field. They commented that ICs organized to address specific diseases or body systems tend to focus on screening, managing, and coordinating care for those diseases or body systems by primary care providers. ICs organized to address particular populations were also noted to sponsor research related to broader influences on population health, such as the effect of insurance payment and coverage of care.
Federal and research interview participants observed that some ICs within NIH have a stronger emphasis on HSR and PCR than others based on their particular congressional mandates and other directives, as well as concerns of their extramural research communities, such as to increase the adoption of evidence-based clinical interventions by practitioners. ICs identified by study participants as sponsoring significant amounts of HSR and PCR include the National Institute of Mental Health (NIMH), National Institute on Drug Abuse (NIDA), National Institute on Alcohol Abuse and Alcoholism (NIAAA), National Cancer Institute (NCI), National Heart, Lung, and Blood Institute (NHLBI), and National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK). Other ICs identified by study participants to fund HSR and/or PCR include the National Institute on Aging (NIA), the National Institute on Child Health and Human Development (NICHD), the National Institute on Minority Health and Health Disparities (NIMHD), the National Institute on Nursing Research (NINR), and the National Center for Advancing Translational Sciences (NCATS).
In addition, federal interview participants noted that certain trans-NIH initiatives funded through the agency’s Common Fund have provided opportunities for its research to more broadly engage health care delivery systems. These initiatives include the Common Funds’ Health Economics program from 2001 to 2017 and the Health Care System Research Collaboratory begun in 2012, which aims to enhance the capacity to conduct research with health care systems and strengthen the relevance of research results to health practice. ICs now also directly support additional pragmatic clinical trials in the Collaboratory.
2 NIH also contains Institutes, Centers, and Offices that focus on professions (NINR), treatment modalities (NCCIH), and research areas (Office of Behavioral and Social Science Research).
Veterans Health Administration
Improving the VHA health care delivery system and veterans’ health. The VHA conducts HSR intended to improve the health care and health of veterans, and to advance the field based on the unique capabilities of the VHA system. The VHA’s HSR portfolio is focused on the particular health challenges facing veteran populations and the priorities of the VHA system. These include improving the quality and safety of specific services that are significant for veterans, such as pain management, suicide prevention and other types of mental and behavioral health care, trauma and traumatic brain injury, and health care for women—the fastest-growing segment of the veteran population. To address specific challenges in the VHA system, its research portfolio has also focused on broader HSR issues, such as ways to define and improve access to care, expanded use of telehealth, rural health care, and coordination of care with other health delivery systems outside of the VHA. Major topics in the VHA’s PCR portfolio, in addition to access and telehealth, have included management of multimorbidities for complex, high-risk patients; integrated primary and behavioral health care; and evaluation of a national rollout of medical home transformation across the VHA system.
Federal and research stakeholders additionally commented on the VHA’s strength in certain methods related to its role as the largest nationally integrated health system in the country, including development of systemwide, standardized quality and safety measures, and the application of 25 years of comprehensive, longitudinal electronic health record (EHR) data to perform unique types of HSR analyses, as well as to understand the use of EHR data systems to measure and improve care. Other methods areas for HSR within the VHA include implementation methods and evaluation for translating evidence-based practice into veterans’ care, and best practices for embedding researchers with clinical and operational partners in health services.
Agencies Without Reported HSR or PCR Portfolios
We confirmed with representatives of the following agencies that their operating division or office funds no or negligible amounts of HSR or PCR according to the definitions of this study: ATSDR, FDA, ONC, and SAMHSA. Federal stakeholders reported that ONC works closely with AHRQ and other agencies, including the National Library of Medicine in NIH on research related to the implementation and use of HIT. SAMHSA was reported to rely on evidence of effective interventions for mental health and substance abuse from NIH institutes and other agencies, and engage with ASPE for research needs related to SAMHSA’s programs.
We were not able to confirm with representatives of ACF or IHS whether their agencies support any research portfolios in HSR or PCR. Other federal stakeholders noted that ACF’s extramural grant funding is mostly dedicated to supporting service provision and infrastructure, but as described for ACL and HRSA above, this does not preclude funding of HSR or PCR. Although IHS has a focus on quality improvement for the Indian health care system it supports, the agency was described as having relatively limited capacity to conduct research per se and tending to engage with ASPE for HSR or PCR needs related to Indian health care programs.
Environmental Scan of Federally Funded Extramural HSR and PCR
Next, we present results of our systematic scan of extramurally funded HSR and PCR portfolios from the six agencies for which the study was able to collect comprehensive project-level data and review results with agency contacts. As discussed in Chapter 2, there were two primary sources of these data—the RePORTER database, which includes federal grants for health-related research across HHS and the VHA; and agency-provided data, which include both contract and grant projects. We report the number of projects collected and in-scope between FY 2012 and FY 2018, as well as the percentage of an agency’s projects by research classification in order to emphasize focus areas in each agency’s portfolio.3
Table 3.2 shows the total number of extramurally funded projects collected for each agency (excluding those missing abstracts or coded in RePORTER as related to research infrastructure), and the proportion of projects identified as in-scope according the report’s definitions of three classifications: HSR—the scientific investigation of health care services broadly defined— including health, mental health, substance abuse, long-term care, and social and other services, as they connect to health care; PCR—scientific investigation of primary care, which includes projects that are HSR as well as those that are not; and MTD—which includes development of research methods (e.g., psychometric instruments, quality measures, risk adjustment, and stratification models) and other tools (e.g., HIT applications, implementation toolkits) related to the study of health care.
Many, though not all, projects with MTD components are also classified as HSR or PCR research (i.e., investigations examining a research question intended to contribute to scientific knowledge on an HSR or PCR topic). Thus, all three in-scope classifications are not mutually exclusive, and the percentages reported in each category do not sum to 100. Go to Chapter 2 for research domain definitions and Appendix B for detailed category coding specifications.
In Table 3.2, the total number of projects for ACL and CMS consists only of projects that were directly provided by these agencies in response to RAND’s request for HSR, PCR, and MTD projects as defined by the study (CMS extramurally funded projects are exclusively contracts, which are not included in the RePORTER database, and ACL relies on a different database to archive its extramural projects). Unsurprisingly, relatively large proportions of the projects collected from these agencies were in-scope to the study—CMS (99 percent) and ACL (85 percent)—which reflect the selective identification of projects by the agencies for the study as well as the nature of the research and evaluation work they fund, especially CMS.
3 Go to Appendix B for confidence intervals for the results, and results by year.
Table 3.2. HSR, PCR, and Methods and Tool Development Projects, by Funding Agency
| ACL | AHRQ | CDC | CMS | NIH | VHA | All Agenciesa | |
|---|---|---|---|---|---|---|---|
| Total projects in scan data setb | 68 | 1,155 | 1,093 | 75 | 86,321 | 3,013 | 93,075 |
| Projects classified as HSR | 53 (78%) | 653 (57%) | 267 (24%) | 57 (76%) | 6,891 (8%) | 794 (26%) | 8,767 (9%) |
| Projects classified as PCR | 2 (3%) | 149 (13%) | 28 (3%) | 9 (12%) | 750 (1%) | 150 (5%) | 1,090 (1%) |
| Projects classified as HSR and/or PCR | 53 (78%) | 668 (58%) | 268 (25%) | 58 (77%) | 6,948 (8%) | 797 (26%) | 8,845 (9%) |
| Projects classified as MTD | 26 (38%) | 432 (37%) | 104 (10%) | 17 (23%) | 3,501 (4%) | 310 (10%) | 4,416 (5%) |
| Projects classified in-scope (HSR, PCR, and/or MTD) | 58 (85%) | 915 (79%) | 328 (30%) | 74 (99%) | 8,707 (10%) | 889 (30%) | 11,045 (12%) |
Note: Percentages do not sum to 100, as projects can meet criteria for multiple or no categories. These results are based on a combination of manual reviews and machine learning–based automated classification of projects in the scan data set. For additional details, go to Chapter 2 and Appendixes A and B.
a The “All Agencies” tallies include projects from the six agencies in the table as well as 46 FDA and ACF projects from the RePORTER database that our machine learning algorithm classified as in-scope to the scan. FDA and ACF projects are not reported separately since we did not perform member-checking of those results with the agencies as required by the scan methodology.
b The “Total Projects” numbers are based on the scan data set, which includes data from the RePORTER database and directly from individual agencies, following merging of nonunique project records and the exclusion of projects that met any of the following criteria: (a) projects with pre-FY 2012 start dates; (b) projects with post-FY 2018 start dates; (c) projects missing abstracts; (d) projects with RePORTER activity codes related to research infrastructure and support; and (e) projects conducted intramurally by agency staff. Go to Chapter 2 for additional details.
The total number of projects for AHRQ, CDC, and VHA consists of both RePORTER and agency-provided projects (with duplicates removed between the two sources).4 NIH projects are all contained and sourced from the RePORTER database. Of these four agencies, AHRQ has the highest proportion of in-scope projects across all three classifications (79 percent). NIH has the smallest proportion of in-scope projects (10 percent) but contributes the largest number (8,707).
Table 3.3 displays the number and percentage of agency projects by the main research domains of health care outputs and inputs in the framework presented in Chapter 2. The denominator of the percentages is the total number of projects identified as HSR and/or PCR, shown in the top row of the table.5 A single project often was counted as meeting the classification criteria for multiple domains; therefore, the domain percentages do not sum to 100. The first four domains in Table 3.3 are the output domains (Quality of Care, Cost and Utilization, Access to Care, and Equity).
4. Further details on the source and extent of data available for each of the funding agencies can be found in Tables A.1 and A.2 in Appendix A.
5. MTD-only projects were not coded for the research domains.
Table 3.3. HSR and/or PCR Projects, by Research Domain and Funding Agency
| Research Domain | ACL | AHRQ | CDC | CMS | NIH | VHA | All Agenciesa |
|---|---|---|---|---|---|---|---|
| Total HSR and/or PCR projects | 53 | 668 | 268 | 58 | 6,948 | 797 | 8,845 |
| Quality of Care | 87% | 80% | 80% | 90% | 89% | 87% | 88% |
| Cost and Utilization | 17% | 39% | 37% | 88% | 27% | 24% | 28% |
| Access to Care | 34% | 13% | 36% | 19% | 26% | 39% | 26% |
| Equity | 15% | 14% | 35% | 10% | 33% | 16% | 30% |
| Organization of Care | 85% | 90% | 87% | 74% | 86% | 93% | 87% |
| Financing of Care | 15% | 13% | 4% | 66% | 2% | 3% | 4% |
| Social Factors | 40% | 16% | 28% | 2% | 29% | 18% | 27% |
| Personal Preferences and Behaviors | 62% | 26% | 56% | 14% | 76% | 60% | 69% |
Note: Percentages do not sum to 100, as projects can meet criteria for multiple or no categories. These results are based on a combination of manual reviews and machine learning–based automated classification of projects in the scan data set. For additional details and definitions of research domains and categories, go to Chapter 2 and Appendixes A and B.
a The “All Agencies” tallies include projects from the six agencies in the table as well as FDA and ACF projects from the RePORTER database that our machine learning algorithm classified as HSR and/or PCR. FDA and ACF projects are not reported separately since we did not perform member-checking of those results with the agencies as required by the scan methodology.
In the first row, the HSR and PCR portfolios of all agencies have a strong emphasis on Quality of Care (from 80 percent to 90 percent). Quality of Care includes studies that measure process of care (e.g., the provision of a treatment as appropriate), as well as both intermediate (e.g., disease control, adverse patient safety events, patient experience) and definitive outcomes of care (e.g., quality of life, mortality), whether assessed at the level of a specific health care intervention, clinical service, or health care systems more broadly. As might be expected, CMS has the strongest emphasis on Cost and Utilization (90 percent), given its primary role as the health care payer for beneficiaries in its Medicare, Medicaid, and other programs. Agencies with the next strongest emphasis on Cost and Utilization are AHRQ (39 percent) and CDC (37 percent). VHA, CDC, and ACL have the strongest relative emphasis on Access to Care of these six agencies (39 percent, 36 percent, and 34 percent, respectively), which accords with the descriptions of their HSR and PCR portfolios. CDC and NIH have the strongest relative emphasis on Equity (35 percent and 33 percent, respectively).
The last four domains in Table 3.3 are the input domains (Organization of Care, Financing of Care, Social Factors, and Personal Preferences and Behaviors). The HSR and PCR portfolios of all agencies have a strong emphasis on Organization of Care. This is particularly the case for AHRQ (90 percent) and VHA (93 percent), but also NIH, ACL, and CDC (ranging from 85 percent to 87 percent). Similar to the Quality of Care domain, Organization of Care includes structures and routines of care spanning organizational levels from provider teams to clinical services and higher levels of health care systems. Based on the portfolio descriptions, it might be expected that some agencies, such as NIH, would emphasize the former, while others, such as AHRQ, would emphasize the latter; however, this is not differentiated in these data.
CMS has the highest proportion of HSR/PCR projects focused on Financing of Care (66 percent), similar to the results for the Cost and Utilization output domain above, followed by ACL (15 percent) and AHRQ (13 percent). Of note, Financing of Care has a much lower emphasis overall compared to the rest of the research domains (only 4 percent of the total HSR/PCR projects across all agencies). ACL has the highest proportion of HSR/PCR projects focused on Social Factors, which includes social, economic, and community determinants of health and health care. NIH (29 percent) and CDC (28 percent) follow on relative emphasis of Social Factors. NIH has the largest percentage of projects addressing Personal Preferences and Behaviors (76 percent), followed by ACL (62 percent), VHA (60 percent), and CDC (56 percent), with lower proportions for AHRQ (26 percent) and CMS (14 percent). The portfolios of these agencies may vary on the degree of emphasis they place on personal preferences (e.g., preference for particular care, shared care decisionmaking) versus personal behaviors (e.g., smoking, exercise, treatment adherence, other health-related behaviors).
Table 3.4 presents more-specific research categories of interest in HSR and PCR. Definitive health outcomes (e.g., mortality and quality of life) and patient safety (i.e., medical errors and harms produced by health care and their prevention) are subsets included in the Quality of Care domain.6 CMS, NIH, and VHA have the strongest emphasis on Definitive Health Outcomes (36 percent, 28 percent, and 38 percent, respectively). CDC’s HSR and PCR portfolio has a relatively lower emphasis on definitive health outcomes (15 percent). Despite the agency’s noted expertise in this area, most of CDC’s epidemiological and surveillance research on health outcomes without a link to health care services is out-of-scope for this study. This result also indicates that CDC’s health service–related research is less focused on definitive outcomes than on other process and intermediate health outcomes that fall under the Quality of Care domain more generally.
Table 3.4. HSR and/or PCR Projects, by Research Areas of Interest and Funding Agency
| Area of Interest | ACL | AHRQ | CDC | CMS | NIH | VHA | All Agenciesa |
|---|---|---|---|---|---|---|---|
| Total HSR and/or PCR projects | 53 | 668 | 268 | 58 | 6,948 | 797 | 8,845 |
| Definitive health outcomes | 21% | 19% | 15% | 36% | 28% | 38% | 28% |
| Aging | 21% | 22% | 15% | 76% | 18% | 18% | 19% |
| Patient safety | 6% | 40% | 29% | 7% | 12% | 19% | 16% |
| Pediatrics | 6% | 10% | 19% | 2% | 19% | 0% | 16% |
| Prevention | 9% | 37% | 82% | 17% | 71% | 38% | 65% |
Note: Percentages do not sum to 100, as projects can meet criteria for multiple or no categories. These results are based on a combination of manual reviews and machine learning–based automated classification of projects in the scan data set. For additional details and definitions of research domains and categories, go to Chapter 2 and Appendixes A and B.
a The “All Agencies” tallies include projects from the six agencies in the table as well as FDA and ACF projects from the RePORTER database that our machine learning algorithm classified as HSR and/or PCR. FDA and ACF projects are not reported separately since we did not perform member-checking of those results with the agencies as required by the scan methodology.
Table 3.5 presents results for MTD and methods of interest categories by funding agency.7 The denominator for these categories includes all in-scope projects, since MTD includes projects that were not coded to the other research domains.
7 Unlike the MTD categories (HIT Applications and Tools, Model Development, and Validation), the Methods of Interest categories (Evidence Review and Synthesis, and Simulation Modeling) were also coded for the other research domains in the framework.
Table 3.5. All In-Scope Projects, by Methods and Tool Development/Methods of Interest Categories and Funding Agency
| MTD and Methods of Interest Categories | ACL | AHRQ | CDC | CMS | NIH | VHA | All Agenciesa |
|---|---|---|---|---|---|---|---|
| Total projects classified in-scope | 58 | 915 | 328 | 74 | 8,707 | 889 | 11,045 |
| HIT applications and tools | 17% | 26% | 6% | 7% | 13% | 9% | 14% |
| Model development and validation | 29% | 23% | 16% | 23% | 24% | 22% | 23% |
| Toolkit development | 12% | 9% | 5% | 0% | 2% | 4% | 3% |
| Evidence review and synthesis | 9% | 13% | 2% | 4% | 4% | 1% | 5% |
| Simulation modeling | 0% | 2% | 3% | 0% | 4% | 2% | 4% |
Note: Percentages do not sum to 100, as projects can meet criteria for multiple or no categories. These results are based on a combination of manual reviews and machine learning–based automated classification of projects in the scan data set. For additional details and definitions of research domains and categories, go to Chapter 2 and Appendixes A and B.
a The “All Agencies” tallies include projects from the six agencies in the table as well as FDA and ACF projects from the RePORTER database that our machine learning algorithm classified as HSR and/or PCR. FDA and ACF projects are not reported separately since we did not perform member-checking of those results with the agencies as required by the scan methodology.
AHRQ has the strongest emphasis on HIT Applications and Tools (26 percent) in accordance with its specific program of research on implementation and use of HIT for health care improvement. Model Development and Validation appears to have similar levels of emphasis (22–24 percent) within the AHRQ, CMS, VHA, and NIH portfolios of in-scope projects, with this MTD category slightly more common in the ACL portfolio (29 percent) and slightly less common in the CDC portfolio (16 percent). ACL has the strongest relative emphasis on Toolkit Development (12 percent), which might be expected given the role of its NIDILRR research institute on facilitating the implementation of interventions for the agency’s focal populations (i.e., care and support for community-living elders and disabled individuals). AHRQ, CDC, and VHA follow in relative emphasis on Toolkit Development (9 percent, 5 percent, and 4 percent, respectively).
AHRQ shows the highest emphasis on Evidence Review and Synthesis (13 percent), also in accord with the agency’s focus on this function to accelerate implementation of evidence into practice. NIH and CDC have the highest relative emphasis on Simulation Modeling (4 percent and 3 percent, respectively), with smaller emphasis on this set of research methods by AHRQ and VHA (2 percent each).
Chapter Summary
HHS agencies and the VHA have developed research portfolios of HSR and PCR around particular focus areas that address the requirements of their individual congressional authorizations, missions and operational needs. These portfolios differ along three key dimensions—scope of the health care system examined, research objectives, and research audiences. For example, AHRQ, as the only agency that has statutory authorizations to generate HSR with a mission to do so across the healthcare system and to serve as the home for federal PCR, focuses its research portfolio on systems-based outcomes and approaches to implementing improvement across health care settings and populations in the United States. NIH’s portfolio of HSR and PCR addresses a similarly broad scope of health care but tends to be organized around specific diseases, body systems, or populations. The CDC’s portfolio of HSR and PCR is organized around diseases, conditions, and injuries, but focuses on prevention and health promotion spanning community and health care settings. The portfolios of other agencies tend to focus on specific health care settings or other populations (e.g., ACL on community-living elderly and disabled individuals, CMS on Medicare and Medicaid beneficiaries, and VHA on veterans’ health care and health), or research audiences (e.g., ASPE on federal policymakers).
Results of the environmental scan and portfolio analysis’s systematic enumeration of HSR and PCR projects confirmed some of these distinct emphases. These included the relative emphasis of AHRQ on Patient Safety, HIT Applications and Tools, and Evidence Review and Synthesis; CMS on Cost and Utilization and on Financing of Care; CDC on Prevention; and ACL on Social Factors.
At the same time, the scan results showed other agency portfolios to include projects in these areas, albeit to a lesser extent, as well as strong emphasis across all agencies in the analysis on the research domains of Quality of Care and Organization of Care. The qualitative results from TEP and interview participants indicated that agencies would be expected to fund different approaches to these topics, such as portfolios within NIH focusing on quality of care for specific diseases, and portfolios within AHRQ focusing on quality outcomes for health care systems more generally. However, the relatively broad research categories of the scan analysis were not able to detect such distinctions.
The next chapter uses the qualitative TEP and interview data to examine the degree to which overlap in research funded by agencies in similar topic areas is complementary or redundant, and the extent and ways in which federal HSR and PCR funding is coordinated among agencies.
