Recommendations
Outcomes research, like technology assessment, can be thought of as a set of analytic methods that produce information
about the benefits, risks, and costs of health care services. Outcomes studies can support a particular decision, but they do
not make decisions, and therefore have no impact unless they are acted upon. The impact of this enterprise therefore is
critically dependent on the action of decisionmakers in health care, who must be aware of, understand, accept, and use this
information. The options described below are each influenced by the objective of enhancing OER impact, taking into
consideration this perspective on the essential nature of OER as knowledge, not action.
Enhancing the impact of OER in the dissemination phase has often been thought of as the final activity of a research effort.
Dissemination is listed as the last activity of the PORT projects, and generally received the least attention and achieved the
least success. Educational outreach, and/or social marketing, represent the standard of success in this aspect of most OER
studies. In many cases, dissemination activities simply received little serious attention. In addition, researchers' goals for
dissemination (publications; dissemination of knowledge) and decisionmakers' needs (positive changes in practice) are not
necessarily linked.
To affect practice and outcomes, these issues must be considered much earlier in the research process, and should
influence how research questions are framed, what methods used, etc. It would not be premature to consider issues of
dissemination as part of the criteria that enter into the funding decision—do the questions addressed and the proposed
approach have a reasonable chance of affecting policy and practice? Critical components of successful marketing include a
market analysis, which helps to determine which products are desired by consumers, and what features those products
should have. For consumers of OER, the health care decisionmakers, very little effective effort has gone into linking
research funding to information needs.
Several general options to enhance the probability of positive impact are identified and discussed in greater detail in this
section of the report:
- Prioritize research according to some measure of "likelihood of impact," with particular preference given to proposed studies that build on previously funded work.
- Fund more "problem-solving" studies that explicitly look at various strategies and interventions to change practice.
- Promote improvements in strategies and methods used to study effectiveness.
- Promote development and use of practical patient-oriented outcome measures.
- Identify research findings that should be implemented, and provide usable information and technical assistance to "change agents" that are in a position to influence practice.
- Create a surveillance mechanism that will systematically document changes in practice that may be related to Agency-supported work.
Return to Contents
Set Research Priorities: Orchestrated Follow-On
One of the most significant contributions of AHCPR and COER in the past decade has been to identify specific gaps in
scientific knowledge that lead to high levels of practice variation and suboptimal outcomes. Many studies supported by
COER over the years have served as hypothesis-generating studies, but there has not been a systematic effort to orchestrate the hypothesis-testing follow-on. COER could take on the responsibility to make sure that once these critical
knowledge gaps are identified, they are addressed in follow-on studies. Limited resources prevent the Agency from funding
all such studies, but there are increasing opportunities to leverage outside resources. Private-sector organizations and other
public Agencies have considerable resources and increasing motivation to undertake applied research. COER should take
on the role of a "catalyst" for additional research, attempting to leverage these resources when an appropriate topic has
been identified.
The goal of increasing measurable impact can be approached by selecting research topics and methods that optimize likely
impact. To the extent that COER is able to link internally supported hypothesis-generating studies with more applied studies, the Center will be creating a chain of efforts that will ultimately lead to measurable improvements in patient care.
Proactive efforts to forge these linkages will create a track record of accomplishments that link the descriptive studies (e.g.,
level 1) to visible impacts on policies, practices, and outcomes (levels 2 through 4).
For example, the study of treatment of otitis media in the Medicaid population was described earlier. The principal
investigator for those studies went on to collaborate with policymaking organizations to develop guidelines for antibiotic
choice. However, despite the clear suggestions made by this study that clinical practice should change, we were not able to
identify any followup reports of specific efforts to change practice in the direction indicated.
There are many other examples of OER studies that provide descriptive findings that strongly suggest ways in which care
and outcomes could be improved. There are, however, only rare—and serendipitous—examples of those finding being
effectively pursued to document that the proposed changes in practice can be achieved, and that they result in improved
outcomes. Currently, it is left up to investigators to pursue the logical next steps from their research, and they may lack the
interest, knowledge, or resources to do so. There is no strategic plan that would give priority to this follow-on work.
There is clearly a tension between efforts by the Agency to channel research in desired direction, and the need to preserve
investigator initiative and creativity in research. While unchanneled intellectual exploration is a time-tested and vital
approach to research, it may not be entirely satisfactory for AHCPR, given the expectations for measurable impact
discussed throughout this report. Similarly, there is an implicit tension associated with the strategy of giving preference to
promising follow-on work over supporting work in a new area. A specific approach would be to develop a targeted
initiative promoting follow-on efforts for completed studies, coupled with supplements to studies in progress intended to
enhance impact.
Another practical aspect of strategically pursuing research opportunities through higher levels of impact is to identify
potential partners. The Health Care Financing Administration (HCFA) and other payers, MCOs and medical groups, medical professional organizations, and medical
product manufacturers would all be potential research partners. Collaboration with these organizations would not only
allow AHCPR to leverage resources, but would ensure that potential studies are crafted to meet the applied needs of these
organizations. If this approach is successful, dissemination and implementation are built into the research plan, since the
target of impact is also a partner in study design and funding. AHCPR could produce a Request for Applications (RFA)
that motivates the
investigators to develop partnerships by requiring that the project obtain 50 percent of its funding from a source other than
AHCPR. This would serve both to leverage AHCPR resources, and would also promote partnerships with "end-user"
organizations to design and conduct research. Effective and efficient strategies for establishing these partnerships need
further development.
Return to Contents
Clarify Criteria for Relevance and Importance
The review process for new grants will need to adopt new criteria that give preference to studies that build upon basic
findings and convert those findings into system impacts. This "potential impact review" would be added before or after the
existing scientific review, and would specifically consider the likely impact of research from a clinical and policy
perspective. It may be valuable to have health care decisionmakers with operational responsibilities participate in this
"potential impact" review process, rather than solely the research community, either prior to funding or as part of a
retrospective review.
Review criteria should also be sensitive to the likelihood that private sector OER will be adequate. The overview of
private sector OER recently completed by Lewin provides a description of the types of OER performed in the private
sector, and what gaps remain in these activities (Mendelsohn et al., 1998). The main conclusions from the Lewin report are that
the OER of private-sector organizations are closely tied to the financial interests and business viability of the respective
sponsors. They are most likely to take on high-prevalence conditions, with high unit costs, but also those for which a
potentially high-cost new technology, service, or product is available. Important but commercially unattractive problems,
such as sexually transmitted diseases (STDs) and substance abuse, are less commonly addressed. Private-sector organizations are less likely to invest in
the development of tools, methods, or basic research strategies. Most health-related quality of life (HRQL) instruments and analytic methods that are
well designed and validated and used by private organizations were developed with public sector support.
While the Agency should consider increasing its funding for more "applied" studies, there is a need to determine how best
to balance the research portfolio. The tradeoff to explore is between studies that demonstrate near-term high-level
impacts, versus more basic studies (tools development, hypothesis generation) for which there may be important long-term
payoff. AHCPR should invest in studies that provide level 1 impacts, while also supporting an adequate number of studies
that target level 3 and level 4 impacts. One model for this "pyramid of effort" (or funding) for OER would be wide at the
bottom (develop methods, identify important problems) and narrow at the top (implement programs to change practice).
This is depicted in Figure
1 (12 KB).
Figure
1 primarily reflects the gradations for types of study that are likely only to be supported by the public sector, and
types of studies that can potentially be supported in the private sector. A focus on "special populations," including children,
minorities, and women represents a unique public interest.
Return to Contents
Make OER Approach Responsive to Existing Quality of the Evidence
The existing quality of research completed in a clinical field has implications for what level of impact is likely over a short
time period as a result of OER. In addition, the current state of evidence will also have implications for what type of OER
is most likely to be valuable. For example, in fields such as surgery, for which the state of the science begins at a low level,
it may be most efficient to focus on studies that raise questions, or identify important hypotheses. In addition, development
of patient-reported outcome tools, methods for severity adjustment, etc., may be needed before more sophisticated studies
can be done.
It should be possible to develop an approach for assessing the existing quality of evidence in a given field. This would be
parallel to a "readiness to change" assessment used for behavioral interventions. To improve impact, OER studies
should be supported in fields that are determined to be "ready" for change. Again, it will be necessary to maintain a
balanced portfolio, as more basic studies will still be needed in clinical disciplines for which the evidence base is less well
developed.
Return to Contents
Increase Emphasis on Problem-Solving
The first decade of OER has been particularly successful in describing what actually occurs in medical practice, and in
creating and refining tools to be used in studying what works. It has, however, become clear that these activities
themselves do not lead inexorably to practice improvement. A critical question for COER is to determine the appropriate
fraction of effort and resources that should be devoted to studies that incorporate strategies for change as part of the study.
This next stage of OER would focus attention on creating designs and blueprints for improving care, and selectively
applying the OER tools when they are needed. It would represent a move from a "tool-using" culture to a "problem-solving" culture, for which the primary objective is developing an empirical science of clinical improvement. The COER
could be refashioned into the "CHOIR" (the Center for Health Outcomes Improvement Research).
In the past, an OER investigator might begin with a clinical condition and a particular large dataset and ask the question:
"What can I learn about condition x from database y using method z?" The CHOIR approach to this would frame the
question as: "What do decisionmakers need to know about condition x, and which method and database are most likely to
provide this knowledge?"
This change would require a new emphasis on a number of activities at AHCPR. First, a systematic effort would be
needed to define in detail what high priority problems remain for specific clinical conditions. This will require more than
identifying conditions that are high cost, or associated with large variations in practice. More specific, testable hypotheses
about particular clinical practices need to be defined. These could be derived in part from intramural analysis of claims and
other data.
The most important step is to make sure to ask the right question and address that question with a methodology that is
appropriate. There should be a "scientific assessment of the evidence needed to stimulate practice change" (Clancy and Eisenberg, 1998).
Over time, it may be possible to develop standardized approaches to deciding what type of study is needed, given specific
features of the issue being addressed. One model for this approach is a recent analysis of the value of trials vs.
observational studies to support policy decisions (Hornberger and Wrone, 1997). A major consideration in choosing an analytic
approach will be to ask who the decisionmakers are that the research is intended to influence, and what sort of evidence
will be required to alter their decisions. These questions merit serious empirical analysis.
This process can go on internally at AHCPR, and should also be required of investigators submitting proposals to the
Agency. In either case, the shift in culture should ensure that future studies are explicitly linked to problems in need of
solutions, and use study designs that reflect the specific features of the problem being addressed. Because this approach
represents a significant shift for AHCPR, it may be most practical to solicit proposals for prototype projects. Development
of the RFA would itself allow focused internal discussion about the conceptual, methodological, and operational challenges
of adopting this strategic approach.
Promote Improvements in Strategies to Study Effectiveness
One of the specific areas in which COER work has been less successful than hoped is in providing definitive answers about
the comparative effectiveness and cost-effectiveness of health care interventions (i.e, what works best in health care). We
need to develop innovative ways to address the many unanswered questions about effectiveness and cost-effectiveness,
including the effect of environmental characteristics.
Little progress has been made in addressing the well-recognized limitations of prospective clinical trials (i.e., cost,
generalizability, timeliness). While the Agency has increased its support for RCTs, these still tend to resemble traditional
efficacy studies, and therefore remain complex, slow, and costly. There has been no concerted effort to craft a new
methodological, organizational, and ethical framework for these studies. The "conceptual infrastructure" for conducting
effectiveness trials remains to be developed.
Effectiveness trials attempt to blend the "real world" context of database studies with the bias protections of traditional
RCTs. They have broad eligibility criteria, non-intrusive study protocols, and make efficient use of data collected as part
of routine practice. These studies would seek to address issues of effectiveness in the context of the expanding diversity of
health care providers, settings, and financing—looking at the outcomes associated with, for example, long-term care
facilities, day surgery, various models of managed care, paraprofessionals, providers of alternative medicine, etc.
Essentially, this approach minimizes the distinction between research and clinical practice, for the purpose of creating a
systematic and efficient mechanism that allows the medical profession to learn while doing. By requiring a linkage
between research and practice, effectiveness trials are by themselves one approach to effectively translating evidence into
practice. Personal involvement and participation is one of the important conditions for producing changes in professional
behavior (Davis et al., 1994). Informal evidence of this is the dramatic increase in the use of aspirin after MI in England
followed the ISIS trial (Collins and Julian, 1991), which may well have resulted in part because a critical mass of British physicians
were investigators in that large trial. Practice did not change among Italian physicians until the ISIS results were replicated
in Italy in the GISSI trial.
COER could engage in a major initiative to create and refine new approaches to conducting prospective effectiveness trials.
This effort could be similar in visibility and scope to the "effectiveness initiative" of the late 1980s, which motivated the
research community to develop strategies for using large administrative databases as an alternative to traditional clinical
trials (Roper et al., 1988). The creative energy and ingenuity of the health services research community was directed at this problem with notable
success, and it is likely that similar contributions could be made in crafting new approaches to studying effectiveness in
real-world settings using prospective, experimental methods. Current practice research networks represent one important
opportunity for establishing this initiative and the requisite infrastructure.
This recommendation would require a significant amount of energy and leadership from the Agency to be pursued
successfully. It may be a similar undertaking to that required to launch the RCT as the "gold standard" of efficacy
research. AHCPR has already accomplished something of similar magnitude with the PORTs, a large-scale effort that
similarly created a new model of conducting research. As with the PORTs, the new effectiveness initiative would support
methods development, enhance specific technical skill in the research community, promote conceptual discussion of
effectiveness trials, and create networks of researchers and institutions. In the same way that clinical guidelines
development and observational studies were given a major boost in the early 90s, this new initiative would support a
subpopulation of researchers to identify and explore all of these issues, and hopefully launch this field as successfully as
OER was launched.
Promoting creative strategies for prospective studies would require a significant conceptual shift. Prospective effectiveness
studies can only be undertaken when professionals involved in delivering care become participants in the research
enterprise. These studies can best be done as partnerships between researchers, delivery systems, and funding
organizations. The operational and organizational challenges of this are quite significant. For example, creating the
infrastructure and techniques which facilitate data collection in routine practice and implementing them in ways that do not
increase clinician workload will obviously require some ingenuity. Similarly, issues of identifying how funding
responsibilities should be shared, obtaining adequate sample size, managing conflicts of interest among partners, recruiting
patients, obtaining patient consent, and protecting patient confidentiality all need considerable attention.
Analysis of large administrative databases and observational studies of all kinds should continue to be an activity supported
by COER. A shift in the weight of emphasis toward prospective studies for some period of time will help to establish the
feasibility and cost of developing and implementing such methods. It may be useful to have an initial conference bringing
together investigators who have already worked in this area, and who have at least identified many of the complex
challenges. This event could also serve as an announcement of this initiative by the Agency. Following this, an RFA
could be issued that would support several prototype effectiveness trials.
Promote Routine Use of Outcomes Measures
Routine use of patient-reported outcome measures in clinical practice remains the exception (e.g., the prostate symptom
score developed by the BPH PORT and the American Urological Association [AUA]) rather than the rule. Expanding the metrics for answering the question
"What works?" cannot be successful until clinicians incorporate these tools in daily practice. A strategic focus on
clarifying the validity of these measures and identifying strategies that encourage their use is essential to COER's future
efforts. Additional work on the incorporation into practice, feasibility, and credibility of disease-specific and general
outcomes measures, and development of tools and strategies to make this possible (e.g., a database of measures analogous
to CONQUEST) are a critical requisite to getting to outcomes that people experience and care about. HCFA's "Health Outcomes Survey" measure, and other measures being developed by the Measures Advisory Panels (MAPs) at NCQA, are likely to
provide a new opportunity for pushing the current frontier, but important conceptual and philosophical challenges must
first be addressed.
For example, because current efforts to include patient-reported outcome measures are conceived as mailed surveys, the
feedback loop to clinicians is assumed rather than specified. The importance of incorporating patients' preferences and
values is widely acknowledged, but practical approaches for valid and reliable measurement remain to be determined.
Work that clarifies the conceptual domains of HRQL measures and permits enhanced interpretability across measures is
needed before many measures can be widely used (Gill et al., 1994). Strategies for estimating the clinical significance of
differences in outcomes remain much less well developed than those for statistical significance (Deyo and Patrick, 1995). One
difficulty in promoting widespread use of patient-reported outcomes measures is that clinicians are unlikely to use them
routinely unless the reports are as familiar to them as blood pressure and other physiologic measures—but this cannot occur
until outcomes measures are developed that are easy to include in day-to-day practice. Improvements in outcomes
measures and development of strategies to encourage their routine use are an essential future direction for COER.
Continue Tools and Methods Development
New challenges in terms of methods development and refinement have developed from the previously discussed advances
in OER as well as the shift towards projects that will have greater impact on policies, practices, and outcomes. One such
challenge is in the developing area of cost-effectiveness analysis (CEA). Contemporary medicine has been greatly
influenced by increases in health care costs to over 14 percent of our Nation's gross domestic product with discussion of
rationing, the shift towards managed care, involvement of consumers in decisionmaking, and chronic diseases in which
there are often tradeoffs between quantity and quality of life. Such changes have resulted in greater demands for
information about value. While the use of CEA has increased dramatically in recent years (Elixhauser et al., 1998), there is little
agreement about methodologies in determining both costs and metrics of effectiveness making it difficult to assess the
validity of a study or comparisons between studies. As CEA examines incremental change in resources required and health
effects gained by using one intervention compared with another, some studies base comparisons on the best available
alternative, while others are based on viable low-cost or do-nothing alternatives. Other unresolved areas include which
costs to include and how to measure them, how to discount costs and outcomes over time and whose preferences to use and
how to measure them in determining effectiveness.
The first decade of OER has demonstrated that database analysis and systematic literature review cannot substitute for
clinical trials. Instead, it has raised the question of how trials can best be designed so that results are rapidly applied to
practice. Although the randomized controlled trial is presently considered the gold standard, other designs including
minimization have been suggested as alternatives. Under minimization, group allocation is not based solely on chance
(Treasure and MacRae, 1998). Instead, patients are stratified by known or suspected determinants of outcome, such as diabetes and
hypertension in the study of coronary artery disease. Such stratification substitutes for multivariate analysis in other study
designs. With further experience with and understanding of trial designs, researchers should be able to more appropriately
choose methodologies to match the question being asked and the level of evidence needed to stimulate change by
practitioners and policymakers.
Create the Capacity to Build Upon and Transfer Knowledge
The Agency should consider developing the capacity to identify important research findings (generally level 1 impacts),
and to assist those in moving to higher levels. This capacity should be proactive, intensive and consistent, and not left
solely to the ingenuity of each research team. The research methods (and probably the resources) for confirmatory and
follow-on studies suggested by OER work require technical skills and organizational knowledge that is not well developed
within the mainstream health services research community. In addition, the skills to disseminate information effectively and provide support
to health care organizations that hope to translate research into practice are also not common. Academic incentives tend to
favor analytic exercises over dissemination, since the former require less effort to produce publishable results.
A dedicated knowledge transfer function would systematically consider findings from OER studies and carefully assess
them to determine how best that work can be translated into higher levels of impact. In some cases, the appropriate next
step would be a specific type of follow-on study, while in other cases, some effort to influence practice in the direction
suggested by the study would be indicated. Under other conditions, it may be appropriate to convene affected specialists to
discuss the clinical and research implications of the findings, and have this group suggest a follow-on strategy. The core
theme to each of these alternatives is that the Agency should participate in developing a structured, scientific, and
systematic approach for guiding research from its basic stages through clinical applications, and finally to measured
improvements in outcome.
Knowledge transfer could be accomplished by working with individual investigators, creating an AHCPR center devoted to
knowledge transfer, or creating an external entity with this function. The Center for OER Knowledge Transfer could be
supported by the Agency, possibly through a relationship similar to that between the Agency and Evidence-based Practice
Centers. An internal or external center would develop a strategy for collecting and documenting examples of OER impact
stories in real time. In addition, the Center could serve as the coordinator to ensure that research is acted upon. Center
staff would act as brokers for new, useful knowledge—working out how particular knowledge might best be acted upon,
making necessary connections between user groups, new knowledge, and potential funding. The Center could then also
provide technical assistance to organizations in implementation projects.
Promote Clear Vision of Roles and Impact
For COER to be seen as accomplishing its goals, those goals must be agreed upon and clearly stated. A revised
and updated Program Announcement, coupled with articles in the peer-reviewed literature, should be developed to
articulate and reinforce these key themes. These themes should be documented, widely distributed, and frequently referred
to in Center activities, deliberations, and presentations. If, for example, COER decides that the largest part of its activities
should be devoted to the development of basic tools and methods, then this strategy should be clearly described and
explained. One benefit of explicit and measurable goals for COER is that it will be possible to review efforts of the Center
and evaluate the degree to which its stated objectives have been accomplished. It should also be possible to determine
whether the assumptions and beliefs upon which these objectives were established were valid. One of the difficulties of the
current efforts to learn from past experiences is the lack of an explicit framework and stated objectives for the work that
has been performed.
For example, COER might decide to emphasize basic studies to develop and validate research tools and outcome
measurement instruments. For this approach to be considered successful, the output of these efforts would have to be
incorporated by other researchers and by health care organizations. The Agency could set explicit goals concerning the
adoption of these tools and methods. At the same time, surveillance systems would be introduced to detect and measure
these effects. In 3 to 5 years, the actual accomplishments could be compared to the stated objectives, and reasons for
success or failure could be explored.
Clarifying the vision for OER requires that important terminology and concepts are clear to those who conduct OER, and
those who are expected to use it. Responses to calls for priority topics from the Agency suggest that there is persistent
confusion about the difference between OER and guidelines. Some attention has been given to clarifying the relationship
between outcomes studies, effectiveness studies, outcomes management, quality measurement, etc. The quality-accountability continuum in use at AHCPR for the past several years provides some conceptual linkage for these related
activities. Translating that continuum into an internal strategy for channeling resources would be a useful planning
exercise.
It is also important to clearly convey the complementary nature of OER studies with traditional basic science and clinical
research. Policymakers need to have a firm understanding that OER provides the link between basic research and clinical
practice. While clinical research identifies potentially valuable new drugs and procedures, OER studies help to clarify how
new technologies can be applied in practice with greatest health benefit and efficiency. For example, in the case of congestive heart failure (CHF),
clinical trials have demonstrated the efficacy of angiotensin-converting enzyme (ACE) inhibitors and other therapies. It is not, however, within the domain
of traditional clinical research to examine such questions as which CHF patients benefit from being managed in an intensive care unit (ICU) setting. AHCPR has sponsored studies that show that many low-risk patients are managed in the ICU, and some patients
are inadequately managed outside the critical care setting. OER investigation into these issues helps to generate strategies
to most effectively and efficiently deploy clinical interventions and systems of care. Although these studies complement
clinical research advances in managing CHF, the value of OER is currently less well appreciated by congressional funders
than the value of clinical research.
Increase Interaction with Major Stakeholders and Customers
The prevailing model of health care delivery has changed dramatically, and the science of OER must be responsive to these
changes. As an applied discipline, OER is faced with a constantly shifting target. A high level of interaction with
stakeholders in the health care system will ensure that the basic studies are supportive of real problems faced by those
involved in health care delivery. The adoption and use of information by the private sector will serve to increase the
apparent and measurable impact of more basic studies supported by the Agency. While health care providers and other private sector groups may have incentives and resources to conduct applied research or demonstration programs, they may not
have all of the technical resources or perspective necessary to initiate these studies. COER (or CHOIR) can serve as a
catalyst to engage the resources and incentives in the private sector under these circumstances. This again requires a higher
level of engagement and interaction with these organizations than there has been in the past.
COER should continue to develop a detailed understanding of the operational organizations that comprise the health care
system. Ideally, MCO managers and medical directors, hospital administrators, insurance company business managers,
medical product industry marketing directors, pharmaceutical product managers, private sector outcomes researchers, and
others could directly participate in a range of Center activities. These individuals could serve on grant review committees,
participate in panels and workshops, be assembled in regular focus groups, be involved in site visits, etc.
The level of involvement with other organizations interested in conducting or using outcomes studies could also be increased. Medical professional organizations, medical group practices, and others bring an understanding of the needs and
challenges of their respective members and institutions. Through a greater level of ongoing interaction, these needs could
become factors that influenced decisions about research priorities and methodology. Cooperation could also result in
concrete opportunities to collaborate on specific studies, bringing these organizations into a more active research role, and
ensuring that their real-world perspectives were integrated into these research efforts.
It may also be useful to conduct a major survey or convene focus groups of OER stakeholders (managed care
organizations, pharmaceuticals, consumers, others) to determine opportunities and strategies for better serving end-users.
However, the direct involvement of these individuals in the activities of AHCPR will increase the degree to which their
perspectives are represented.
Return to Contents
Conclusion
The effectiveness initiative, launched a decade ago, has yielded both important findings and opportunities for
improvement. A new emphasis on evidence-based decisionmaking and interest in assessing and improving quality of care,
along with interest among a broad array of private-sector organizations in OER, set the stage for this analysis. The attempt
to systematically review research output from an organization has few clear precedents, and necessarily reflects the limited
perspective of the reviewers themselves, as well as the difficulty of determining the value of generated knowledge. A key
conclusion from this analysis is that the accomplishments of the first decade provide a foundations from which to advance.
However, it is clear that interests and incentives of the research enterprise and the "change agents" within the health care
system must be more effectively aligned in order to achieve the ambitious promise envisioned when AHCPR was
established.
Return to Contents
Proceed to Next Section
540 Gaither Road Rockville, MD 20850