The National Health Plan Collaborative: Overview of Its Origins, Accomplishments, and Lessons Learned
II. Purposes of the Evaluation
AHRQ contracted with Mathematica Policy Research (MPR) to evaluate the NHPC, starting in June 2005, and followed that with another contract that extended the evaluation to cover the Collaborative's second phase. The purpose of the evaluations in both phases was to answer several questions:
- How was the NHPC structured, and what did it do?
- What did the NHPC accomplish, and how sustainable will these efforts be?
- Did the support provided by the NHPC's process contribute to the firms' ability to make progress in addressing issues related to disparities, and how valuable did the firms consider their participation to be?
- What can AHRQ learn about whether or how to engage in similar collaboratives in the future?
To guide the evaluation, MPR developed a logic model that highlighted the various ways the NHPC could contribute to significant progress in addressing racial and ethnic disparities (Figure 1). The framework highlights steps that health plans can take to reduce racial and ethnic disparities and how the Collaborative might contribute to plans' success. We assessed the NHPC in terms of its ability to help firms:
- Encourage and support their leadership in championing work on disparities.
- Collect data on or estimate the race and ethnicity of their members to better identify potential disparities.
- Develop and pilot interventions to reduce disparities.
- Communicate the outcomes of their work beyond the NHPC. (Go to boxes in logic model with asterisks.)
In both phases, the evaluation was largely qualitative, with little primary data collection. For instance, MPR obtained and reviewed all NHPC documents and sat in on the NHPC's meetings and conference calls as a "silent observer." In Phase I, we also conducted three rounds of discussions with the lead staff of all participating organizations, as well as a larger group of staff from participating firms. We also asked all participants to complete a "network feedback form" to support a formal analysis of the NHPC as a communications network. In Phase II, we again conducted several rounds of discussions with NHPC participants, but targeted only the lead staff in each organization. Moreover, to minimize burden on the part of NHPC participants, these discussions were substantially shorter and more focused.
The NHPC's work evolved over time; sponsors, support organizations, and participating firms learned more about each other and how they could work together. As the NHPC evolved, so did the evaluation. In particular, in addition to providing formative feedback on the overall evolution of the NHPC and the way it was working, the Phase II evaluation examined existing work in NHPC's priority areas (through literature reviews and discussions with national experts and relevant program staff). Evaluation goals in Phase II focused on generating two types of information: formative feedback on overall progress (as in Phase I) and case studies of outcomes of work in priority areas.vii
vii The case studies originally were intended to provide information on external use of NHPC's work in data collection, language services, and the business case for reducing disparities. But external dissemination was not sufficient to make this worthwhile. Therefore, the case studies briefly discuss the NHPC's work but focus more broadly on national and State activities in data collection and language services, as well as employer perspectives on reducing disparities.