Establishing an AHRQ Learning Collaborative, A White Paper
At the 2010 National Research Service Award (NRSA) meeting, directors of T32 training programs funded by the Agency for Healthcare Research and Quality (AHRQ) discussed the importance of sharing knowledge and working more closely together. Following this rich discussion, AHRQ issued a request for proposals for the formation of a study group to explore the feasibility of establishing a mechanism for collective knowledge production, specifically the formation of a learning collaborative. Collaborations are formed when two or more stakeholders invest their resources (e.g., talent, information, money), to solve problems that they could not solve by themselves. Central to this concept of collaboration is knowledge translation and knowledge transfer.
Collaborations have become necessary for organizations performing complex work, with emerging technologies and rapidly changing environments. The rapidly changing field of health services research necessitates knowledge transfer and translation among health services researchers spanning multiple disciplines and housed in a number of organizations representing the public and private sectors in academic, medical, public health, and numerous other settings.
Recognizing the rapidly increasing complexity of the field of health services research, all T32 program directors, assisted by Brenda Harding and external staff from Team PSA (Professional and Scientific Associates), were invited to participate in an initial conference call that provided an overview of the concept of an AHRQ Learning Collaborative. This initial conference call helped establish the broad goals of the AHRQ Learning Collaborative, which are to: foster partnerships across institutions; facilitate the exchange of information, networking, and mentoring opportunities; and improve the training of health services researchers by sharing best practices, curricula, and innovative training efforts across institutions.
To gain a better understanding of the unique needs, preferences, and challenges of AHRQ training programs with respect to achieving these goals, a brief initial survey involving 10 of the 18 training programs was conducted. Responses were received from 8 program directors, 14 predoctoral trainees, and 2 post-doctoral trainees. Respondents recommended three key functions for an AHRQ Learning Collaborative: sharing, collaborating, and networking. In particular, some of the specific areas for sharing included: curricula, training opportunities, methods, best practices, ideas and information, career resources, data sources, and presentation opportunities. Collaboration on research projects and publications was often noted as a potential function of the collaborative. Lastly, the prospect of developing relationships among students, alumni, and fellow researchers was seen as a way to network, seek mentorship, enhance communication about the exchange of ideas and future opportunities, and perhaps acquire publications.
To explore these themes in more detail, 15 semi-structured interviews were conducted (five each with program directors and pre- and postdoctoral trainees). Additionally, interviews were conducted with individuals with expertise in establishing and working with collaboratives. These experts represented: AcademyHealth, AHRQ, The Institute for Healthcare Improvement, the NEWMEDS Project, and the PCMH/MU (Patient-Centered Medical Home/Meaningful Use) Collaborative. Applying a conceptual framework developed from a literature review of collective learning, themes were identified and consolidated as they emerged. Through the in-depth interviews, it became clear that both program directors and trainees view the collaborative as a tremendous opportunity for augmenting the learning process at their home institutions, specifically through sharing, building relationships, and developing an identity.
With regard to sharing, the AHRQ Learning Collaborative could shed light on program characteristics with respect to research methods training, knowledge of techniques for secondary data collection and analysis, and overall competence in health services research. With regard to building relationships, trainees expressed a strong belief that mentoring from senior health services researchers was a critical element of their training and professional development and that faculty at their home institutions could leverage their networks of expertise for these purposes. Related to building relationships is the development of a health services research identity. Specifically, trainees agreed that, within the field of health services research, there is a substantial but untapped opportunity to develop a well-solidified community of health services researchers and indicated that the AHRQ Learning Collaborative could be a valuable mechanism for creating this community among fledgling health services researchers, specifically through T32 training programs.
In order to organize and manage the collaborative, AHRQ could play a critical role in institutionalizing the collaborative, and program directors could be co-champions of the collaborative alongside trainees. Both program directors and trainees agreed that, although a variety of communication strategies could be effective, strong networks require periodic face-to-face meetings, at least initially. Additionally, an effective learning collaborative must operate within the information-rich environment in which we exist, and it must be sensitive to time and money constraints.
In order to assess the impact of the collaborative, measures of success should be directly linked to the stated goals and activities of the collaborative, which may change over time and/or with changes related to different issues the collaborative wishes to address. Additionally, these measures would have to be clearly linked with changes in individual training programs that are expected to result as a direct consequence of the efforts of the collaborative.
Based on a synthesis of the current literature as well as the essential insight from current trainees, program directors, and experts in the field, recommendations for an AHRQ Learning Collaborative are as follows:
Structure: Explore establishing two mechanisms for mutual helping among T32 programs: one, an informal, voluntary network of AHRQ T32 programs, and two, several ongoing learning collaboratives that would be supported by formal communication mechanisms, such as in-person meetings and virtual forms of communication.
- Following the T32 program directors meeting, program directors, trainees, and faculty were invited to join a LinkedIn AHRQ Training Program Group, which included three sub-groups: one for training directors and one each for pre- and postdoctoral trainees. The network could be informal and Internet-based, with portals and other electronic media.
- In the startup phase, one or more learning collaboratives could be created based on new issues generated at the T32 annual meetings and/or organized around specific research interests. Each collaborative would be championed by several people and could cover a wide range of technical or problem-driven areas. The work led by Chris Forrest and Diane Martin on health services research core competencies, as well as their affiliations (at the time of that work) can be found at http://www.biomedcentral.com/1472-6963/9/107.
Activities: The key activities of an AHRQ T32 Network would include:
- Establishing mentoring relationships between trainees and senior researchers within and across T32 programs.
- Networking through social media and in-person events.
- Solving problems.
- Helping each other with requests for information.
- Collaborating based on complementary skills and experience.
- Pooling resources, including data, expertise among faculty and trainees, and other assets.
- Discussing critical training issues.
- Sharing best practices.
Roles and Responsibilities: The AHRQ Learning Collaborative would be a joint effort by trainees and program directors with support from AHRQ. Representatives from AHRQ, program directors, and trainees will play an important role in investing in and building social relationships across programs.
- AHRQ T32 program directors meet once a year at the annual NRSA meeting, which offers an opportunity for productive helping and collaborating.
- Trainees could be invited to participate with program directors during T32 meetings to brainstorm about how to build capacity and enhance innovation across programs. Trainees, with the support of program directors, could co-lead and be engaged in these discussions.
- AHRQ could support the learning collaborative by setting aside time during the annual T32 meeting to engage in collaborative activities (for example, sharing what we are learning about tough problems and identifying candidate issues for ongoing learning collaboratives).
Funding: Trainees and directors could apply for AHRQ's Small Conference Grant Program to fund and establish one-year learning collaboratives that result in workshops and sessions at the annual NRSA conference. This program is intended to encourage members to share learning, connect with stakeholders and programs, develop new thinking, and build capacity in health services research.
Launching the Collaborative: Begin preliminary conversations on issues relating to engagement and evaluation, which may include:
- How do we create a national identity?
- How do we cultivate and leverage an open network of training programs whose members have promising ideas and want to help each other?
- How do we create opportunities for faculty and student exchanges or linkages across programs among faculty and students with common interests (e.g., student rotations at AHRQ and/or collaboration with AHRQ researchers)?
- How do we engage the more than 1,500 past and present trainees in new and interesting ways?
- How do we find better ways to connect and develop existing AHRQ research and dissemination awards?
- How do we connect with AcademyHealth and other stakeholders around the work they are doing?
- What communication media will help to rally the people interested in collaborating?
- In addition to the Annual AHRQ T32 Program Directors Meeting, what other forms of collaboration could be conducted (e.g., webinars, in-person local or regional meetings, discussion forums, and wikis)?
- Can we establish a set of common performance measures with common definitions for the purpose of comparative benchmarking across programs?
- How would the network and future collaborations help to establish a national identity for AHRQ trainees that transcends fellowship appointments and funding?
- Should one criterion for AHRQ training grant renewal be the amount of mutual helping, networking, and collaborating with other training programs the grantee has done? Would AHRQ want to establish "proof of collaboration" as part of renewal? How would this be measured?
- For each core competency, can we create a knowledge map (K-Map) that identifies the experts, practitioners, locations, and sources of knowledge? The model for this is the work that Jonathan Weiner, Diane Martin, Tim Carey, and other programs have been doing to map courses, resources, and other capabilities to health services research core competencies.
- How can we highlight achievements of former and current trainees to demonstrate the value of this investment and solidify a national identity?
- How do we work to sustain this effort and maintain buy-in over the long term?
During the 2011 NRSA conference, feedback on a draft white paper was solicited from attendees of the director's meeting. Comments have been integrated into this final white paper, which will guide the next steps for establishing an AHRQ Learning Collaborative.
We offer the following concrete suggestions for moving forward with the AHRQ Learning Collaborative:
- Brand the AHRQ Learning Collaborative with a name and a catchphrase that is memorable or appealing. For example: "SpAHRQ: Training the Next Generation of Health Services Researchers." Individual T32 programs could be referred to as "SpAHRQ Plugs" because they are individually necessary but only collectively sufficient for powering the health services research engine.
- Solicit trainees' thoughts on a name and catchphrase. The thinking behind the name and catchphrase would have to be explained briefly and be immediately clear to others.
- Develop an explicit mission or charter.
- Develop clear objectives and/or goals. These could be selected from those suggested in this White Paper, and they could be field tested for relevance among AHRQ program directors and trainees.
- Develop routine activities. For example, a periodic check-in call or webinar on training program issues where trainees are the prime participants. Program directors, alongside AHRQ personnel, could take turns hosting the call/webinar. Another potential activity would be to publish a brief periodic letter about the AHRQ T32 programs to share program highlights and opportunities for collaboration. Finally, trainees could be "invited" into other programs' classrooms.
- Collate a list of current AHRQ T32 trainees (pre- and postdoctoral) that includes information about research interests, current project work (intra- and inter-institutional), leisure interests (to integrate a more informal component), and contact information.
- "Research interests" could be specific or more general. Also, information to supplement research interests could include: area(s) of expertise, research and/or statistical methodologies of interest, and theories of interest (and discipline in which they are housed).
- Another option for this listing (or a complementary action) is to create a searchable database of trainees (similar to academic institutions' online directories).
- Extend the list/directory to alumni; a particular effort could be made to reach out to recent alumni (i.e., those one 1-2 years out).
- Establish "Career Ladder" interest groups (again with a memorable name) for trainees heading towards a postdoctoral degree and/or academia, the private sector, the government, a think tank, etc.
- One possible way to capture the attention of trainees might be to title these groups using their words. For example, "Major Players in... [various sectors]."
- Collate curricula and include them in a searchable database. Curricula could also be mapped onto the core health services research competencies to highlight where opportunities for improvement exist and/or to allow programs to highlight specifically (and uniquely) how they are ensuring training in the core health services research competencies.
- Identify brokers within AHRQ T32 programs (e.g., directors, faculty, and trainees) who could create buy-in for working collaboratively across T32s.
- Brokers could begin with the questions identified in the "Engagement and Evaluation" portion of the recommendations and report back (perhaps during the periodic check-in call and/or webinar as part of the routine activities).