Implementing AHRQ's Tools in the Field: Successes and Lessons Learned 

On September 20, 2011, Bonnie Ohri made this presentation at the 2011 Annual Conference.

Slide 1

Implementing AHRQ's Tools in the Field: Successes and Lessons Learned

Bonnie Ohri
Deputy Director, Operations/Marketing and Implementation
Office of Communications and Knowledge Transfer (OCKT)
Agency for Healthcare Research and Quality

Bethesda, MD— September 20, 2011

Slide 2

Implementing Knowledge in the Field

• Cheryl Thompson: Partnership development.
• Barbara Kass: Technical assistance/training.
• Margie Shofer: Learning networks.
• Discussion.

Image: A man wears a sandwich-board sign that reads "Putting the KT [Knowledge Transfer] in OCKT."

Slide 3

Putting Knowledge Where People Are

• Increasing quality, safety, and access.
• Improving outcomes.
• Enhancing efficiency and effectiveness.
• Disseminating messages.

Slide 4

The OCKT Marketing & Implementation Continuum

Image: A graphic shows the following steps in a continuing cycle:

Media
Marketing
Knowledge Transfer

Slide 5

Knowledge Transfer (KT) is More Than Marketing

• KT implementation is:
  • An interactive process.
  • An interchange of knowledge.
• KT implementation is not:
  • A one-time discrete event.
  • Simply dissemination.

Slide 6

System Transformation Drives Need for KT

• Goal:
  • An outcome oriented, inclusive health care system.
• Strategy to achieve:
  • Move research into practice.
• Need:
  • Increased awareness, outreach.
• Primary challenge:
  • Use of the right tools to reach target audiences.

Slide 7

Effective KT Improves Knowledge Use

• Enhances awareness.
• Increases knowledge.
• Ensures implementation of tools.
• Identifies successes and barriers.
• Provides feedback to the Agency.

Slide 8

Feedback/Follow Up Are Critical in KT

1. Is the product being used?
2. Is it being used correctly?
3. Is it useful in its current form?
4. How can it be improved/refined?

Slide 9

Implementing Knowledge in the Field

• Cheryl Thompson: Partnership development.
• Barbara Kass: Technical assistance/training.
• Margie Shofer: Learning networks.
• Discussion.

Slide 10

What is Partnership?

Image: Two puzzle pieces are shown being fitted together.

Slide 11

Why Do We Do Partnership Development?

1. Connection to Real World.
2. Share of Common Interests.
3. Support in Shared Goals.
4. Wider Spread/ Further Dissemination/ Greater Implementation.
5. Greater Awareness Raising/ More Informed Thinking/ Broader Behavior Change.

Slide 12

Partnership Development in Five Steps

Image: Graphic shows the following steps in a continuing cycle:

• The Introduction.
• We're Dating.
• Yes! Let's Do This.
• The Sweet Spot.
• Good as it Gets.

Slide 13

Partnership Development in Five Steps

Image: Graphic shows the following steps in a continuing cycle:

• The Introduction.
• We're Dating.
• Yes! Let's Do This.
• The Sweet Spot.
• Good as it Gets.

The section captioned "The Introduction" is broken out of the circle.

• Getting a foot in the door:
  • Watch.
  • Do Homework.
  • Why are we a good match?
  • How might we work together?
  • Practice your pitch.

Show off your assets!

Slide 14

Partnership Development in Five Steps

Image: Graphic shows the following steps in a continuing cycle:

• The Introduction.
• We're Dating.
• Yes! Let's Do This.
• The Sweet Spot.
• Good as it Gets.

The section captioned "We're Dating" is broken out of the circle.

• Mutual interests identified and shown.
• Partner shows familiarity with AHRQ Portfolios/Work.
• Partner asks valuable, on-target and engaging questions.

There may be something here!

Slide 15

Partnership Development in Five Steps

Image: Graphic shows the following steps in a continuing cycle:

• The Introduction.
• We're Dating.
• Yes! Let's Do This.
• The Sweet Spot.
• Good as it Gets.

The section captioned "Yes! Let's Do This" is broken out of the circle.

• Partner exhibits attribute and behaviors from #2.
• Requests/Accepts orders of publications and tools to share with colleagues/members or patients.
• Partner acknowledges your relationship publically (eNews, journal, blog...).
• Talks about doing more together.

Only you're still doing most of the calling.

Slide 16

Partnership Development in Five Steps

Image: Graphic shows the following steps in a continuing cycle:

• The Introduction.
• We're Dating.
• Yes! Let's Do This.
• The Sweet Spot.
• Good as it Gets.

The section captioned "The Sweet Spot" is broken out of the circle.

• Partner exhibits attributes and behaviors of #2 and most of #3.
• Partner begins to suggest ways to collaborate:
  • Speaker for an event/annual meeting.
  • Offer to post special announcements.
  • Co-host/sponsor Web conferences.
  • Offer names of other potential leads for outreach and dissemination.
• Partner willing to carry some of the burden/develop and sign an agreement.

Slide 17

Partnership Development in Five Steps

Image: Graphic shows the following steps in a continuing cycle:

• The Introduction.
• We're Dating.
• Yes! Let's Do This.
• The Sweet Spot.
• Good as it Gets.

The section captioned "Good as it Gets" is broken out of the circle.

• We've passed #2, some of #3, all of #4.
• Partners report or we can track measurable change in:
  • Behavior/process of organization/ staff/ patients.
  • Health conditions/ outcomes of patients.
• Can capture meaningful case studies.

Slide 18

Lessons We've Learned About Partnership Development

Slide 19

Partnership Lessons Learned

1. Bigger is not always Better.

Image: A cartoon shows two cars on either side of a gas pump, one tiny and the other huge. The owner of the tiny car is whistling as he fills up his tank. The owner of the huge car says, "Oh, shut up!"

Slide 20

Partnership Lessons Learned

2. Slow and Steady Wins the Race.

Image: A cartoon shows the Tortoise and the Hare in a race against each other.

Slide 21

Partnership Lessons Learned

3. Listen, Engage, Connect.
4. Individualized Attention vs. Cattle Calls.
5. Think Creatively.

Slide 22

Partnership Lessons Learned

6. Primary Care Doctor Not the Only Target.

Image: A photograph shows several different types of medical personnel.

Slide 23

Partnership Lessons Learned

7. Identify:
   • Connectors, Mavens, Salesmen/women.

Image: A graphic titled "The Tipping Point + Social Media" shows three figures, representing the connector, the mavens, and the salesman, and described their functions:

• The Connector connects people to each other.
• The Maven connects people through sharing knowledge.
• The Salesman uses knowledge to engage and persuade.

Source: M. Gladwell, The Tipping Point.

Slide 24

Partnership Lessons Learned

8. Mindful of Cultural Appropriateness:
   • Race.
   • Ethnicity.
   • Urban.
   • Rural.
   • Low Socio-Economic Status (SES).
   • Health Literacy Levels.

Image: A photograph shows numerous hands belonging to people of different skin colors/races grasping each others' wrists to form a circle around an image of the Earth.

Slide 25

Partnership Lessons Learned

9. Manage Your Partners:
   • Keep Partners Engaged.
   • Balancing Act.
   • Juggling Act.
   • Forging/Forming Unusual Parings.
   • Make Every Partner Feel Important.

Image: One orange-colored figure stands out in the midst of a crowd of non-descript white figures.

Slide 26

Partnership Principles

10. Enjoy Your Work.

Image: A cartoon shows two chickens at work.

Slide 27

Implementing Knowledge in the Field

• Cheryl Thompson: Partnership development.
• Barbara Kass: Technical assistance/training.
• Margie Shofer: Learning networks.
• Discussion.

Slide 28

What is Technical Assistance and Training?

• Technical Assistance:
  • Subject matter experts.
  • AHRQ staff and consultants.
• Training:
  • Correct usage of tools, research, and products.
  • Impact.
  • Feedback—how can we improve?

Slide 29

Goals

• Transfer knowledge.
• Develop partnerships among AHRQ stakeholders.
• Raise awareness.
• Ensure implementation.

Slide 30

Summary of Two KT Projects

• Outreach to Large Hospitals and Health Systems to Implement Project RED (Re-Engineered Discharge)—a hospital readmissions reduction initiative (2009-2012).
• Outreach to Health Professional and Education Groups to Implement U.S. Preventive Services Task Force Recommendations (2009-2011).

Slide 31

Project RED: Overview

• Ongoing AHRQ-funded project at Boston University Medical Center—Brian Jack, M.D. (P.I.).
• Standardized methods to prevent readmissions:
  • Discharge planning.
  • Patient teaching.
  • Post discharge follow up.

Image: The Project RED Avatar 'Louise' is shown.

Slide 32

Project RED: Methods

• Developed training program:
  • Curriculum:
    • Preparation.
    • Patient admission and education.
    • Discharge and follow-up.
    • Launch.
    • Metrics.
  • Technical assistance and training:
    • Processes and components.
    • Implementation.
    • Evaluation of the impact.

Slide 33

Project RED: Successes

• Leadership and staff were engaged and supportive.
• New approaches for ensuring timely outpatient appointments.
• Actual implementation of the tool with feedback.

Slide 34

Project RED: Challenges and Solutions

• Slow initial response from hospitals:
  • Solution: Collaboration and outreach through State hospital associations.
• Few resources to fill discharge advocate role or to purchase software:
  • Solution: Divide the discharge advocate job among existing staff.
  • Solution: Used AHRQ's free tool to teach patients.
• Customized training:
  • Solution: Adaptability.

Slide 35

Overview: U.S. Preventive Services Task Force (USPSTF) Outreach

• Increase awareness of recommendations among health professions educators and students:
  • Curricula of graduate health professions education.
• Outreach to nursing and pharmacy clinicians and educators, faculty and student associations, priority populations:
  • Presentations at national professional associations.

Slide 36

USPSTF Outreach: Methods

• Reaching out to Area Health Education Centers (AHECs) to recruit educators at graduate medical programs.
• Contacting national professional organizations for physicians, nurses, nurse practitioners, pharmacists, and physician assistants.

Slide 37

USPSTF Outreach: Successes

• Partnership with AHECs:
  • 4-hour listening session National AHEC Organization (NAO) annual conference.
  • Attended by 60 leaders from AHECs and schools of osteopathic medicine.
  • Led to partnership with American Association of Colleges of Osteopathic Medicine (AACOM).
• Identified over a dozen graduate medical programs already teaching USPSTF recommendations.

Slide 38

USPSTF Outreach: Successes

• Presentations to national professional organizations:
  • Association for Prevention Teaching and Research.
  • Society of Teachers of Family Medicine.
  • Society of Osteopathic Medical Educators.
  • American Association of Colleges of Osteopathic Medicine.
  • American Osteopathic College of Occupational and Preventive Medicine.
  • National Council of Asian and Pacific Islander Physicians.
  • American Academy of Nurse Practitioners.
  • American Academy of Physician Assistants.
  • American Pharmacists Association.

Slide 39

USPSTF Outreach: Successes

• Web conferences:
  • National Council of Asian Pacific Islander Physicians.
  • American Academy of Nurse Practitioners.
  • American Academy of Physician Assistants.
• American Pharmacists Association:
  • Self-study slide deck.

Slide 40

USPSTF Outreach: Challenges and Solutions

• Role of AHECs misunderstood:
  • Few AHECs influence medical school curricula.
  • Role is managing preceptors and clinical rotations.
  • Preceptors are too busy to create and teach prevention.
• Solution: provide technical assistance in developing materials:
  • University of Colorado Health Sciences Center, School of Medicine.
  • Southwestern Colorado AHEC.

Slide 41

USPSTF Outreach: Challenges and Solutions

• Changing curriculum is difficult:
  • No time to prepare new course materials.
  • Curriculum additions requires approval by medical school board.
• Solution: create new curriculum materials:
  • Technical Assistance Document.
  • "Understanding the Methods Used by the USPSTF" slide deck.
  • "Putting Prevention into Practice" slide deck:
    • Included expert advisory board recommendations.
    • AHEC meets the American Association of Colleges of Osteopathic Medicine (AACOM).
    • Schools of Osteopathic Medicine.

Slide 42

Implementing Knowledge in the Field

• Cheryl Thompson: Partnership development.
• Barbara Kass: Technical assistance/training.
• Margie Shofer: Learning networks.
• Discussion.

Slide 43

Learning Networks

Some definitions...

• Groups of people who share a concern, a set of problems or passion about a topic and who deepen their knowledge in this area by interacting on an ongoing basis.
• Set up for the primary purpose of increasing knowledge.

Slide 44

Learning Network Characteristics

• Domain:
  • Creates common ground, inspires, guides.
• Community:
  • Social structure for learning.
• Practice:
  • Specific knowledge.

Slide 45

Types

• Practice-based.
• Task-based.
• Knowledge-based.

Slide 46

Examples

• High Reliability Organizations (HRO) Learning Network.
• Quality Improvement Organizations (QIOs) Learning Network.
• Medicaid Medical Directors (MMD) Learning Network.

Slide 47

HRO Learning Network

• Task-based: focused on operationalizing HRO concepts.
• 19 organizations from across U.S.
• Operational for 1.5 years:
  • Activities included in-person meetings, web conferences, member extranet.

Slide 48

Successes and Challenges

Successes:

• A core of very engaged members.
• Members very connected.
• Developed HRO Guide.

Challenges:

• Some members not engaged.
• Those that were wanted to go to the next step- but could not agree what that step was.

Slide 49

Lessons Learned

• Ownership.
• Degree of commitment.
• Learning network focus.
• Flexibility.
• Communication Methods.

Slide 50

QIO Learning Network

• Task-based: focused on implementing 2 AHRQ tools.
• QIOs in 16 State 243 providers.
• Two 1-year projects (2010-2011):
  • Activities included in-person meetings, QIO-specific technical assistance calls, national support calls, member extranet.

Slide 51

Successes and Challenges

Successes:

• Training highly rated.
• Implementation in a short period of time.
• Improved process measures.

Challenges:

• Resource intensive.
• Some dropped out.
• Reliance on toolkit experts.

Slide 52

Lessons Learned

• Difficult to determine who is ready for intervention during recruitment.
• Needed to over-recruit to account for drop off.
• Can't assume that learning will continue once network ends.

Slide 53

Medicaid Medical Directors Learning Network

• Practice-based focused on improving quality of care for Medicaid recipients.
• 46 active members.
• Operational since November, 2005:
  • Activities include 3 in-person meetings/year, Web conferences, teleconferences, member extranet, active steering committee, group projects.

Slide 54

Successes and Challenges

Successes:

• Operational since 2005.
• Very engaged with each other and AHRQ (product use, providing input, on committees).
• Truly member owned.
• Group projects.
• Less MMD turnover.

Challenges:

• Future direction.

Slide 55

Lessons Learned

• Extranets can work.
• SC helps with decisionmaking.
• Group needs evolve over time.
• Continually assess AHRQ support.

Slide 56

Overall Lessons: Learning Networks

• Not all knowledge transfer problems are best solved with learning networks.
• Learning Networks in which the members do not interact are not effective.
• Funded support structures are often required to fully enable larger learning networks.
• Learning Networks can take a long time to become successful.

Slide 57

Implementing Knowledge in the Field

• Cheryl Thompson: Partnership development.
• Barbara Kass: Technical assistance/training.
• Margie Shofer: Learning networks.
• Discussion.

Slide 58

Leading Through Innovation and Collaboration

Images: A series of book covers, logos, and screen captures for AHRQ's publications, services, and Web sites are shown.