Patient-Centered Medical Homes: What Do We Know and How Can We Learn More

On September 20, 2011, Michael Parchman, Debbie Peikes, and David Meyers made this presentation at the 2011 Annual Conference. Select to access the PowerPoint® presentation (2.4 MB). Plugin Software Help.

Slide 1

Patient-Centered Medical Homes: What Do We Know and How Can We Learn More

September 20, 2011
AHRQ Annual Meeting

Deborah Peikes, Ph.D.—David Meyers, M.D.—Michael Parchman, M.D.
Stacy Dale, M.P.A., Janice Genevro, Ph.D., Eric Lundquist,
Michael Parchman, M.D., M.P.H., Aparajita Zutshi, Ph.D.

Slide 2

Presenters

• David Meyers, M.D.:
  • Director.
  • Center for Primary Care, Prevention and Clinical Partnerships.
  • Agency for Healthcare Research and Quality.
• Deborah Peikes, Ph.D.:
  • Senior Researcher.
  • Mathematica Policy Research.
• Michael Parchman, M.D.:
  • Senior Advisor for Primary Care.
  • Center for Primary Care, Prevention and Clinical Partnerships.
  • Agency for Healthcare Research and Quality.

Slide 3

Session Objectives

• Provide a brief overview of the patient-centered medical home (PCMH).
• Present preliminary findings from a review on early PCMH evaluations.
• Discuss methodological challenges in evaluating the PCMH.
• Suggest how to refine and improve PCMH evaluations.
• Describe AHRQ resources.

Slide 4

What Are Your Interests?

Who are you?

• Health service researcher.
• Evaluator.
• PCMH implementer.
• Funder of PCMHs.
• Primary care professional.
• Other.

Image: A photograph shows children at their desks at school raising their hands.

Slide 5

The Patient-Centered Medical Home

• A model of primary care delivery that is:
  • Patient-centered.
  • Comprehensive.
  • Coordinated.
  • Accessible.
  • Continuously improved through a systems-based approach to quality and safety.
  • Supported by health information technology (Health IT), workforce development, and payment reform.

AHRQ PCMH Definition: http://www.pcmh.ahrq.gov.

Slide 6

Growing Enthusiasm for the PCMH Model

• 27 multi-stakeholder pilots under way in 18 states.
• In 2010, private insurance plans and state Medicaid programs had enrolled over 5 million patients in PCMH demonstration programs.

Bitton A, Martin C, Landon BE. A nationwide survey of patient-centered medical home demonstrations. JGIM 2010;25:584-92.

Slide 7

Strong Federal Interest in the PCMH

• Centers for Medicare & Medicaid Services (CMS):
  • Multi-Payer Advanced Primary Care Practice (APCP) demonstration:
    • CMS participating in 8 state-based multi-payer demonstrations.
    • 1 million Medicare fee-for-service beneficiaries.
  • Federally Qualified Health Center (FQHC)/APCP demonstration:
    • 3-year demonstration.
    • Special Care Management Fee Payments.
    • ~ 200,000 beneficiaries.

Slide 8

Strong Federal Interest in the PCMH

• CMS.
• Veterans Affairs:
  • Patient Aligned Care Team (PACT) initiative:
    • PCMH model of team-based primary care in all VHA Primary Care sites.
  • 5 million veterans.
• TRICARE:
  • Comprehensive redesign of military health plan based on PCMH.
  • 2 million beneficiaries.
• Indian Health Service.

Slide 9

PCMH Recognition

• National Committee for Quality Assurance (NCQA):
  • PCC-PCMH 2011.
• Utilization Review Accreditation Commission (URAC):
  • Patient-Centered Health Care Home (PCHCH) Practice Standards 2011.
• The Joint Commission:
  • Primary Care Medical Home 2011 Standards and Elements for Performance.
• Accreditation Association for Ambulatory Health Care (AAAHC):
  • 2011 Medical Home Standards.

Slide 10

Image: A map of the United States shows the number of sites recognized as a medical home by NCQA as of December 31, 2010. 1,506 sites are recognized.

Slide 11

But Does It Work?

Slide 12

But Does It Work?

• AHRQ commissioned a systematic review:
  The Medical Home: What Do We Know, What Do We Need to Know?: A Review of the Current State of the Evidence on the Effects of the PCMH Model
  Deborah Peikes, Aparajita Zutshi, Kimberly Smith, Melissa Azur, Janice Genevro, and David Meyers.
• Expected release: December 2011.

Slide 13

So, Does It Work?

Slide 14

So, Does It Work?

• Hang on, hang on, I'm getting there...

Slide 15

But First a Bit of Context

Slide 16

Does What Work?

• It takes time to design an intervention, implement it, evaluate it, analyze the results, prepare a manuscript.
• It takes more time for the manuscript to be peer-reviewed, revised, and published.
• Given that the current conceptualization of the PCMH was not formulated until 2007, we did not expect to find a significant amount of evidence.
• In 2010, only very early results were available.

Slide 17

Does What Work?

• We limited our review to interventions that included a focus on at least 3 of the 5 core elements of the PCMH:
  1. Patient-centered.
  2. Comprehensive.
  3. Coordinated.
  4. Accessible.
  5. Continuously improved through a systems-based approach to quality and safety.

Slide 18

Does What Work?

• We limited our review to interventions that included a focus on at least 3 of the 5 core elements of the PCMH.
• We synthesize evidence from interventions assessed to have used high or moderate quality evaluation and analysis methods.

Slide 19

What We Found

• Almost 500 papers reviewed (498).
• 15 studies representing 13 interventions met initial inclusion criteria.

Slide 20

What We Found

• Almost 500 papers reviewed (498).
• 15 studies representing 13 interventions met initial inclusion criteria.
• Almost no studies were designed to evaluate the PCMH:
  • No studies designed to evaluate the PCMH were available that utilized good evaluation and analysis methods.

Slide 21

What We Found

• Almost 500 papers reviewed (498).
• 15 studies representing 13 interventions met initial inclusion criteria.
• Almost no studies were designed to evaluate the PCMH:
  • No studies designed to evaluate the PCMH were available that utilized good evaluation and analysis methods.

Slide 22

What We Found

• Almost 500 papers reviewed (498).
• 15 studies representing 13 interventions met initial inclusion criteria.
• Only 6 were assessed to have utilized good quality evaluation and analysis methods:
  • All of these evaluations reported results of interventions designed before the emergence of the PCMH model:
    • All of these interventions included aspects of at least 3 of the 5 domains of the PCMH.

Slide 23

Many Often-Cited Interventions Do Not (Yet) Provide Good Quality Evidence

• Group Health Cooperative:
  • Evaluation of only 1 intervention clinic (larger study under way).
• Community Care of North Carolina (3 studies):
  • 1 did not report methods.
  • 1 did not report comparability at baseline.
  • 1 had dissimilar treatment and comparison groups at baseline.
• AAFP National Demonstration Project (TransforMed):
  • Good quality evaluation and analysis methods.
  • Intervention did not test effect of PCMH but rather tested effects of externally facilitated versus self-directed transformation.

Slide 24

Pre-PCMH Interventions That Provide Good Quality Evidence

• Care Management Plus (CMP).
• Geisinger Health System (GHS) ProvenHealth Navigator.
• Geriatric Resources for Assessment and Care of Elders (GRACE).
• Guided Care.
• Improving Mood-Promoting Access to Collaborative Treatment for Late-Life Depression (IMPACT).
• Veterans Affairs Team-Managed Home-Based Primary Care (VA TM/HBPC).

Slide 25

Conclusion

• Despite significant and growing interest in the PCMH, it is a relatively new innovation:
  • The field of PCMH evaluation is thus very young.
  • We should not expect to have a strong evidence base at this time.
• A review of pre-PCMH interventions reveals some positive findings in all aspects of the triple aim and highlights the need for more and better evaluations.

Slide 26

Outcomes

• We believe the PCMH should be evaluated for its effect on all three dimensions of our Nation's triple aim:
  • Quality.
  • Affordability.
  • Experience.

Slide 27

Outcomes

• Quality:
  • Processes of care.
  • Health outcomes.
• Cost and Utilization:
  • Total costs.
  • Hospitalization.
  • Emergency department (ED) use.
• Experience:
  • Patients.
  • Caregivers.
  • Health care professionals.

Slide 28

Limited Evidence from Pre-PCMH Interventions

• Quality:
  • Processes of care: 3 studies.
  • Health outcomes: 3 studies.
• Cost and Utilization:
  • Total costs: 4 studies.
  • Hospitalization: 5 studies.
  • ED use: 3 studies.
• Experience:
  • Patients: 3 studies.
  • Caregivers: 2 studies.
  • Health care professionals: 1 study.

Slide 29

Advanced Sneak Peak at the Results

Image: A photograph of a couple standing with their backs to the camera is shown.

Peikes D, Zutshi A, Smith K, Azur M, Genevro J, Meyers D, The Medical Home: What Do We Know, What Do We Need to Know?: A Review of the Current State of the Evidence on the Effects of the PCMH Model, forthcoming.

Slide 30

What We Know: Quality

• Processes of Care: 3 studies:
  • 1 demonstrated positive effect.
  • 2 inconclusive due to limits in statistical analysis.
• Health Outcomes: 3 studies:
  • 1 strong positive results.
  • 1 moderate positive results.
  • 1 without effect.

Slide 31

What We Know: Cost

• Total Cost: 4 studies:
  • 1 study with cost savings for high-needs patients in year 3:
    • But unfavorable effects for:
      • Low-risk patients all 3 years.
      • All patients the first 2 years.
  • 1 study with total cost increase over 1 year.
  • 2 studies with no statistically significant findings:
    • Had non-statistically significant total cost savings (5-10%).
    • May be due to lack of effect but likely due to lack of statistical power.

Slide 32

What We Know: Utilization

• Hospitalization: 5 studies:
  • 3 documented reduced use (all or high-needs patients).
  • 1 had uncertain effects:
    • Due to issues accounting for clustering.
  • 1 found no statistically significant reductions.
  • Diverse populations, interventions, and time frames make it difficult to combine results in a meta-analysis.
• ED use: 3 studies:
  • 1 had significant reduction in year 2.
  • 1 had uncertain effects:
    • Due to issues accounting for clustering.
  • 1 had no statistically significant findings.

Slide 33

What We Know: Experience

• Patient experience: 3 studies:
  • 2 found improvements, 1 did not.
• Caregiver experience: 2 studies:
  • 1 found improvements, 1 did not.
• Health care professional experience: 1 study:
  • 1 study found no improvement.

Notes: The same study found no statistical improvement in experience in all three categories. These interventions were designed before the current focus on improving patient-centeredness.

Slide 34

Summary of What We Know

• Despite significant and growing interest in the PCMH, it is a relatively new innovation:
  • The field of PCMH evaluation is thus very young.
  • We should not expect to have a strong evidence base at this time.
• A review of pre-PCMH interventions reveals some positive findings in all aspects of the triple aim and highlights the need for more and better evaluations.

Slide 35

Cause for Concern

• A 2010 review found that nearly 60 percent of current demonstrations and pilots did not have a detailed evaluation plan.
• Of those with planned evaluations:
  • Many were conceptualized and funded well after demonstrations and pilots had begun.
  • Only 38 percent were collecting data from a comparison group of practices.
  • Most use pre-post designs, making it impossible to conclude that results are due to the intervention.
  • Many interventions and planned evaluations are underpowered.

Bitton A, Martin C, Landon BE. A nationwide survey of patient-centered medical home demonstrations. JGIM 2010;25:584-92.

Slide 36

AHRQ Response

• Based on our work reviewing the evidence, we developed some practical guidance for evaluating PCMH demonstrations for implementers, evaluators, and funders:
  • Improving Evaluations of the Medical Home: A Decisionmaker's Brief:
    • Available today at the mAHRQetplace and on PCMH.AHRQ.gov.
  • How to Ensure that Studies of the PCMH Can Answer Key Research Questions: Choosing the Appropriate Sample and Sample Size for Medical Home Evaluations:
    • Coming in October.

Slide 37

Reflection

Slide 38

We Face Challenges Assessing Medical Homes

• Early adopters are not typical–what is counterfactual?
• Correlation of outcomes within practices ("clustering"):
  • The interventions change entire practices.
• Limited number of practices in each study.
• High variation in costs and health care service use.
• Hard to improve some outcomes for low-risk patients, so difficult to detect effects.
• These challenges make it hard to determine if the intervention worked (versus random noise).

Slide 39

And the Need Is Great Because the Current Evidence Is Limited

• The literature to date examines effects of the earliest precursors and prototypes of the medical home—the pioneers.

Image: A photograph of a covered wagon is shown.

Slide 40

Lessons Learned: How to Improve Future Evidence So We Can Achieve the Triple Aim

Image: A drawing of a house with a white picket fence and swingset is shown.

Slide 41

Focus Evaluations on Quality, Cost, and Experience

• Among 13 studies, limited coverage of the key outcomes.
• 7 of 13 looked at aspects of all 3.
• 5 of 13 looked at patient experience (less of a focus then).

Image: Three interlocked puzzle pieces are shown.

Slide 42

Include Comparison Practices to Make Evaluations Credible

Study designs are ranked according to the quality of evidence they can produce:

• Excellent: Randomized controlled studies. If well-implemented, changes in outcomes can be attributed to the intervention itself.
• Very good: Matched comparison studies. Practices and patients in the intervention and comparison groups should be similar before the intervention begins in terms of number and specialty of providers, use of health information technology, patient demographics, and pre-intervention values of the outcome measures of interest.
• Poor: Pre-post evaluation. Difficult to conclude that changes observed are due to the intervention.

Slide 43

Recognize That the PCMH Is a Practice-Level Intervention: Adjust for Clustering

• Commission evaluations that account for clustering at two phases:
  • Design Phase—Not doing so will lead to underpowered studies, increasing the chance that we will conclude there was no effect when there was (false negative).
  • Analysis Phase—Not doing so will increase the chance we believe the intervention worked, when it did not (false positive).

Slide 44

Not Accounting for Clustering in Analysis Leads to False Positives

• False positive rates when ignoring the effects of clustering are likely to exceed 65%.

Image: A line graph shows that risk when clustering is accounted for remains at a 10% false positive rates regardless of the Intra-class Correlation Coefficient (ICC); however, with risk when clustering is ignored, the false positive rate rises nearly to 90%.

Peikes D, Dale S, Lundquist D, Genevro J, Meyers D, Building the Evidence Base for the Medical Home: What Sample and Sample Size Do Studies Need? October 2011.

Slide 45

Include as Many PCMH Practices as Possible

Include more practices rather than more patients to be able to detect effects.

Note: These are based on the small number of estimates of clustering and variation in the literature. Ask your evaluators to tailor these to your study context. Assumes an equal number of control practices.

Slide 46

Identify the Right Samples of Patients to Answer Each Evaluation Question

• The medical home alters the way the whole practice operates, but different outcomes must be assessed using different patient samples.
• This increases the likelihood of finding a true effect.

Slide 47

Use High-Risk Patients to Measure Costs

• Interventions are very unlikely to generate large enough cost reductions among all patients for studies to detect them:
  • Cost reductions greater than 5% across all patients are not seen in the literature.
  • Because there is so much variation in costs, it is hard to distinguish effects of programs from noise.
  • Same is true for service use.
• It is easier to detect effects on cost among high-risk patients:
  • There are better opportunities to reduce costs for chronically ill patients.
  • There is less variation in costs in this subsample.

Slide 48

Fewer Practices Are Needed to Detect Effects on Cost in Chronically Ill Patients

Note: These are based on the small number of estimates of clustering and variation in the literature. Ask your evaluators to tailor these to your study context. Assumes an equal number of control practices.

• Detectable effects are similar for hospitalizations and even worse for bed days.

Slide 49

Use All Patients to Assess Quality of Care and Experience

• For quality of care and satisfaction outcomes:
  • There is much less variation in measures that take on a limited number of values (typically true of survey items, quality indicators).
  • It is plausible to alter these outcomes for all patients.
  • With 10 treatment practices, it is possible to detect a roughly 25% change, equivalent to moving the mean from 50% to 63%.
  • With 20 treatment practices, it is possible to detect effects of moving the mean from 50% to 57%.
  • Can measure these outcomes for all patients, but only need to include a fraction of patients at each practice for evaluation.

Slide 50

Rethink the Number of Patients from Whom Data Are Collected for Key Outcomes

• Save money. If including more patients per practice increases data collection costs, it might be worth sampling 100 of the patients in each practice (or even fewer, depending on the outcome). Gathering data from more patients doesn't improve the chance of detecting effects of a given size.

Image: A photograph of a pile of money is shown.

Slide 51

Summary: How Can We Learn More

• Focus evaluations on quality, cost, and experience.
• Include comparison practices that are comparable at baseline.
• Recognize that the PCMH is a practice-level intervention and account for clustering.
• Include as many intervention practices as possible.
• Be strategic in identifying the right samples of patients to answer each evaluation question.
• Rethink the number of patients from whom data are collected to answer key evaluation questions.

Slide 52

Visit PCMH.AHRQ.GOV for Evidence-Based Resources

• Primary Care Medical Home.
• Transforming Primary Care.
• Improving the Quality, Safety, Efficiency, and Effectiveness of U.S. Health Care.

Slide 53

PCMH-Focused White Papers and Decisionmaker Briefs

• Roles of the PCMH and Accountable Care Organizations in Care Coordination.
• Coordinating Care in the Medical Neighborhood.
• Choosing the Right Population and Sample Size for Medical Home Evaluations.
• Serving Adults with Complex Health and Social Support Needs.
• Health IT.
• Patient Engagement.
• Mental Health Integration.
• Objective evidence review (forthcoming).

Slide 54

Resources for Researchers and Decisionmakers Alike

• Foundational Articles.
• Care Coordination Measures Atlas:
  • Provides a framework for evaluating care coordination in primary care settings.
  • Reviews more than 60 validated measures.
• Searchable Citations Database:
  • Approximately 850 citations.
  • Journal articles, reports, policy briefs, newsletters.
  • Search by topic, population, keyword, or bibliographical data.
• Practice Facilitation How-To Guide (forthcoming):
  • How to establish a practice facilitation program to support primary care transformation.

Slide 55

Catalogue of Federal PCMH Activities

• Summarizes PCMH-related work of departments and agencies participating in a Federal PCMH Collaborative, including AHRQ, CMS, Department of Defence (DOD), Health Resources and Services Administration (HRSA), National Institutes of Health/National Cancer Institute (NIH/NCI), Substance Abuse and Mental Health Services Administration (SAMHSA), and the VA.
• Details how agencies are working together to improve primary care delivery system.

Slide 56

For More Information Visit

• http://pcmh.ahrq.gov.

Slide 57

Discussion

• In addition to reflecting on what this information means for you, we are interested in hearing from you about where you think AHRQ should go next to support the further development of the PCMH:
  • Research.
  • Evaluation.
  • Methods.
  • Technical assistance.