Using Patient Experiences Surveys in Health Plan and Practice Evaluati

Slide presentation from the AHRQ 2009 conference.

On September 19, 2009, Sarah Hudson Scholle made this presentation at the 2009 Annual Conference. Select to access the PowerPoint® presentation (626 KB) (Plugin Software Help).

Slide 1

Using Patient Experiences Surveys in Health Plan and Practice Evaluation
Sarah Hudson Scholle
Assistant Vice President, Research


Slide 2


  • NCQA
  • Health plan accreditation model and CAHPS
  • Incorporating patient experiences surveys into evaluation of physician practices


Slide 3

NCQA: A Brief Introduction

  • Private, independent non-profit
    health care quality oversight organization founded in 1990
  • Committed to measurement,
    transparency and accountability
  • Unites diverse groups around common goal: improving health care quality


Slide 4

NCQA Health Plan Accreditation

  • Key Components
    • Rigorous on-site review of key systems and processes
    • Evaluation of clinical performance through HEDIS® measures
    • Member experience surveys - CAHPS® 4.0H for adults and children


Slide 5


Clinical Performance(HEDIS) + Member Experience (CAHPS) =43%
Health Plan Systems(Accreditation Standards) = 57%

Accreditation is Performance-based:NCQA Accreditation is the only health plan Accreditation that requires reporting on clinical performance


Slide 6

What is NCQA's HEDIS?

The Healthcare Effectiveness Data and Information Set:

  • Process and outcomes measures
  • Standardized member experience surveys
  • Used by commercial, Medicare, and Medicaid plans alike
  • Allows plan-to-plan comparisons by quality, not just by price


Slide 7

CAHPS® 4.0H Surveys Development and Reporting

  • Quality Compass® (plan-to-plan comparisons)
  • State of Health Care Quality Report
  • National CAHPS Benchmarking Database (NCBD)
  • Other products-report cards, Quality Dividend Calculator, etc.


Slide 8

NCQA Recognition Programs

  • >14,000 Physicians Recognized nationality across all Recognition Programs
  • Clinical ptograms
    • Diabetes recognition program(DRP)
    • Heart.Stroke recognition program(HSRP)
    • Back pain recognition program(BPRP)
  • Medical practice process and structural measures
    • Physician practce connections
    • Physician practice connections-patient-centered medical home(PPC-PCMH)


Slide 9

Goals for Physician Practice Connections (PPC)

  • Evaluate systematic approach to delivering preventive and chronic care (Wagner Chronic Care Model)
  • Build on IOM's recommendation to shift from "blaming" individual clinicians to improving systems
  • Create measures that are actionable for physician practices
  • Validate measures by relating them to clinical performance and patient experience results


Slide 10

Theoretical Frameworks Informing Development of PPC-PCMH
Based on best available empiric evidence in each area and on testing of reliability and validity of elements in field tests using on site audit as "gold" standard

Chronic Care Model Patient Centered Care Cultural Competence Medical Home 
Clinical information Systems Respect Patient Values Culturally competent interactions Personal physician 
Decision Support Accessible  Language services Physician directed team 
Patient Self-Management Family-Centered  Reducing disparities Whole person orientation 
Delivery System Redesign Continuous   Care is coordinated and integrated 
Community Linkages Coordinated   Quality and safety 
Health Systems Community Linkages  Enhanced access 
 Culturally Appropriate    
 Emotional Support    
 Information and Education    
 Physical Comfort   
 Quality Improvement  

First contact-comprehensive-continuous-coordinated


Slide 11

Adapting PPC for the Patient-Centered Medical Home

  • New PPC-PCMH version released in January 2008
    • Aligned standards with Joint Principles
    • Incorporated critical attributes of PCMH
    • Defined foundational elements ("must pass" requirements)
  • PPC-PCMH endorsed by ACP, AAFP, AAP, AOA, other specialties and PCPCC for use in demos

Endorsed by National Quality Forum Sept 2008 (as “Medical Home System Survey”)

Slide 12

PPC-PCMH Content and Scoring

**Must Pass Elements


Slide 13

Examples of Initiatives Using PPC-PCMH

  • Multi-payer - Colorado, Pennsylvania, Rhode Island
  • State-wide - Pennsylvania, Vermont, Maine
  • Single payer - EmblemHealth, Humana
  • Government - Medicare, New York City, Louisiana


Slide 14

Significant PPC-PCMH Issues for Future

"Measures of Meaningful Use" inside

  • How to further assess patient-centeredness, including patient survey results?
    • How to engage patients?
    • How to make name resonate positively?
  • When should performance results be part of scoring?
  • How to adapt to promote quality and cost gains across settings?
    • Primary care-subspecialty
    • Physician-hospital, other facilities
  • How to streamline requirements, documentation?
    • For all practices
    • For practices renewing


Slide 15

Timeline, Evolution of PPC-PCMH


Solicit input: Website, calls, meetings

Oct. - Convene Advisory Committee; develop draft changes

April -Review draft changes with CPP; Public Comment

July to Oct. - Advisory

Dec. - CPP, BOD approval


Slide 16

Barriers to Incorporating
Patient Experiences Results

  • Lack of agreement on core content
  • Whether existing tools are able to detect change in performance
  • The burden of conducting patient experiences surveys
  • Conflicting priority of accountability versus quality improvement goals
  • Structure/process versus outcome scoring


Slide 17

Aims of Proposed Research

  • Identify a core set of survey items
  • Explore feasibility of alternative sampling and other data collection strategies
  • Examine the impact of alternative scoring approaches in blending results from the PPC-PCMH and patient experience surveys


Slide 18

Getting to Core Set of Measures

  • Identify possible domains/items
    • Access
    • Communication
    • Coordination
    • Shared decision-making
    • Self Management
    • Whole person orientation
  • Ranking exercise involving broad stakeholder participation
  • Review of psychometric properties
  • Recommendations to PPC-PCMH review panel


Slide 19

Getting to Standardized Data Collection

  • Profiles of existing efforts to collect, analyze and report patient experiences survey results
    • Purpose of survey
    • Unit of Analysis
    • Sampling
    • Data collection
    • Analysis
    • Quality assurance
    • Tool
    • Cost/Finance
  • Review of literature on impact of different methods
  • Recommendations to PPC-PCMH review panel


Slide 20


  • Patient's views are critical to evaluations of health care, both at health plan and physician level
  • Standardized tools and methodology needed to allow fair, national comparisons
  • Feasibility and relevance to key stakeholders must be addressed


Slide 21

  • For more information:
  • Sarah Hudson Scholle, MPH, DrPH
  • 202-955-1726
Current as of December 2009
Internet Citation: Using Patient Experiences Surveys in Health Plan and Practice Evaluati. December 2009. Agency for Healthcare Research and Quality, Rockville, MD.