Reporting in a World of Multiple CAHPS Measures

Select to access the PowerPoint® presentation.

Slide 0

Reporting in a World of Multiple CAHPS Measures

Mark Schlesinger, Yale University
David Kanouse, RAND
September 9, 2012

Images: Consumer Assessment of Healthcare Providers and Systems (CAHPS®) and AHRQ logos appear at the top of the slide.

Slide 1

Overview

• Challenge 1: Availability of measures with overlapping content creates potential for confusion in reporting.
• Challenge 2: Proliferation of domains to be reported creates a demand for "roll-up" measures that summarize across domains.

Slide 2

Challenge 1: Overlapping Content

Example:

• The Clinician & Group Surveys produces the following four measures:
  • Getting timely appointments, care, and information (5 items).
  • How well providers (or doctors) communicate with patients (6 items).
  • Helpful, courteous, and respectful office staff (2 items).
  • Patients’ rating of the provider (or doctor) (1 item).

Slide 3

Challenge 1: Overlapping Content

• Supplemental item sets designed for the Clinician & Group Surveys have additional composite measures and individual items that relate to the core domains; e.g.:
  • Getting timely appointments through e-mail or a Website (HIT composite, 1 item).
  • How well providers communicate about medicines (Health Literacy composite, 4 items).

Slide 4

Challenge 1: Overlapping Content

• New composites and items that address specific topics but also fall within the domains covered by core composites create additional challenges for labeling of composites and reporting:
  • Labels need both to describe content of composite and distinguish it from other composites.
  • Potential for consumer confusion and overload.
• Overlapping measures also create a temptation for sponsors to create their own (non-standard) meta-composites.

Slide 5

Challenge 1: Recommendations

• Do not publicly report items not recommended for reporting; these items are for QI purposes.
• When testing reports with consumers, examine their understanding and interpretation of composites in relation to other potentially related composites, not merely in isolation.
• Don't even think about building your own in-house meta-composite.

Slide 6

Challenge 2: Roll-up Measures

• As the number of performance measures grows, integrating that information becomes more difficult for consumers.
• Example: The Hospital CAHPS Survey contains six composite measures and three individual questions covering highly diverse domains (e.g., communication with nurses and doctors, pain management, discharge information).

Slide 7

Challenge 2: Roll-up Measures

• Integrating information across multiple domains to arrive at an overall evaluation is difficult, and , use of roll-up measures offers a way to make that task easier.
• Use of roll-up measures also tends to increase the number of domains that are explicitly weighed in the decision.
• However, roll-up measures have disadvantages as well as advantages.

Slide 8

Challenge 2: Roll-up Measures

• Roll-up measures may provide consumers with an easy way to make a decision without understanding the dimensions on which options vary.
• Without some kind of tailoring based on consumer preferences or characteristics, roll-up measures tend to be "one size fits all."

Slide 9

Challenge 2: Roll-up Measures

• Some of the drawbacks of roll-up measures may be offset by offering them as a tool along with more detailed measures.
• Providing roll-up measures on a Web site makes it possible to tailor the roll-up to an individual's preferences, e.g., by asking which domains should be underweighted or dropped from consideration.
• Research on the best ways to use roll-up measures in public reports on quality is needed.