AHRQ NAC Subcommittee on Children's Healthcare Quality Measures for Medicaid and CHIP Programs

Progress Report

This is the text version of a slide presentation by Rita Mangione-Smith and Jeffrey Schiff titled AHRQ National Advisory Council (NAC) Subcommittee on Children's Healthcare Quality Measures for Medicaid and CHIP Programs presented on September 17, 2009.

This is the text version of a slide presentation by Rita Mangione-Smith and Jeffrey Schiff titled AHRQ National Advisory Council (NAC) Subcommittee on Children's Healthcare Quality Measures for Medicaid and CHIP Programs presented on September 17, 2009.

Select to access the Microsoft PowerPoint file Microsoft PowerPoint version - 290 KB .


  1. AHRQ NAC Subcommittee on Children's Healthcare Quality Measures for Medicaid and CHIP Programs
  2. Children's Health Insurance Program (CHIP)
  3. Potential Impact of Core Measures Identification Work
  4. Our Charge
  5. Conceptual Framework
  6. Scope for Core Measurement Set
  7. Consensus on Criteria Definitions: Validity
  8. Consensus on Criteria Definitions: Feasibility
  9. Consensus on Criteria Definitions: Importance
  10. Consensus on Criteria Definitions: Importance (continued)
  11. Consensus on Criteria Definitions: Transparency
  12. Delphi Scoring Process Completed by Subcommittee Members
  13. Delphi Scoring Process Completed by Subcommittee Members (continued)
  14. 65/121 Measures Passed Delphi Round 2
  15. Meeting 9/17-9/18: Goal is to identify a parsimonious, core, grounded, and balanced set of measures.
  16. Process for this Meeting: Nomination and VFI process
  17. Process for this Meeting: Pruning
  18. Process for this Meeting: Creation of a unified "whole"

Slide 1.

AHRQ NAC Subcommittee on Children's Healthcare Quality Measures for Medicaid and CHIP Programs

Cover slide of presentation AHRQ NAC Subcommittee on Children's Healthcare Quality Measures for Medicaid and CHIP Programs
Rita Mangione-Smith and Jeffrey Schiff September 17 & 18, 2009

Slide 2.

Children's Health Insurance Program (CHIP)

Slide contains a photo of a dog looking at a computer screen.
Slide contains a photo of a dog looking at a computer screen.

Slide 3.

Potential Impact of Core Measures Identification Work: 48% of America's Children Under 19

Slide presents a pie chart showing percentage of children covered by insurance programs: Privately insured, 31.8, 41%; Medicaid , 29.9, 38%; CHIP, 5.3, 7%; Uninsured, 11, 14%.
Slide presents a pie chart showing percentage of children covered by insurance programs: • Privately insured, 31.8, 41%. • Medicaid , 29.9, 38%. • CHIP, 5.3, 7%. • Uninsured, 11, 14%. Sources: AHRQ based on: Coverage estimates: Based on 2008 national participant and spending data derived from CMS and U.S. Census Current Population Survey data sources, reported in National Center on Children in Poverty, http://www.nccp.org/profiles/index_32.html. CHIP estimates are for number of children in separate SCHIP programs. Medicaid estimates include children in Medicaid SCHIP programs. Coverage estimates reflect Medicaid and CHIP enrollees whether or not they received health care services. Total number of children under 19 interpolated from U.S. Census Bureau figures for number of children 17 and under (73.9 million) and 19 and under (83 million).

Slide 4.

Our Charge

Core set of measures that when TAKEN TOGETHER—USED TO ESTIMATE THE OVERALL NATIONAL QUALITY OF HEALTH CARE FOR CHILDREN.
Core set of measures that when "TAKEN TOGETHER—USED TO ESTIMATE THE OVERALL NATIONAL QUALITY OF HEALTH CARE FOR CHILDREN."

Slide 5.

Conceptual Framework—Guiding Determination of the Scope for Core Measurement Set

Grounded Measures —> Intermediate Measures —> Aspirational Measures. Lean towards recommending more grounded measures: Grounded: 10-25, currently feasible, many already in place; Intermediate category: #?, good specifications, some States already using them; Aspirational: needed measures to fill in the gaps.
Grounded Measures —> Intermediate Measures —> Aspirational Measures • Lean towards recommending more grounded measures: ◦Grounded: 10-25, currently feasible, many already in place. ◦Intermediate category: #?, good specifications, some States already using them. ◦Aspirational: needed measures to fill in the gaps.

Slide 6.

Scope for Core Measurement Set

Text Description is below the image.
1.Must be realistic about staffing/funding needs for collecting/analyzing/reporting available data. 2. Comprehensive effort to find good measures for all service categories, duration of enrollment, and other aspects of care required by the legislation; however, if no good measures currently exist for a given aspect of care, a measure will not be recommended for the core set. 3. Include measures not currently used by Medicaid/CHIP. ◦e.g., State and national measurement efforts. 4. Choose measures that are actionable. ◦There should be clear steps a State, plan, or provider can take to improve on performance—the measure should inform what these steps need to be.

Slide 7.

Consensus on Criteria Definitions: Validity

Text Description is below the image.
•Measures must be supported by scientific evidence or, where evidence is insufficient, by expert consensus. •Measures must support a link between: ◦Structure and outcomes of care. ◦Structure and processes of care. ◦Processes and outcomes of care. •The measure must represent an aspect of care that is under the control of health care providers and systems. •The measure should truly assess what it purports to measure. •Measures supported by evidence from unpublished data should be considered for inclusion.

Slide 8.

Consensus on Criteria Definitions: Feasibility

Text Description is below the image.
•The data necessary to score the measure must be available to State Medicaid and CHIP programs: ◦Administrative data, medical records data, survey data. •Detailed specifications must be available for the measure that allow for reliable and unbiased scoring of the measure across States and institutions.

Slide 9.

Consensus on Criteria Definitions: Importance

Text Description is below the image.
•The measure should be actionable. •Cost of the condition to the Nation should be high. •Health care systems are clearly accountable for the quality problem assessed by the measure. •The extent of the quality problem should be substantial. •There should be documented variation in performance on the measure. •The measure should be representative of a class of quality problems: "sentinel measure" of quality of care provided for preventive care, mental health care, or dental care, etc.

Slide 10.

 

Consensus on Criteria Definitions: Importance (continued)

Text Description is below the image.
•The measure assesses an aspect of health care where there are known disparities. •The core set should represent a balanced portfolio of measures and be consistent with the intent of the legislation. •Improving on performance for the core set of measures should have the potential to transform care for our Nation's children.


Slide 11.

 

Consensus on Criteria Definitions: Transparency

Text Description is below the image.
•For all measures recommended for inclusion in the core set: ◦The level of scientific evidence supporting the measure will be reported. ◾Example: USPSTF grades A, B, C, or I; Level I, II, III ◦The level of expected burden for obtaining the data needed to score the measure will be reported. ◾Low, moderate, high


Slide 12.

 

Delphi Scoring Process Completed by Subcommittee Members

Text Description is below the image.
•Measures scored on a 9-point scale: ◦7-9 Measure definitely valid, feasible, and important. ◦4-6 Measure has uncertain validity, feasibility, or importance. ◦1-3 Measure is not valid, feasible, or important. •Measures with a Validity score of ≥7 pass. •Measures with Feasibility and Importance scores ≥4 pass. Delphi Round 1 completed at July 2009 meeting. •Assessed validity, feasibility, and importance of measures in use by State Medicaid and CHIP programs.


Slide 13.

 

Delphi Scoring Process Completed by Subcommittee Members (continued)

Text Description is below the image.
Delphi Round 2 completed, Monday September 14th in preparation for second meeting. •Four groups of measures were assessed: ◦Measures that had passing scores for validity, feasibility, and importance in Delphi Round 1. ◦Measures that were judged to be "controversial" during scoring for validity and feasibility in Delphi Round 1. ◦Measures identified through environmental scans but that were not included on the original list of measures scored during Delphi Round 1. ◦Measures nominated by SNAC members, Federal partner agencies, and the public, between July 24th and August 24th.


Slide 14.

 

65/121 Measures Passed Delphi Round 2

Text Description is below the image.
•Prevention and Health Promotion: 27/50 •Management of Acute Conditions: 16/25 •Management of Chronic Conditions: 18/31 •Family Experiences with Care: 3/6 •Most Integrated Health Care Systems: 0/1 •Availability of Services: 0/5 •Health Status: 0/1 •Duration of Enrollment: 1/2


Slide 15.

 

Meeting 9/17-9/18: Goal is to identify a parsimonious, core, grounded, and balanced set of measures.

Text Description is below the image.
•65 measures passed Delphi Round 2 and will be discussed for final inclusion at the meeting. •Balancing grid developed to track the following constructs: ◦Ages covered by the measure ◦Disparities in performance on the measure ◦Sites/Types of care addressed by the measure ◦Aspects of the Care Continuum addressed by the measure ◾e.g. Outpatient, Inpatient, Mental Health, Dental ◦Aspects of the System Continuum addressed by the measure ◾e.g. Structure, Process, Outcomes, Efficiency ◦Types of Entities Using the measure ◾e.g. Medicaid programs, health plans, clinics/providers, researchers ◦The Data Source for the measure ◾e.g. Administrative, Medical Records, Survey


Slide 16.

 

Process for this Meeting: Nomination and VFI process

Text Description is below the image.
(VFI = Validity, feasibility, and importance) Review interval process. Vote to select only one of "nearly same" measures.


Slide 17.

 

Process for this Meeting: Pruning

Text Description is below the image.
•Discuss and confirm "domains" on the balancing grid. •Reaffirm the concepts of a "core, grounded, parsimonious" set of measures. •Discuss measures by legislative category (e.g., prevention, health promotion—PHP). •Rank order within each category or subcategory. •Retain top 33% in each category. •Present this set on the balancing grid after the meeting today.


Slide 18.

 

Process for this Meeting: Creation of a unified "whole"

Text Description is below the image.
•Review balancing grid from Thursday. •Identify and acknowledge holes. •Vote/rank remaining measures as a whole set. •Review on the balancing grid for varying numbers of measures (10-15-20-25). •Discuss the number for a parsimonious set. •Vote for a set by size. •Record group comments about the nominated core set.
Current as of May 2013
Internet Citation: AHRQ NAC Subcommittee on Children's Healthcare Quality Measures for Medicaid and CHIP Programs: Progress Report. May 2013. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/policymakers/chipra/chipraarch/snac091709/msschiff/mss0917slides-contents.html