Background Report on 2013 Retirement of Measures from the Child Core Set

Appendix D. Example Analytic Report Template for Measures Considered for Retirement

Basic Measure Information

Basic Measure Information Description
1 Measure #  09
2 NQF Measure # 0033
3 Measure Steward NCQA/HEDIS (http://www.ncqa.org)
4 Brief Measure Name and Description
4.a. Measure Name Chlamydia Screening in Women
4.b. Measure Description (brief narrative) The percentage of women ages 16 to 20 that were identified as sexually active and had at least one test for Chlamydia during the measurement year.
5 Numerator At least one chlamydia test during the measurement year as documented through administrative data. A woman is counted as having had a test if she had a claim/encounter with a service date during the measurement year with one or more of the applicable codes.
6 Numerator Exclusions Women who had a pregnancy test during the measurement year, followed within 7 days (inclusive) by either a prescription for isotretinoin (Accutane) or an x-ray. This exclusion does not apply to women who qualify for the denominator based on services other than the pregnancy test alone. 
7 Denominator The eligible population
8 Denominator Exclusions No more than one gap in enrollment of up to 45 days during the measurement year. 

 

Source for this table:  Core Set of Children's Health Care Quality Measures for Medicaid and CHIP (Child Core Set): Summary from Technical Specifications and Resource Manual for Federal Fiscal Year 2013 Reporting (http://www.medicaid.gov/Medicaid-CHIP-ProgramInformation/By-Topics/Quality-of-Care/Downloads/Medicaid-and-CHIP-Child-Core-SetManual.pdf; 3.6 MB)

 

Section # Information Category or Criterion Data Sources Used Data Pertaining to This Measure This column is available for SNAC to add comments
(e.g., rationale for  preliminary score)
1. IMPORTANCE 
1.a Prevalence/Incidence for Focus of Measure
1.a.i Medicaid / CHIP prevalence / incidence for focus of measure  MAX 2009 CAVEAT: Counts exclude enrollees age 20 and under who are not eligible for full Medicaid benefits, who are dually eligible for Medicare, and for whom eligibility category was unknown.

Number of sexually active women in Medicaid/CHIP, ages 16–20 

Total:  2,167,125

1.a.i.1 Variation in prevalence/incidence within Medicaid/CHIP 
Note to SNAC: Whether a difference is a disparity and what type and level of disparities/differences are important to consider in measure retirement are judgment calls best made by the SNAC. For informational purposes, the HHS National Healthcare Disparities report currently considers a difference a disparity if it is statistically significant and a 10% difference.
1.a.i.1.a Variation by race and ethnicity CAVEAT: Counts exclude enrollees age 20 and under who are not eligible for full Medicaid benefits, who are dually eligible for Medicare, and for whom eligibility category was unknown.

Number of sexually active women in Medicaid/CHIP, ages 16–20:
White, non-Hispanic: 871,404
Black, non-Hispanic: 559,505
Hispanic:                   536,864
Other, non-Hispanic: 199,352

1.a.i.1.b Variation by SES Not available
1.a.i.1.c Variation by special health care need Not available
1.a.ii Optional CONTEXTUAL DATA—Prevalence/Incidence for focus of measure for  all U.S. children or on average for U.S. children
(Enter if data are readily available (e.g., from source used to estimate prevalence/incidence among Medicaid/CHIP children)
 
  Not available
1.a.iii Optional CONTEXTUAL DATA—Variation in Prevalence/Incidence U.S. Children overall/on average (Enter if readily available
(e.g., from source used to estimate prevalence/incidence among Medicaid/CHIP children) 
1.a.iii.a Variation by race  and ethnicity Not available
1.a.iii.b Variation by SES Not available
1.a.iii.c Variation by special health care need Not available
1.b Health care utilization and costs related to the focus of the measure
1.b.i. Health care utilization specific to Medicaid / CHIP  Not available
1.b.i.1 Variation in Medicaid/CHIP-specific health care utilization
1.b.i.1.a Variation by race and ethnicity Not available
1.b.i.1.b Variation by SES Not available
1.b.i.1.c Variation by special health care need Not available
1.b.ii Health care costs specific to Medicaid / CHIP  MAX 2009 Total Payments, Women Ages 16–20 (FFS Claims Only):
$4,260,709
1.b.ii.1 Variation in Medicaid/CHIP-specific health care costs
1.b.ii.1.a Variation by race/ethnicity MAX 2009

Payments, Sexually Active Women Ages 16–20 (FFS Claims Only):
White, non-Hispanic: $1,467,201
Black, non-Hispanic: $1,801,925
Hispanic:                    $608,393
Other, non-Hispanic:  $383,190

1.b.ii.1.b Variation by SES Not available
1.b.ii.1.c Variation by special health care need Not available
1.b.iii OPTIONAL INFORMATION FOR CONTEXT (Enter only if readily available)
1.b.iii.1 Health care utilization overall Not available
1.b.iii.2 Health care costs overall Not available
1.c Summary of evidence for focus of the measure 
(NOTE TO SNAC: SNAC agreed with AHRQ's recommendation that only those measures with less than a B grade in 2009 would be subject to AHRQ staff's rapid literature review.) 
  AHRQa 2009 Evidence Grade:  B
1.d Performance on the measure overall and variations
(NOTE  TO SNAC: In interpreting this section, SNAC should attend to different numbers of States reporting in different years for this measure and different approaches to measurement among States as shown below in the Feasibility section.)
1.d.i Average State Medicaid/CHIP performance on the measure (mean, median; and # of reporting States)
1.d.i.1 FFY 2010 CMS, from CARTS CAVEAT: Data is only for the 20 States that reported the measure using HEDIS specifications for all 3 years.
Mean: 42%
Median: 44%
20 States reporting
1.d.i.2 FFY 2011 CMS, from CARTS CAVEAT: Data are only for the 20 States that reported the measure using HEDIS specifications for all 3 years
Mean: 46%
Median: 48%
20 States reporting
1.d.i.3 FFY 2012 CMS, from CARTS CAVEAT: Data are only for the 20 States that reported the measure using HEDIS specifications for all 3 years
Mean: 47%
Median: 49%
20 States reporting
1.d.ii Cross-State Medicaid/CHIP variation in performance on the measure  
1.d.ii.1 FFY 2010 CMS, from CARTS  CAVEAT: Data are only for the 20 States that reported the measure using HEDIS specifications for all 3 years
25th percentile: 25%
75th percentile: 59%
20 States reporting
1.d.ii.2 FFY 2011 CMS, from CARTS CAVEAT: Data are only for the 20 States that reported the measure using HEDIS specifications for all 3 years
25th percentile: 40%
75th percentile: 59%
20 States reporting
1.d.ii.3 FFY 2012 CMS, from CARTS  CAVEAT: Data is only for the 20 States that reported the measure using HEDIS specifications for all 3 years
25th percentile: 39%
75th percentile: 57%
20 States reporting
1.d.iii Overall state Medicaid / CHIP performance on the measure—other data sources IF AVAILABLE
(e.g., HEDIS / NCQA REPORTS)
NCQA, measurement year 2012 Medicaid HMOs:  Average (10th percentile, 90th percentile)
[Provided to SNAC members only]
1.d.iii.1 OPTIONAL— Contextual information on performance
(e.g., commercial performance data on measure from HEDIS/NCQA)
NCQA, measurement year 2012 Commercial HMOs:  Average (10th percentile, 90th percentile)
[Provided to SNAC members only]

Commercial preferred provider organizations:  Average (10th percentile, 90th percentile)
[Provided to SNAC members only]
1.d.iv Variation in State Medicaid/CHIP performance on the measure using other data sources IF AVAILABLE. 
1.d.iv.1 Variation by race and ethnicity Not available
1.d.iv.2 Variation by SES Not available
1.d.iv.3 Variation by special health care needs Not available
OPTIONAL—AHRQ STAFF OBSERVATIONS THAT MAY BE OF INTEREST TO SNAC:
SNAC preliminary member score for this measure on Importance Criterion (scale of 1–9b)   [Enter score here]   [Enter comments here]  

 

a. AHRQ. Background Report on request for public comment on initial, recommended core set of Children's Healthcare Quality Measures for voluntary use by Medicaid and CHIP Programs. http://www.ahrq.gov/policymakers/chipra/overview/background/tables.html. Accessed January 7, 2014. 
 

Section # Information Category or Criterion Data Sources Used Data Pertaining to This Measure This column is available for SNAC to add comments
(e.g., rationale for  preliminary score)
2. SCIENTIFIC ACCEPTABILITY 
2.a Reliability   NCQA, measurement year 2012     [Provided to SNAC members only]
2.b Validity Not available
2.c Medicaid/CHIP Program Deviation from technical specifications provided by CMS: Number (%) of reporting States with a deviation, by Federal fiscal year
2.c.i FFY 2010 CMS Not available 
2.c.ii FFY 2011 CMS Not available
2.c.iii FFY 2012 CMS 0 (0%)
OPTIONAL—AHRQ STAFF OBSERVATIONS THAT MAY BE OF INTEREST TO SNAC:
SNAC preliminary member score for this measure on Scientific Acceptability (scale of 1–9) [Enter score here]   [Enter comments here]  

 

b. In the Modified Delphi approach that we are using, the scale of 1–9 is typically broken into three categories: a score of 1–3 is low, a score of 4–6 is considered medium, and a score of 7–9 is considered high. For more information about the use of Delphi scoring to select quality measures, see the following:  

Schuster MA, Asch SM, McGlynn EA, et al. Development of a quality of care measurement system for children and adolescents. Methodological considerations and comparisons with a system for adult women. Arch Pediatr Adolesc Med. Nov 1997;151(11):1085-1092. 

Mangione-Smith R, Schiff J, Dougherty D. Identifying children's health care quality measures for Medicaid and CHIP: An evidence-informed, publicly transparent expert process. Acad Pediatr. May-Jun 2011;11(3 Suppl):S11-21. 

Jeffs L, Law M, Straus S, et al. Defining quality outcomes for complex-care patients transitioning across the continuum using a structured panel process. BMJ Qual Saf. Jul 12 2013;22:1014-1024. 

Davies S, Romano P, Schmidt E, et al. Assessment of a novel hybrid Delphi and Nominal Groups technique to evaluate quality indicators. Health Serv Res. Dec 2011;46(6 pt 1):2005-2018.  

 

Section # Information Category or Criterion Data Sources Used Data Pertaining to This Measure This column is available for SNAC to add comments
(e.g., rationale for  preliminary score)
3. FEASIBILITY 
3.a.i Number (%) of States, including District of Columbia, reporting, by fiscal year
CAVEAT: If a State doesn't report, SNAC should not necessarily infer that there is a technical/feasibility problem with the measures. Some States may not wish to participate at all or have to be selective about resources. 
3.a.i.1 FFY 2010 CMS, from CARTS 21 (42%)
3.a.i.2 FFY 2011 CMS, from CARTS 32 (63%)
3.a.i.3 FFY 2012 CMS, from CARTS 35 (70%)
3.a.ii Number (%) of State Medicaid/CHIP programs reporting the measure at least once during the 3 fiscal years CMS, from CARTS 37 (74%)  
3.a.iii Number (%) of State Medicaid/CHIP programs reporting the measure all 3 years CMS, from CARTS 20 (40%) 
3.a.iv Number (% of reporting states) by program in 2012 CMS, from CARTS 35 (100%)
3.a.iv.1 Medicaid only CMS, from CARTS 5 (14%)
3.a.iv.2 CHIP only CMS, from CARTS 5 (14%)
3.a.iv.3 Combined CMS, from CARTS 25 (71%)
3.b State Medicaid/CHIP program challenges experienced with data collection and/or reporting (State reported) in 2012
3.b.i. Data not available CHIPRA TA Mailbox 8
3.b.ii Population not covered 0
3.b.iii Sample size too small 0
3.b.iv Other  10
3.b.v Not specified  0
3.c State reporting challenges reflected in the TA requests submitted to the CHIPRA TA mailbox
3.c.i Number of TA requests re measure for period May 2011–May 2013 and number of States making requests CMS 2
3.c.ii TA topics for the measure  CMS Calculation of denominator 
3.d Data source for the measure FFY 2013 CMS Resource Manual and Technical Specifications Administrative claims
3.e Alignment with measures for other Federal programs
(For each measure, report the name of the measure and the Federal program using the measure)
CAVEAT:  The HHS measure inventory provides information on measures in other HHS programs that may include measures similar to one or more Child Core Set measures. In addition, the National Quality Measures Clearinghouse includes information about use for each measure it lists. Both databases are available at http://www.qualitymeasures.ahrq.gov/. We did not have the resources to do a comprehensive search of either database in time for the 2013 SNAC process.
3.e.i Name of other measure(s) CMSc Same
3.e.ii Federal program(s) using the measure CMS Adult Medicaid Core Set; CMS Electronic Health Record Incentive Program for Eligible Professionals; Physician Quality Reporting System; proposed in Value-Based Payment Modifier
OPTIONAL—AHRQ STAFF OBSERVATIONS THAT MAY BE OF INTEREST TO SNAC:
SNAC preliminary member score for this measure for Feasibility (scale of 1–9)

[Enter score here]

[Enter comments here]

 

c. State Health Officials Letter #13-002. Children's Core Set of Health Care Quality Measures. January 24, 2013. http://www.medicaid.gov/Federal-PolicyGuidance/downloads/SHO-13-002.pdf (165.64 KB).  Accessed September 20, 2013.

 

Section # Information Category or Criterion Data Sources Used Data Pertaining to This Measure This column is available for SNAC to add comments
(e.g., rationale for  preliminary score)
4 USABILITY
4.a Action(s) Taken by States on measure topic      CAVEAT:  It is important to note that what is listed on the grid of State activities is only the tip of the iceberg as to what is really happening out there. Because of resource constraints and, more importantly, our inability to survey all Medicaid and CHIP programs for examples of quality improvement strategies, AHRQ had to limit its search to only projects with documentation on the Web or elsewhere.   
4.a.i State Medicaid/CHIP efforts   [Preliminary findings shared with SNAC only]
4.a.ii Other State program initiatives (not Medicaid/CHIP)   Not available
4.b Improvability (potential to improve)—summary of studies demonstrating that performance can be improved   [Preliminary findings shared with SNAC only] 
OPTIONAL—AHRQ STAFF OBSERVATIONS THAT MAY BE OF INTEREST TO SNAC:
SNAC preliminary member score for this measure for Usability  (scale of 1–9) [Enter score here]  

[Enter comments here]

 

SNAC member comments:
Observations, concerns, questions about the measures not covered by domains in the template  
Taking all criteria into consideration, should measure be retired from the Child Core Set?
(Circle YES or NO)
Yes No   

Note: This column is available for SNAC to add comments (e.g., rationale for preliminary score)" was space offered to individual SNAC members as they worked through their recommendations. 

Key: AHRQ =  Agency for Healthcare Research and Quality; CARTS = CHIP Annual Reporting Template System; CHIP = Children's Health Insurance Program; CHIPRA = Children's Health Insurance Program Reauthorization Act of 2009; CMS = Centers for Medicare & Medicaid Services; FFS = fee for service; FFY = Federal fiscal year; HEDIS = Healthcare Effectiveness Data and Information Set; HHS = U.S. Department of Health and Human Services; MAX = Medicaid Analytic eXtract; NCQA = National Committee for Quality Assurance; NHIS = National Health Interview Survey; NSCH =  National Survey of Children's Health; SES = socioeconomic status; SNAC = National Advisory Council on Healthcare Research and Quality 2013 Subcommittee on Quality Measures for Children's Healthcare for Medicaid and CHIP; TA = technical assistance. 

Return to Contents
Proceed to Next Section

Page last reviewed February 2014
Internet Citation: Background Report on 2013 Retirement of Measures from the Child Core Set: Appendix D. Example Analytic Report Template for Measures Considered for Retirement. February 2014. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/policymakers/chipra/measure_retirement/measure_retirementapd.html