Identifying Health Care Quality Measures for Medicaid-Eligible Adults: Background Report

3. Methods

Criteria for Assessment of Measures

Working with the Subcommittee co-chairs, AHRQ and CMS identified five criteria that were relevant to the goals of the Medicaid adult core set, were responsive to public comment themes, and represented sound measurement.

  1. Importance

    This criterion considers whether a measure assessed a condition that had high prevalence, impact on the health care system, and/or impact on health care costs for the Medicaid adult population. An additional consideration was whether the area of measurement had demonstrated gaps in care, either documented in the literature or seen in current performance rates. The measure was also considered important if it was seen as actionable by States, Medicaid health plans, or relevant provider organizations.

  2. Scientific evidence around the issue

    This criterion addresses whether the measure promoted an intervention (e.g., screening, management, or follow-up) or process (e.g., care coordination) that is supported by evidence and clinical guidelines.

  3. Scientific soundness of the measure

    This criterion addresses whether the measure was shown to be valid (credible, measures what it intends to measure) and reliable (the extent to which a measure gives consistent results).

  4. Current use in and alignment with existing programs

    In response to public comment requests, a measure's use (or proposed use) in existing programs was included as a criterion for evaluation. Existing programs included the Initial Core Set of Children's Health Care Quality Measures, HEDIS Health Plan Accreditation, Physician Quality Reporting, the National Quality Strategy aims and priorities, and the Medicare and Medicaid Electronic Health Record Incentive Payment Programs.

  5. Feasibility for State reporting

    As the Medicaid adult core set aims to produce State-level information, this criterion considers whether the measure as currently specified could be feasible for States to report.

Measure Information and Documentation

In order to assess how each measure fared against the five criteria, AHRQ and CMS collected the following information on each measure from the measure owners/developers.

  • Measure description, numerator, denominator, and exclusions.
  • Data sources (e.g., claims, medical records, electronic health records).
  • Description of health importance, prevalence, financial importance, and opportunity for improvement, including what is known about gaps in care and health care disparities.
  • Brief description of the scientific literature, including what is known about effectiveness of the intervention being addressed, and what is known about management and follow-up.
  • Published clinical guidelines relevant to the measure.
  • Validity and reliability results, including a description of the study sample and methods used.
  • Use (or proposed use) in existing programs (Federal or other).
  • Performance rates (most recent and two years prior).

These questions were listed on a measure form that was sent to each measure owner/developer for completion. When these measure forms were not received from owners, forms that had been completed by measure stewards for National Quality Forum endorsement were substituted (if they existed). All forms were compiled and sent to Subcommittee members prior to the Subcommittee's two-day meeting.

Subcommittee Deliberation Process

AHRQ and CMS convened the Subcommittee for an in-person meeting on August 9-10, 2011 in Washington, DC. As was done in the initial meeting, the Subcommittee was divided into four workgroups that addressed areas important to Medicaid-eligible adults: Maternal/Reproductive Health, Overall Adult Health, Complex Health Care Needs, and Mental Health and Substance Use.

Each of the 51 measures in the recommended core set was assigned to one of the workgroups based on its relevance to the topic area. The 43 measures suggested in public comment were assigned to workgroups in the same manner. However, members were advised that these measures would only be moved to a vote if there was an active recommendation by a workgroup for that measure to be added for consideration by the whole Subcommittee.

Each workgroup discussed its assigned measures, reviewing the measures' supporting documentation and public comment results. Workgroups were instructed to strive for consensus on how each of their measures fared against the five criteria. Each workgroup completed a Measure Evaluation Form (Appendix 7) and rated measures as Low, Medium, or High for each criterion. After this exercise, the workgroup discussed each measure as a whole, balancing the different criteria, identifying any trade-offs among the criteria, and coming to consensus on an overall recommendation for the measure.

Following the workgroup reviews, the full Subcommittee convened for voting. Prior to each measure's vote, the chair of the workgroup that evaluated the measure summarized the workgroup's discussion, overall recommendation, and rationale. The Subcommittee then voted using an electronic process that allowed for anonymous voting by each member.

Page last reviewed October 2014
Page originally created December 2010
Internet Citation: 3. Methods. Content last reviewed October 2014. Agency for Healthcare Research and Quality, Rockville, MD.