National Advisory Council Subcommittee: Identifying Quality Measures for Medicaid-eligible Adults

III. Methods

The proposed initial core quality measure set for adults in Medicaid as required under the Affordable Care Act was developed using a transparent and evidence-based process, with input from State Medicaid programs and other stakeholders. The Subcommittee held a public meeting and considered public comments. The Subcommittee's advice was reported to the Chair of AHRQ's National Advisory Council and considered further by Medicaid officials and staff in the HHS' Office of the Secretary prior to the public posting in the Federal Register. The Subcommittee will reconvene in mid-2011 to review the public comments and to provide advice to HHS on the final initial core measures set to be published by January 1, 2012.

Creation of the Subcommittee

In September 2010, the AHRQ Director approved a charter creating the National Advisory Council for Healthcare Research and Quality Subcommittee on Identifying Quality Measures for Medicaid Eligible Adults (go to Appendix 3 for a list of members and Appendix 4 for a list of staff). The Subcommittee included individuals with expertise in measurement as well as the health and health care needs of Medicaid-eligible adults. Elizabeth A. McGlynn, Ph.D., Associate Director for RAND Health, and Foster C. Gesten, M.D., FACP, Medical Director at the New York State Department of Health, served as co-chairs. Six State Medicaid programs (Florida, New York, Pennsylvania, Missouri, Colorado, and Washington) were represented on the panel. The Subcommittee held a 2-day public meeting, preceded by a Web conference to orient members to the Subcommittee's task.

Web Conference

The Subcommittee was convened by Web conference. Nancy J. Wilson, M.D., M.P.H., Senior Advisor to the Director of AHRQ, introduced the committee's charge, the process for identifying measures, and the timeline for public comment and publication of the final set. The Subcommittee was charged with providing guidance on an initial core measurement set consistent with the legislative requirement that the list include "existing adult health quality measures that are in use under public and privately sponsored health care coverage arrangements, or that are part of reporting systems that measure both the presence and duration of health insurance coverage over time, that may be applicable to Medicaid eligible adults."

The Subcommittee was encouraged to consider several desired attributes for the measurement set. These attributes included a proposed measurement set that addresses the critical issues facing the diverse populations served by Medicaid; provides information to meet the legislative needs for State-level reporting as well as point to opportunities for quality improvement; offers information relevant to all insurance categories; is parsimonious; and meets other regulatory requirements where feasible. An example of such requirements are measures currently designated for inclusion in the Meaningful Use incentive payments for eligible health care professionals and hospitals that adopt certified electronic health record technology under the Health Information Technology for Economic and Clinical Health (HITECH) Act of 2009.

Subcommittee members noted that the selection of measures depended on how the measures would be used, and in particular they raised concerns about the likelihood that measures would be used for payment. The Subcommittee was encouraged to consider the primary use of the measures as described in the legislation (to provide a report to Congress on the quality of care of adult Medicaid enrollees in each State) but acknowledged the need for the measures to be useful for quality improvement efforts at the State level. Another point of discussion was the size of the core set. Some members noted the potential burden on States of reporting a large number of measures. The Subcommittee was encouraged to consider the feasibility of measure reporting, in terms of the number of measures and the availability of data, while considering the infrastructure capacity building through other Federal initiatives such as the HITECH Act.

The Subcommittee was encouraged to consider the National Quality Forum (NQF) criteria for evaluating quality measures in proposing the core set. These criteria include:

  1. Importance to measure and report (such as high impact, opportunity for improvement, evidence that supports the focus of measurement).
  2. Scientific acceptability of measure properties (reliability, validity).
  3. Usability for public reporting and quality improvement.
  4. Feasibility (including data or information showing that the measure can be implemented).

Measures Inventory

To aid the Subcommittee deliberations, AHRQ, with help from CMS, compiled an inventory of measures, starting with measures currently used by CMS programs. The list was supplemented with measures submitted by the Medicaid Medical Director's Learning Network and all currently relevant measures from the NQF. In addition, the Subcommittee members also recommended measures for inclusion. Through a contract with AHRQ, staff from the National Committee for Quality Assurance (NCQA) assembled the measures inventory. The inventory provided information on measure characteristics (including numerator, denominator, data source, etc.). To address the legislative requirement for the measures to be currently in use, the inventory included information on whether the measure was currently used by Medicaid programs or health plans in the State (including the number of States known to use the measure), or whether the measure was in use by other Federal programs (such as in CMS programs for Medicare Advantage plans or the Physician Quality Reporting Initiative).

A total of 985 measures were included in the inventory. The inventory identified whether measures were most relevant to the care needs of either overall adult health, maternal and reproductive health, mental health and substance use or adults with complex health care needs. To provide a framework for considering measures which addressed these types of health care needs, the measures were grouped according to the domains recommended by the Institute of Medicine (IOM) for its framework for national health care quality reports (IOM, 2010). These domains include safe, timely, effective, efficient, access, patient and family centeredness, care coordination, and infrastructure capabilities for health care, along with the cross-cutting dimensions of health care equity and value. (Go to Appendix 5 for the IOM framework.)

Subcommittee In-Person Meeting

The Subcommittee met in Washington, D.C., October 18-19, 2010, beginning with a welcome by the AHRQ Director and the Chief Quality Officer of the Center for Medicaid, CHIP and Survey & Certification. Opening presentations provided context for the Subcommittee's deliberations. Lekisha Daniel-Robinson, M.S.P.H., from the Family and Children's Health Programs Group, and Anita Yuskauskas, Ph.D., from the Disabled and Elderly Health Programs Group (both within the CMS Center for Medicaid, CHIP and Survey & Certification) offered information about key Medicaid populations. Sarah Hudson Scholle, Dr.P.H., M.P.H., Vice President for Research at the National Committee for Quality Assurance, reviewed information on quality reporting in Medicaid programs and described the contents of the inventory. (Appendixes 6 through 9)

Given the large number of measures available for consideration, the Subcommittee divided into workgroups to review the measures lists. The workgroups focused on four dimensions of health care related to adults in Medicaid: Maternal/Reproductive Health, Overall Adult Health, Complex Health Care Needs, and Mental Health and Substance Use (go to Appendix 3 for workgroup members). Each workgroup received a preliminary measures list from the inventory relevant to their population so they could begin discussions. The workgroups could nominate additional measures if they were not found on the lists. For the Maternal Reproductive Health workgroup, the preliminary list included any measures that applied to women's health and had evidence of use in Medicaid or another program. Similarly, for the Mental Health and Substance Use workgroup, the list included all measures relevant to those conditions and with evidence of any use. Because of the large number of measures that could apply for the Adult Health workgroup, the preliminary list included only measures that were in use in Medicaid. For the Complex Health Care Needs workgroup, the preliminary list focused on the following cross-cutting topics that had any evidence of current use: Functional status, care coordination, health system infrastructure, and avoidable hospitalizations.

The workgroups were asked to identify key issues for their populations of interest, nominate measures that address those issues, and consider alignment with the IOM framework. In addition, the workgroups were asked to consider the scientific evidence base for the measure, the relevance to Medicaid, the feasibility of the measure, and whether the measure would provide actionable information to CMS and to the States.

After deliberations, the workgroups reported back to the full Subcommittee for input and analysis of cross-cutting priorities. The workgroups reconvened on the second day and prepared a revised list of measures. The full Subcommittee voted on the measures through an anonymous electronic voting system. For each measure or measure group, Subcommittee members voted on their agreement with including the measure in the initial core set using a response scale of "strongly agree," "agree," "neutral," "disagree," and "strongly disagree." The Subcommittee then reviewed a list including all measures where at least 50 percent of the voting Subcommittee members marked "strongly agree" or at least 70 percent of the voting Subcommittee members marked "agree" or "strongly agree." The Subcommittee was encouraged to consider whether additional measures were needed to achieve balance for all four populations' groupings. Subcommittee members were given the opportunity to nominate measures that had been considered but did not make it onto the list.

Public Input

Multiple opportunities for public input were provided as part of this process. The 2-day in-person meeting was open to the public and provided opportunities each day for anyone to make formal public comments (go to Appendix 10 for public comments). On October 28, 2010, CMS led a State listening session for Medicaid/CHIP officials and representatives of other State programs to comment on the draft initial core measure set.

Page last reviewed October 2014
Page originally created September 2012
Internet Citation: III. Methods. Content last reviewed October 2014. Agency for Healthcare Research and Quality, Rockville, MD.