The first task of the Home and Community-Based Services (HCBS) Measure Scan Project is to identify existing measures that either directly assess State Medicaid HCBS programs in specified key focus areas or that could be adapted for this purpose. This report presents the proposed methodology for accomplishing this task.
The report was funded by the Agency for Healthcare Research and Quality (AHRQ) under Contract No. HHSA29020060042C.
By Sara Galantowicz, The MEDSTAT Group, Inc.
Inclusion and Exclusion Criteria
A: Deficit Reduction Act, Section 6086(b): "Quality of Care Measures"
B: List of Medicaid Home and Community-Based Services
C: Database Descriptions
D: Technical Expert Panel and Other Invited Meeting Participants—January 29, 2007
The purpose of the first phase of the Home and Community-Based Services (HCBS) Measure Scan Project is to systematically identify existing measures that either directly assess Medicaid HCBS programs in the three key focus areas specified in the Deficit Reduction Act of 2005 or that could be modified for this purpose. These focus areas are:
- Program performance.
- Client functioning.
- Client satisfaction.
The scope of each is discussed in greater detail below.
To comprehensively and efficiently identify potential measures, this project will use a variety of complementary sources and methodologies, including review of electronic and academic resources, consultation with knowledgeable experts and key informants, a formal solicitation, and a review of tools in current usage by State Medicaid HCBS programs.
Prior to initiating this research, it will be essential to articulate consistent criteria for including and excluding potential measures in the final compendium. In addition, this report specifies the data elements that should be captured for each potential measure, in order to assure the most comprehensive evaluation of each one that will take place in a later phase of this project.
Return to Contents
Deficit Reduction Act of 2005
The HCBS Measure Scan Project is the first step in the much larger set of activities mandated in the Deficit Reduction Act of 2005 (Public Law 109-171). Section 6086(b) of the Act, "Quality of Care Measures," directs the Agency for Health Care Research and Quality (AHRQ) to:
- Develop measures of program performance, client functioning, and client satisfaction with HCBS services under Medicaid programs.
- Assess the quality of Medicaid HCBS services and outcomes and of the overall system for providing these services.
- Disseminate information on any best practices identified during this assessment.
Go to Appendix A for the exact language of Section 6086(b).
The MEDSTAT Group, Inc. (Medstat) is providing assistance with this task under a contract to AHRQ. Specifically, Medstat will conduct a comprehensive environmental scan of existing measures to set the baseline of available science for the process of developing any new measures and will identify gaps in existing measure sets. It is important to note that this contract does not include the actual development of any new measures but rather the identification and evaluation of existing ones, under the guidance of a Technical Expert Panel of stakeholders and researchers.
Diversity of Medicaid HCBS Programs
An important context for this review is the breadth and complexity of Medicaid HCBS programs, in terms of both the services provided by a wide array of providers and the variety of populations served. Nationwide, State Medicaid programs fund home and community-based services for both adults (including seniors) and children who have diverse disabilities and impairments. These disabilities may be cognitive—such as mental retardation and related conditions, developmental disabilities, acquired brain injuries, and dementia—or physical—such as paraplegia and quadriplegia, age-related frailty, numerous chronic conditions, and HIV/AIDS.
Furthermore, the new State plan option will allow States to target services to adults with severe and persistent mental illness, who have traditionally not been served by Medicaid HCBS programs. Services can be participant directed, provider managed, or both, and they may be delivered in an individual's home, a community-based setting, or a small alternative residential setting, such as a group home or assisted living facility.
Although services under Medicaid HCBS programs vary by State, they can include the following:
- Personal care assistance.
- Home health care.
- Skilled nursing care.
- Homemaker services.
- Home-delivered meals.
- Behavioral supports.
- Case management.
- Supported employment.
- Congregate housing.
The growing trend of providing program participants opportunities to self-direct their own supports has added more services to this list, including financial management services and support brokerage. Go to Appendix B for a partial list of services that have been provided by States under their Medicaid HCBS programs. Services may vary from program to program, and States may offer additional services.
Such a broad range of services necessarily involves a variety of providers, including home health agencies, case management agencies, group homes, assisted living facilities, transportation contractors, mental and behavioral health specialists, fiscal intermediaries and individual providers hired directly by consumers. HCBS providers may be for-profit or not-for-profit organizations, as well as public or quasi-public entities. Quality measures in any of the domains below may touch upon the services provided by any of these entities.
Return to Contents
As noted above, the Section 6086(b) specifies three focus areas of HCBS quality: program performance, client functioning, and client satisfaction. A proposed scope and definition for these three areas is specified below in order to organize potential measures. For the purposes of the scan only, AHRQ has chosen to define Medicaid HCBS broadly to encompass all services funded by Medicaid that are provided to people with disabilities in their homes or community settings, including HCBS waivers (1915c and 1915b/c), 1115 waivers, PACE services, Medicaid home health, State Plan services, and others. The resulting broad scope of the measure scan will permit identification of measures for all of these services and service settings.
The many dimensions of program performance reflect both the complexity of HCBS programs and the multiplicity of stakeholders. In fact, it could be argued that program performance subsumes the other two focus areas of interest.
Program performance should, at a minimum, comprise the relevant assurances Medicaid HCBS waivers must provide to the Federal Government, as articulated in Code of Federal Regulations Section 441.302, specifically:
- Plans of care reflect individual need and risk.
- Services are provided by qualified providers.
- The health and welfare of program participants is maintained.
- Administrative authority and oversight is retained by the State Medicaid Agency.a
Given the broad definition of Medicaid HCBS adopted by AHRQ for the measure scan, other dimensions of program performance will likely be encompassed here. The Home and Community-Based Services Quality Framework articulated by the Centers for Medicare & Medicaid Services moves beyond the Federal assurances to specify attributes of HCBS quality, including system performance. In addition, it is possible that some of these waiver-specific requirements will not apply.
Other elements of program performance may include:
- Timely determination of need.
- Coordination of long-term care services with other services.
- Efficient and cost-effective service provision.
- Unmet and undermet need.
Finally, given the emerging role of self-directed services and the mandated self-directed State option for Medicaid HCBS, program performance should also include measures relevant to defining quality in a self-directed program.
Within the Medicaid HCBS population, client functioning often assumes a meaning different from that used in the traditional acute care literature. For many participants, the goal of these programs is to allow them to maintain maximal physical and cognitive functioning, in terms of activities of everyday life, with appropriate services and supports. This is in contrast to the focus of rehabilitative and other postacute care services—such as Medicare home health care, skilled nursing facility care, and many inpatient services—of returning functional ability compromised by illness or injury, although some HCBS programs may include rehabilitation services.
An additional dimension of functioning in the context of HCBS programs is social role functioning, which translates into an opportunity to participate fully in community living in a manner that reflects individual goals and preferences.
In terms of measuring functional outcomes, there are at least two key dimensions to this domain, discussed below. Measures in both these areas must be sufficiently broad to encompass a full range of disabilities, including the population with severe and persistent mental illness.
Unmet need for supports in functional activities. Supports for activities of daily living (ADLs) and instrumental activities of daily living (IADLs)—whether technological, hands-on, stand-by, or cued—are critical components of most Medicaid HCBS programs and play an essential role in allowing people to function in community settings. The absence or inadequacy of such ADL and IADL supports could be inferred to compromise client functioning.
In addition, the scan may include direct measures of current and past functioning, but they will need to be evaluated against program focus and expectation. Such measures are useful for analytical classification of results in terms to specifying quality outcomes for specific HCBS subpopulations.
Social role functioning. The second component of functioning is social role functioning. This may be translated into community integration and thus may include elements of having opportunities and making choices around community activities such as schooling, competitive employment and vocational activities, and social and recreational activities.
By definition, measures of client satisfaction allow program participants to directly assess the quality of the services and supports they receive, according to a personal metric. This assessment is a function of several subjective variables, including individual expectations about what supports should be delivered and how.
There are many models of consumer satisfaction instruments for health care in general and for long-term care specifically. Some allow program participants to define the outcomes that matter to them, and then indicate whether those outcomes have been realized. Others attempt to operationalize the dimensions of quality care—such as choice, control, timeliness, and respect—and ask participants about the presence and absence of these traits. Still others include items assessing general satisfaction with experiences.
The term "client experience" (rather than "client satisfaction") may be a more appropriate identifier for this focus area as it would encompass both "satisfaction" with Medicaid HCBS (whether the program met expectations) as well as "experience" more generally. In addition, an element of consumer preference is under consideration. Preference is important in that it goes beyond the presence or absence of a trait by attempting to assess the value or importance a consumer places on it.
Some of the dimensions of client satisfaction expected to be examined include:
- Individual choice and control.
- Interpersonal interactions.
- Technical quality of services.
a. The other two assurances, institutional level of care and financial oversight, are not considered relevant to this project.
Return to Contents
A matrix crossing the 3 focus areas with at least 13 quality domains will be used to organize the measures identified. These domains will not be mutually exclusive because measures could potentially fit into more than one category.
First among these are the six fundamental aims of health care quality, as articulated by the Institute of Medicine.b These are:
Other domains to be employed are:
- Qualified providers.
- Coordination of long-term care services with other services.
- Health and welfare.
- Administrative oversight.
- Unmet need (for current program participants).
- Quality of life.
b. Institute of Medicine, Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press; 2001.
Return to Contents
This section briefly describes the various sources that will be reviewed to identify potential measures.
Formal Solicitation (Call for Measures)
A formal call for measures will be published in the Federal Register during spring 2007. This solicitation will identify the scope of the review, including a brief discussion of each focus area and the inclusion and exclusion criteria for potential measures. The language will be modeled after the formal call issued for the CAHPS® home health tool and will include language seeking permission to use and/or modify any of the measures submitted. This call will provide a basis for the development of new measures that AHRQ may undertake in the future.
After publication, the language of the formal call will be circulated informally to key experts and informants, including the members of the Technical Expert Panel, the outside experts and observers, the Federal monitors, Medicaid and other State operating agencies, and other knowledgeable individuals in the field.
Current HCBS Quality Measure Compendiums and Databases
To make the most efficient use of project resources, existing measure reviews from completed projects and those in the public domain will be assessed. This effort will include a review of the final reports and bibliographies for tools developed under contract to the Federal Government. Sample sources include:
- Booth M, Fralich J, et al. Literature Review: Quality Management and Improvement Practices for Home and Community-Based Care. (Prepared under Centers for Medicare & Medicaid Services Task Order No. CMS-01-00328). Baltimore, MD: Centers for Medicare & Medicaid Services; 2002.
- Karon SL, Ryther B, et al. Review and Discussion of Current Approaches to Outcomes Measurement in Wisconsin's Medicaid Waiver Programs: A Report to the Division of Disability and Elder Services, Wisconsin Department of Health and Family Services. Interim Report. (Supported in part by CMS Real Choice Systems Change Grant No. 11-P-92085/5-01). Madison, WI: Center for Health Systems Research and Analysis University of Wisconsin, Madison; 2005.
- Fralich J, et al. Quality Indicators for Home and Community-Based Services in Maine. (Developed under CMS Real Choice Systems Change Grant No.11-P-92024/1-01). Portland, ME: Muskie School for Public Service, University of Southern Maine; 2005.
- Galantowicz S, Jackson B. Final Report for the Development of the Participant Experience Survey (Developed under Contract No. 500-96-0006 Task Order No. 2 for the Centers for Medicare & Medicaid Services). Cambridge, MA: The MEDSTAT Group, Inc.; 2005.
- Community Living Exchange Collaborative (http://www.hcbs.org/).
- HCBS Clearinghouse for the Community Living Exchange Collaborative (http://www.qualitymall.org/products/prod1.asp?prodid=2272).
- Aging and Disability Resource Center Technical Assistance Exchange (http://www.adrc-tae.org/tiki-index.php?page=PublicHomePage).
- National Center on Outcomes Resources (http://www.ncor.org/aboutncor.html).
- Centers for Medicare & Medicaid Services, Home and Community-Based Services Quality Initiative (http://www.cms.hhs.gov/HCBS/).
Parallel to the review of HCBS measure compendiums in the public domain will be a comprehensive literature review using pertinent keywords linked to the inclusion criteria. Go to Appendix C for a list of the academic and professional databases to the searched.
In addition, selected potential online sources of quality measures that could be relevant and/or adapted to HCBS will be reviewed. Examples of these include:
- National Quality Measures Clearinghouse (http://www.qualitymeasures.ahrq.gov).
- Medicare Quality Improvement Organization Program (http://www.cms.hhs.gov/qualityimprovementorgs/).
- Web sites of accrediting bodies such as the National Committee for Quality Assurance (http://www.ncqa.org) and The Joint Commission (http://www.jointcommission.org).
- Membership organizations working in the area of health care quality improvement, such as the National Quality Forum (http://www.qualityforum.org) and the American Health Quality Association (http://www.ahqa.org).
- Organizations focused on health care quality, such as the Child and Adolescent Health Measurement Initiative (http://www.cahmi.org).
- Organizations representing providers who serve people with disabilities, such as the National Association for Home Care & Hospice (http://www.nahc.org) and ANCOR (http://www.ancor.org).
- Relevant professional clinical associations, such as American Academy of Home Care Physicians (http://www.aahcp.org) and the Home Healthcare Nurses Association (http://www.hhna.org).
The literature review will include bibliographies of key papers devoted to performance measurement and client satisfaction, specifically within the population of people with disabilities outside of institutional settings.
Experts and Key Informants
Certainly not all measures developed or in use will be referenced in the peer-reviewed literature or available online. A critical link to potential measures is expected to be the Technical Expert Panel (TEP) and substantive reviewers, as well as any outside experts identified for this project. These experts and other interested parties will be able to offer comments and suggestions after each TEP meeting.
Developers of extant tools and measures for assessing HCBS quality are expected to be among these experts and will be able to share information, not only about their own tools but also about others considered and reviewed during their development processes. In addition, they will be able to provide insight regarding "successful" measure characteristics for selected populations and domains for the specification of key data elements to be reviewed and the evaluation criteria for the final compendium.
The first TEP meeting was held on January 29, 2007.c As noted earlier, the language of the formal call for measures will be provided to meeting participants and to to all potential experts and key informants. The draft list of measures will be circulated a few weeks before the next in-person TEP meeting, scheduled for June 22, 2007, and all reviewers will have an opportunity to offer additional measures for consideration.
Review of State Programs
A final, important source of measures is the population of State Medicaid HCBS programs currently funding services and responsible to the Centers for Medicare & Medicaid Services (CMS) for assuring their quality. Because of its role as the National Quality Contractor, Medstat has numerous existing contacts in these programs, as well as data on tools and measures being used for Medicaid HCBS programs and recipients, including Independence Plus and other self-directed programs.
Rather than conducting a new, formal State survey, existing knowledge of quality measures in use by State Medicaid programs will be synthesized and identified gaps will be addressed during a focused followup effort. Followup activities may include informal calls to known State contacts and E-mail solicitations for more information, as well as structured interviews in selected instances.
In addition to contacting State Medicaid programs directly about how they are measuring HCBS program performance, CMS regional and central office staff with responsibility for Medicaid quality oversight will be contacted. Ongoing relationships between these staff and State Medicaid quality staff are expected to facilitate gathering of additional data, including those from formal submissions by States to initiate or renew HCBS waivers (specifically, descriptions of quality management systems). This coordination with CMS will also extend to the organizations that represent State Medicaid programs, such as the National Association of State Medicaid Directors and its Chronic Care Technical Advisory Group (TAG) and Quality TAG, in order to identify State practices and concerns.
To facilitate communication with all interested parties and experts, Medstat will work with AHRQ to explore electronic options to receive and disseminate information widely. Options being considered include a LISTSERV® to broadcast upcoming meetings and project updates and an electronic mailbox to collect submitted measures.
c. Go to Appendix D for a list of participants at the January 29, 2007, meeting of the Technical Expert Panel.
Return to Contents
Inclusion and Exclusion Criteria
The inclusion and exclusion criteria are expected to evolve considerably based on the input of the disability and measurement experts involved in this project. In addition, prior to the formal evaluation of measures, it is important to ensure that the scan of potential measures is sufficiently broad to produce thorough results. Typically, these measures are expected to be constructed from person-level data, which can then be aggregated up to system-level measures.
It is important to note that, although the unit of analysis is generally a person/Medicaid recipient, the data source for constructing the measure may be individual service recipients, administrative or other program data, providers, or State staff, among others. Presented below are some minimal criteria for initial consideration.
- Applicable to at least one of the following populations:
- Physically and/or cognitively impaired elders, including those with dementia.
- Adults or children with mental retardation, a related condition, or developmental disabilities.
- Children whose physical, cognitive, and/or mental health disabilities significantly impair their ability to participate in age-appropriate activities (such as schooling and play).
- Adults with severe and persistent mental illness.
- Adults with acquired brain injuries.
- Adults with physical disabilities and/or chronic conditions (such as HIV/AIDS) that place them at risk of institutional care.
- Must have been either used with, tested with, and/or relevant to at least one of the populations referenced above, with the possible exception of acute care and preventive health care measures.
- Applicable to HCBS that could be funded through Medicaid, even if Medicaid is not necessarily the source of funding.
- Measures developed specifically for institutional care, which cannot be modified for any community settings, including alternative residential care settings.
- Measures for HCBS not eligible for Medicaid funding via any authorized Medicaid mechanism (for example, 1915c and 1115 waivers, State plan option, etc.).
- Measures that do not reflect the experience of individual current HCBS recipients or HCBS program applicants, such as measures of staff satisfaction or caregiver assessment data. One exception is services that are delivered to someone other than the Medicaid card holder (such as respite or child care or family therapies), which do fall within the scope.
It should be noted that caregiver data and other nonparticipant measures are valuable correlates of program quality, but they are outside the scope of this project's central question of how well HCBS programs are working for current participants in the three specified focus areas. However, a list of such measures will be maintained for possible compilation later in a supplemental State scale or similar companion measure set. Any such measures will be made publicly available at the end of the HCBS Measure Scan Project.
Return to Contents
One final methodological decision that must be addressed early in the review, in consultation with the relevant project advisors, is the unit of analysis and the data elements to be captured for each measure. Individual measures, rather than data collection tools, are proposed for evaluation for two main reasons: First, tools, such as surveys or review instruments, may contain or generate multiple measures, not all of which may be relevant to this scan. And second, upon evaluation, some measures may be more robust than others from the same tool.
For each measure, the following data elements, at a minimum, will be systematically collected for the compendium and stored in a database developed for the project:
- Narrative description of measure.
- Construction of the metric, including numerator and denominator if appropriate.
- Tool used to collect data.
- Source of measure (such as literature review, State, etc.).
- Developer affiliation.
- Citation/Web site address (if applicable).
- Development process.
- Release date.
- Subsequent updates, if applicable.
- Populations covered.
- Intended setting (such as assisted living facility, group home, etc.).
- Measure type.
- Current usage.
- Payment source for services (for measures in current use).
- Satisfaction among current users (that is, does measure meet their needs).
- Current users' perception of ease of use.
- Application of findings (how measures are used for quality management).
- Data collection methods:
- Mode (mail, phone, etc.).
- Sampling methodology.
- Frequency of data collection.
- Data collection entity (such as vendor, State, etc.).
- Testing done:
- Testing results.
- Endorsements or recognitions by professional organizations.
- Materials on file:
- Tracking number (assigned internally).
Return to Contents
AHRQ Publication No. 07-0050-EF
Current as of May 2007
Medicaid Home and Community-Based Services Measure Scan: Project Methodology. Contract Report. AHRQ Publication No. 07-0050-EF, May 2007. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/research/ltc/hcbsmethods.htm