The Deficit Reduction Act (DRA) of 2005 (PL 109-171, Section
6086(b)) directs the Agency for Healthcare Research and Quality (AHRQ) to develop
home and community-based services (HCBS) quality measures for the Medicaid
(Title XIX) program. AHRQ is to develop measures in the domains of client
functioning, client satisfaction, and program performance in order to assess
the quality of Medicaid HCBS programs nationwide. In addition, AHRQ is tasked
with publishing best practices information as a result of comparative analyses.
To lay the groundwork for meeting these requirements, AHRQ contracted with
Thomson Reuters to conduct an environmental scan of existing and potential
measures. The measure scan was AHRQ's first step in fulfilling this directive,
through documentation of the available science to support measure development
Per the legislative directive, measures were sought from a
variety of academic, professional, online, and State sources in the three DRA
domains, under the guidance of the Technical Expert Panel (TEP) assembled for
this project. In addition, a formal Call
for Measures was published in the Federal
Register in June 2007. We used a
very broad definition of HCBS services and populations, including populations
such as adults with severe and persistent mental illness who are not
traditional recipients of Medicaid HCBS.
This approach yielded more than 200 measure sources, including survey instruments
designed to yield performance measures, measure sets, and measure databases. Summaries of each, including information
about psychometric testing, development, topics, and prevalence of use, were
compiled in a compendium for public distribution.
A draft version of the compendium of measures (compendium) was
discussed at an in-person meeting of the TEP in June 2007. During this meeting, TEP members began to define
the broad DRA domains and align them with specific measure constructs. This
process was complicated by the overlapping nature of the domains written into
the DRA, particularly the fact that client functioning and client satisfaction
outcomes are really measures of overall program performance. Through further
rounds of voting and stakeholder input, 21 constructs emerged as a list of
important dimensions of HCBS quality that would be generally applicable across HCBS
populations. These were, by DRA domain:
in daily activity function.
of support with everyday activities when needed.
of family relationships.
attendance (children only).
of recommended preventive health care services.
reportable adverse health events.
- Client Experience:
treatment by direct service providers.
to make choices about providers.
to make choices about services.
with case management services.
perception of quality of care.
and choice regarding residential setting.
report of abuse and neglect.
of support for resilience and recovery (mental health service recipients only).
- Program Performance:
- Access to case management services.
of care coordination.
of all services in the care plan.
It should be noted that many other constructs were
considered, and some omissions do not reflect stakeholder consensus. In particular, as the project progressed, it
was recommended by some TEP members that the constructs of pain, depression,
and weakness/fatigue be added to the functioning domain, although not all
stakeholders rated these as important.
A crosswalk of the compendium contents against these 21
constructs was used to populate matrices of measures that aligned with each
construct and that had been subjected to some degree of psychometric testing.
To further evaluate candidate measures, a set of scoring criteria were
developed to rate the testing, target population(s), data collection
requirements, and prevalence of current use (one component of feasibility) associated
with each measure. Measures that did not
meet certain thresholds in each of these areas were removed to create a second
set of matrices. In addition, proprietary measures not formally submitted to
AHRQ and those identified after the cutoff date for formal submission were not
eligible for evaluation.
The final set of matrices, one per domain listed in the DRA,
provided the data for identifying any relevant gaps in the current science in
HCBS quality measurement. Looking at the compendium in consort with the
matrices, certain broad themes from the scan emerge:
consumer survey tools have been developed to assess consumer experience
with HCBS, particularly for individuals with intellectual and/or
developmental disabilities, that include constructs from the above list.
- Many surveys
have been subject to psychometric evaluation and are currently used by
several State programs. Some of these tools also have well-developed
training programs to support their use, which enhances their feasibility.
HCBS consumer survey tools were developed and tested specifically for
one-time evaluation purposes and are no longer in use. However, they align
well with the constructs of interest.
single survey tool or measure set addresses all the constructs included in
the list above.
other extant measures potentially align with these constructs, although
relatively few formal measures have been developed and tested specifically
for HCBS programs and supports.
- Many State-specific
tools have been designed and implemented to solicit consumer feedback and/or
report against the federally required assurances for Medicaid 1915c waiver
programs. However, few have
scientifically documented reliability and validity.
academic studies have yielded well-tested measures with potential
application to Medicaid HCBS programs.
Most of these measures, however, are not in current use by any
existing State programs.
With regard to the three domains articulated in the DRA, a
more detailed analysis yielded the observations summarized below.
Client functioning has a different connotation for HCBS
program participants relative to recipients of other post-acute care services
provided in community settings. HCBS programs provide services and supports that
allow participants to function in their communities, regardless of disability,
rather than aim solely to improve underlying functional ability. To this end, the DRA domain of client
functioning was defined for this project to include the following three
components: functioning in daily activities; social role functioning; and health
promotion/adverse health outcomes. Together, these subdomains make up 10 measure
constructs from the list of quality measurement priorities identified by the
In reviewing only the measures that meet the threshold evaluation
criteria, we saw the following themes emerge:
the goal of HCBS programs is typically to support participants' daily
activity functioning, measures based on client reports of unmet need for
assistance can be used to assess if a State's Medicaid HCBS program is providing
appropriate supports. These types of measures are available and may be
more relevant than measures of change in underlying functional ability.
found multiple measures, primarily in consumer surveys, to assess the identified
dimensions of social role functioning.
HCBS-appropriate definitions of a serious reportable event are lacking,
particularly for non-intellectually disabled populations. Most of
the measures endorsed by the National Quality Forum, for example, apply to
institutional or residential settings only.
regard to avoidable hospitalizations, the Prevention Quality Indicators (PQIs)
developed by AHRQ would require additional testing and/or modification to
determine their appropriateness for the Medicaid HCBS population. There
are several challenges in adopting the PQIs, in both collecting data and
linking hospitalization to HCBS programs.
the U.S. Preventive Services
Task Force's (USPSTF) annual Guide
to Clinical Preventive Services is
considered the evidence-based standard for preventive health care
services, USPSTF recommendations
are designed for the asymptomatic population and are not disease—or individual—specific.
Thus, there is no consensus set of preventive health care services
for the Medicaid HCBS population and their complex care needs.
In our review, the broader concept of "client experience" was
substituted for the "client satisfaction" DRA domain, to better capture both
objective and subjective assessment of program services and supports. Innumerable
dimensions of client experience can be measured, and many State Medicaid
programs try to capture at least some client feedback. In our discussions with
stakeholders and experts, eight dimensions of client experience emerged as
clearly important, which are listed above.
Of the three DRA domains specified, client experience was
the one with the best coverage, in terms of tested and prevalent measures. This was especially true of the population
with intellectual and/or developmental disabilities. Much work has already gone
into the development of consumer survey tools designed to solicit feedback on
these constructs. The challenge is in comparing these instruments. In our
preliminary review, we noted:
is a lack of standardization in the definition of
abuse/neglect/exploitation in the items assessing client experience.
is an opportunity to assess (via psychometric testing) the
applicability/reliability of existing experience measures to HCBS
populations other than the one(s) for which they were developed. The need
for such cross-disability testing depends in part on the future direction
of AHRQ's strategy for meeting the DRA requirements and the use of
cross-disability versus population-specific measures.
Although the DRA language specified measure development in
three separate domains, they are not necessarily mutually exclusive concepts. There is overlap between measures of client
function and experience and assessment of overall program performance. In fact,
this final domain in some ways subsumes the previous two.
Currently, State Medicaid HCBS waiver programs are required
to provide assurances to the Federal Centers for Medicare & Medicaid
Services (CMS) of their compliance with six key dimensions of program
performance. Assessing compliance with
the full array of Federal program performance requirements exceeds the scope of
this exercise. However, feedback from AHRQ's TEP members and others identified
three measure constructs that address portions of program performance (although
by no means fully define it): access to case management; receipt of all
services in the care plan; and care coordination services.
Of the three DRA domains we examined, we found the fewest
potential candidate measures for program performance, as defined for the
purposes of the scan. We found few prevalent, common measures for the receipt
of case management services (including care coordination or support broker
services) or access to case management services that could be drawn from
administrative data. Similarly, we did not find simple, standardized measures
for the receipt of all services specified in the plans of care. In short, the
greatest opportunity for measure refinement and development resides in the area of service delivery.
The measure constructs and associated measures that emerged
from this measure scan process are by no means definitive with regard to HCBS
quality. Many other areas of potential quality assessment did not make the
consensus list, such as qualified providers and exercising individual rights,
which may be worthy of consideration.
Further, although measures developed for institutional settings were
excluded from the lists of evaluated candidate measures, they may offer
potential for modification and inclusion. Others with potential include the home health version of the Consumer
Assessment of Healthcare Providers and Systems (CAHPS®), the CMS Continuity Assessment Record and Evaluation (CARE),
and the Assessment of Health Plans and
Providers by People With Activity Limitations, which is in development. During the public comment period
of the final TEP meeting, participants also raised inclusion of new
instruments, including the Universal Stakeholder Participation and
Experience Questionnaires (U-SPEQ®) consumer interview tool recently developed by the Committee for
Accreditation of Rehabilitation Facilities (CARF).
As AHRQ moves into the second phase of this project, other issues
related to performance measurement will become increasingly important. The mode
of data collection, for example, has bearing on both measure development and
project implementation. Some measure
constructs are best derived from consumer-reported data, particularly those
related to consumer experience. Others may be calculated from one or more less
costly administrative data sources.
Ideally, assessments of HCBS quality may necessitate
triangulation of data sources. TEP
members provided feedback suggesting that some of the constructs we examined,
including avoidable hospitalizations, serious reportable events, school
attendance, and receipt of all services in the care plan, might be obtained
from administrative data sources. In particular, claims data can reveal a great
deal about variations in care and potential quality issues. The mode of service delivery, namely in a
traditional agency model or via a self-directed program, can also influence
data items and data collection strategies.
Finally, in order to conduct the required State-to-State comparisons of HCBS quality, risk adjustment is needed. There are administrative measures
that do not directly assess HCBS program quality but provide an important
context for those that do, including severity of impairment and other
assessment data. They also include participant demographics, such as age, sex,
residential setting and size, and diagnosis. One TEP recommendation and
possible future task is the development of a comprehensive set of individual
demographic and disability measures related to individual functioning. Environmental
service variables are also needed, which would enhance the value of any data
collected by AHRQ in identifying best practices in Medicaid HCBS programs.
Return to Contents
Proceed to Next Section