Environmental Scan of Measures for Medicaid Title XIX Home and Community-Based Services Measure Scan as of July 5, 2007

Appendix III: Compendium of Measures and Tools Identified Through the Medicaid Home and Community-Based Services Measure Scan as of July 5, 2007

Appendix III (continued)

South Dakota

Tool or Measure Set: South Dakota Freedom to Work Survey
Data Source: Consumer Surveys
Respondent: Consumer
Mode of Data Collection: Mail Survey with Telephone Interview Option
Total Items/indicators: 54 close-ended items 10 demographic questions
HCBS MS Domains: Effectiveness, Person Centeredness/Autonomy, Efficiency, Equity, Access, Unmet Need, and Quality of Life
Populations: Adults with mental retardation or developmental disabilities, adults with severe and persistent mental illness, adults with acquired brain injuries, and adults with physical disabilities and/or chronic conditions
Developer: South Dakota Department of Human Services, Medicaid Infrastructure Grant
Use history: Administered in April of 2002 to recipients of Supplemental Security Income and/or Social Security Disability Income, ages 16-64 years (n=4218). Response rate was 24. 2 percent.
States Currently Using: Study completed
Psychometric Testing: Content validity established via stakeholder participation
Summary: Survey designed through a Steering Committee and Survey Workgroup comprised of consumers, providers, and State and local agencies. Focus areas include barriers to employment, use of personal assistance services at home and at work, opinions pertaining to possible expansion of personal assistance services, and opinions regarding Medicaid Buy-In options.

Tool or Measure Set: Staff Turnover Rates for South Dakota's Developmental Disabilities Providers
Data Source: Provider data
Respondent: N/A
Mode of Data Collection: N/A
Total Items/indicators: 1
HCBS MS Domains: Qualified Providers
Populations: Adults and children with intellectual disabilities (DD)
Developer: Division of Developmental Disabilities, South Dakota Department of Human Services
Use history: The Division uses amount of provider staff turnover as an information source when making assessments around quality of services provided. Starting in 2003 the Division requires providers to submit detailed annual reports on their staff turnover.
States Currently Using: South Dakota
Psychometric Testing: Not specified
Summary: The Division requires providers to report detailed information on staff who left including staff names, position titles, dates of service, hourly rate versus salaried, total hours paid, total wages paid. The Division uses this data to gain insight on staff retention issues which it believes impacts quality of services delivered to consumers.

Tool or Measure Set: South Dakota Unusual Incident Reporting for People with Developmental Disabilities
Data Source: Provider data
Respondent: N/A
Mode of Data Collection: N/A
Total Items/indicators: 9 items
HCBS MS Domains: Safety
Populations: Adults with intellectual disabilities (DD)
Developer: Division of Developmental Disabilities (DDD), South Dakota Department of Human Services
Use history: In 2005 the Division created an automated system for reporting unusual incidents. In 2006 and 2007 the Division created two annual reports with statistics on the aggregate data collected on types of incidents reported in the State's waiver and State-funded program for people with DD. In 2006 2,434 consumers received services from 19 provider agencies.
States Currently Using: South Dakota
Psychometric Testing: Not specified
Summary: The Division monitors unusual incidents on an ongoing basis by collecting data on incidents of unplanned medical attention sought, law enforcement interventions, abuse/neglect/exploitation, injuries, highly restrictive measures, suicide threats/attempts, assaults on other people, unplanned psychiatric attention sought, assault victims, missing persons, and property damage. In addition to producing an annual summary report, the Division publishes provider-specific reports on a quarterly basis.

Return to Appendix III Contents

Tennessee

Tool or Measure Set: Tennessee Division of Mental Retardation's Domains, Outcomes, and Indicators
Data Source: QA surveys include analyzing provider's administrative data and conducting consumer surveys.
Respondent: Consumers
Mode of Data Collection: Face-to-face interviews
Total Items/indicators: Outcomes (O) and Performance Indicators (PI) by TN's 10 Focus Areas:
   Access:O= 1, PI =5
   Individual Planning and Implementation: O= 4, PI= 28
   Safety and Security: O=3, PI=23
   Rights, Respect, and Dignity: O=4, PI= 23
   Health: O=3, PI= 16
   Choice and Decision Making: O=2, PI= 13
   Relationships and Community Membership: O=3, PI=8
   Opportunities for Work: O=2, PI= 11
   Provider Capabilities and Qualifications: O= 4, PI=13
   Administrative Authority and Financial Accountability: O=2, PI=3
   Not every outcome/performance indicator is applicable to each service.
HCBS MS Domains: Access, Person Centeredness/Autonomy, Safety, Health and Welfare, Timeliness, Effectiveness, Qualified Providers, Other Coordination With Other Supports
Populations: Children and adults with intellectual disabilities (MRDD)
Developer: Human Services Research Institute
Use history: Review tools for producing outcomes and performance indicators were developed in 2003 and have been used since July 1, 2004. The Department selects at least a 10% random sample of consumers from each of its day, residential, personal assistance, support coordination, and clinical service providers and conducts consumer interviews and an organizational review of the provider. When providers are too large to make selecting a 10% sample not feasible, the department selects an initial 15 people to interview and will increase this number if problems are identified. Data collection tools span the 10 domains so the State can draw conclusions on the provider's overall performance.
States Currently Using: Tennessee
Psychometric Testing: Not specified
Summary: For each provider it reviews, the Division of Mental Retardation Services assigns each focus area a score, which are tallied to generate an overall "score" which equates to one of five levels of performance—exceptional performance, proficient, fair, significant concerns or serious deficiencies. The findings are reported back to the provider with the expectation that they will utilize them in the formation/revision of their overall quality improvement plan.

Tool or Measure Set: Tennessee State Executive Branch Performance Measures—Commission on Aging and Disability
Data Source: Administrative and Utilization data
Respondent: N/A
Mode of Data Collection: N/A
Total Items/indicators: 7
HCBS MS Domains: Access
Populations: Adults with chronic physical conditions, Elders with physical disabilities
Developer: Tennessee Department of Finance and Administration
Use history: 2006 was the first year the Executive Branch created performance indicators for the Commission on Aging and Disability.
States Currently Using: Tennessee
Psychometric Testing: Not specified
Summary: The Tennessee State Executive Branch has built a framework for developing goals, objectives, strategies, standards and performance indicators for each executive branch-level department or commission. These measures are applicable to Medicaid and State-funded services and focus on increasing the public's access to information on home-based programs and services and enrollment in these programs.

Tool or Measure Set: Tennessee State Executive Branch Performance Measures-Bureau of TennCare
Data Source: Administrative and Utilization data
Respondent: N/A
Mode of Data Collection: N/A
Total Items/indicators: 8
HCBS MS Domains: Timeliness, Administrative Oversight, Access
Populations: Adults with chronic physical conditions, Elders with physical disabilities
Developer: Tennessee Department of Finance and Administration
Use history: To gauge how well the Bureau is providing services to people with disabilities.
States Currently Using: Tennessee
Psychometric Testing: Not specified
Summary: The Tennessee State Executive Branch has built a framework for developing goals, objectives, strategies, standards and performance indicators for each executive branch-level department or commission. These measures are applicable to Medicaid and State-funded services and focus on increasing the public's access to information on home-based programs and services and enrollment in these programs.

Tool or Measure Set: TENNCARE Statewide Audit Tools
Data Source: Administrative and Utilization data
Respondent: N/A
Mode of Data Collection: N/A
Total Items/indicators: Approximately 700 total, although not all indicators are applicable to every type of review for which this tool is used
HCBS MS Domains: Health and Welfare, Safety, Coordination of Long-Term Care and Other Services, Unmet Need, Effectiveness, Qualified Providers, Administrative Oversight, Timeliness
Populations: Adults with chronic physical conditions, Elders with physical disabilities
Developer: TennCare and its quality assurance contractor. QSource
Use history: These tools were developed in February 2007 for QSource to monitor waiver participants and providers, Area Agencies on Aging, and the Commission on Aging and Disabilities.
States Currently Using: Tennessee
Psychometric Testing: The audit tools are still undergoing revisions. The tools have not been tested or scored. Over the next year, the State plans to further develop the tool into a measurement instrument and will provide AHRQ updates as requested.
Summary: QSource utilizes the survey tools to provide a comprehensive quality assurance and quality improvement (QA/QI) program review for the Statewide HCBS Waiver for the Elderly and Disabled Program. The HCBS Waiver for the Elderly and Disabled, as authorized under 1915( c) of the Social Security Act, serves the elderly and disabled persons living at home who would otherwise require care in a nursing facility. The following HCBS Statewide Waiver audit tools are used in the annual survey: the Area Agency on Aging and Disability (AAAD) Administrative Review—Personnel/Providers tool, the AAAD Administrative Records tool, the Individual Enrollee Home Visit tool, and the Individual Enrollee Clinical Record Review tool. Please see attachment for tools. Audit questions are organized according to the four types of reviews this tool is designed to accomplish: 1) Waiver participant home visits and clinical record reviews which include service-specific question sets; 2) Provider reviews with provider-specific questions sets; 3) Reviews of Area Agencies on Agings' administrative operations; and 4) Reviews of the waiver's operating agency, the Commission on Aging and Disabilities. All questions have yes/no/NA choices of responses.

Return to Appendix III Contents

Texas

Tool or Measure Set: Texas Rehabilitation Commission Client Satisfaction Survey
Data Source: Consumer Survey
Respondent: Consumer
Mode of Data Collection: Structured phone interview
Total Items/indicators: 17 items with fixed response sets, plus five narrative items
HCBS MS Domains: Unmet Need, Person Centeredness/Autonomy, Access, Timeliness
Populations: Adults with physical disabilities
Developer: Texas Rehabilitation Commission counselors and administrators
Use history: Fielded in 1996
States Currently Using: Originally used in Texas, current use unknown
Psychometric Testing: Factor analysis performed by University of Texas at Austin, School of Social Work to identify any subscales; four scales were identified-satisfaction with employment, satisfaction with services, responsiveness to client, client participation in process. Internal consistency assessed via Chronbach's alpha
Summary: Sixteen of the items have a 3 or 4-point Likert type response scale and ask respondents to rate satisfaction with selected aspects of services received and current employment. Researchers from the University of Texas at Austin, School of Social Work, analyzed the results of more than 11,000 completed surveys to identify underlying determinants of client satisfaction with rehabilitation services.

Tool or Measure Set: Texoma Real Choice Program Pre and Post Consumer Surveys
Data Source: Consumer Surveys
Respondent: Consumer
Mode of Data Collection: Mail Survey
Total Items/indicators: 2 close-ended items and 1 open-ended question on presurvey and 7 close-ended items and 1 open-ended question on postsurvey
HCBS MS Domains: Effectiveness, Person Centeredness/Autonomy, Equity, Access, and Quality of Life
Populations: Elders with physical or cognitive disabilities, adults with mental retardation or developmental disabilities, adults with severe and persistent mental illness, adults with acquired brain injuries, and adults with physical disabilities and/or chronic conditions.
Developer: Texoma, Texas Health and Human Services Commission
Use history: Not specified
States Currently Using: Texas
Psychometric Testing: Not specified
Summary: Texoma Pre and Post Consumer Surveys are designed to provide information pertaining to how individuals learned of the aging and disability resource center as well as pre and post assessment of consumer choice and control, awareness and access to existing resources, and quality of life.

Tool or Measure Set: Performance Indicators for Texas' Community-Based and ICF/MR (Intermediate Care Facilities for Persons With Mental Retardation) programs
Data Source: National Core Indicators Consumer and Family Survey as well as the Participant Experience Survey
Respondent: People who receive services and/or their representative
Mode of Data Collection: Face-to-Face structured interviews and mail surveys
Total Items/indicators: 139
HCBS MS Domains: Qualified Providers, Health and Welfare, Administrative Oversight, Person Centeredness/Autonomy, Access
Populations: Elders and people with physical and intellectual and developmental disabilities
Developer: Thomson/Medstat, Human Services Research Institute in collaboration with the National Association of State Directors of Developmental Disabilities Services
Use history: The Department of Aging and Disability Services is the Operating Agency for long term services and supports for elders and people with disabilities. The Department is in the process of developing a single quality management strategy for all the services it provides. Many of these indicators are common to all programs; a few are unique.
States Currently Using: Texas
Psychometric Testing: Psychometric testing of both consumer tools.
Summary: These indicators encompass all aspects of the Department's quality oversight strategy and data sources. Indicators are grouped into six broad areas: Consumer Outcomes, System Performance, Health, Welfare, Rights, Self-Determination, Access to Care, Choice and Control, Respect/Dignity, and Community Integration/Inclusion

Tool or Measure Set: Participant Experience Survey-Texas Supplement
Data Source: Consumer Survey
Respondent: People who receive services or their representatives
Mode of Data Collection: Face-to-face structured interviews
Total Items/indicators: 6 additional questions to the National Core Indicators (NCI) Consumer Survey and Participant Experience Survey Elderly/Disabled version
HCBS MS Domains: Person Centeredness/Autonomy
Populations: Elders and people with physical and intellectual and developmental disabilities
Developer: Texas Department of Aging and Disability Services
Use history: From 2005-2008, Texas conducted hundreds of interviews with waiver participants using the Participant Experience Survey, elderly/disabled version (PES E/D) and NCI Consumer Survey. These six questions were added to both tools to provide the State with more information about select areas of personal choice and control and community integration.
States Currently Using: Texas
Psychometric Testing: No
Summary: The six additional items are:

1. Do you get to help other people?
2. Can you have a close relationship, like a boyfriend or a girlfriend, if you want one?
3. Do you earn enough money to buy the things you want?
4. Are you free to take risks when you want to?
5. Do you have control over your own transportation?
6. Do you get to decide who comes in and out of your home?

Return to Appendix III Contents

Vermont

Tool or Measure Set: Indicators for Vermont's HCBS Waiver Services
Data Source: Multiple: consumer survey, record review, agency review, direct service staff feedback, family interview
Respondent: N/A
Mode of Data Collection: N/A
Total Items/indicators: 49 indicators
HCBS MS Domains: Person Centeredness/Autonomy, Access, Unmet Need, Quality of Life, Safety, Health and Welfare, Coordination of Long-Term Care and Other Services, Qualified Providers, Timeliness
Populations: Adults with acquired brain injuries, elders with physical and cognitive disabilities, adults with physical disabilities, adults with intellectual disabilities
Developer: Vermont Agency of Human Services, Department of Disabilities, Aging and Independent Living, Division of Disability and Aging Services
Use history: Quality management strategy, including indicators, released in April 2007. Contracting providers will be reviewed every few years based on the indicators, including interviews with 10% of individuals served by the agency.
States Currently Using: Vermont
Psychometric Testing: Indicators developed with stakeholder input to provide content validity. Limited cognitive testing of consumer survey tool.
Summary: The indicators in the Vermont Quality Management Plan are organized along ten desired outcomes identified by consumers, families and providers in the State: respect; self-determination; independent living; relationships; participation; person-centered practices; well-being; communication; collaboration; and support systems. Indicators are phrased as statements; reviewers determine, based on available data sources, whether indicator is met. For example, "Individuals may change service staff and providers without retaliation" can be determined both based on responses to consumer interview questions and a review of agency policies.

Return to Appendix III Contents

Virginia

Tool or Measure Set: Virginia Department of Medical Assistance Services Quality Management Review Tool-Elderly Disabled Consumer Directed Waiver
Data Source: Administrative Data, Provider Data
Respondent: N/A
Mode of Data Collection: N/A
Total Items/indicators: 52
HCBS MS Domains: Health and Welfare, Administrative Oversight, Unmet Needs, Qualified Providers, Safety, Coordination of Long-Term Care and Other Services, Effectiveness
Populations: Elders with physical disabilities, Adults with chronic physical conditions
Developer: Virginia Department of Medical Assistance Services
Use history: The Commonwealth started to design this tool in January 2006 and used it for the first time in October 2006. At that time 38 providers and 484 consumers were reviewed. The goal is to develop performance indicators based on the information collected from using this tool.
States Currently Using: Virginia
Psychometric Testing: Indicators still under development
Summary: The Medicaid agency developed this extensive Excel spreadsheet—and four similar tools for each of its other HCBS waivers—for reviewers to collect and organize their information during provider reviews. Sets of questions are categorized into seven focus areas that the Commonwealth plans to develop performance indicators for: 1) Financial Accountability, 2) Prescreening/Recipient Choice, 3) Assessment by Provider, 4) Plan of Care/Service Plan, 5) Provider Qualifications, 6) Health & Welfare & Regulatory, and 7) Plan of Action/Corrective Action. All questions have closed responses so the Department will be able to monitor frequency of findings and types of corrective actions requested.

Tool or Measure Set: Virginia Department of Medical Assistance Services Quality Management Review Tool-Persons with HIV/AIDS Waiver
Data Source: Administrative Data, Provider Data
Respondent: N/A
Mode of Data Collection: N/A
Total Items/indicators: 51
HCBS MS Domains: Health and Welfare, Safety, Qualified Providers, Administrative Oversight, Effectiveness, Unmet Needs, Coordination of Long-Term Care and Other Services
Populations: Adults with chronic physical conditions (HIV/AIDS)
Developer: Virginia Department of Medical Assistance Services
Use history: While this tool is fully operational it has not been used yet. The goal is to develop performance indicators based on the information collected from using this tool.
States Currently Using: Virginia—planned
Psychometric Testing: Not specified, indicators still under development
Summary: The Medicaid agency developed this extensive Excel spreadsheet—and four similar tools for each of its other HCBS waivers—for reviewers to collect and organize their information during provider reviews. Sets of questions are categorized into seven focus areas that the Commonwealth plans to develop performance indicators for: 1) Financial Accountability, 2) Prescreening/Recipient Choice, 3) Assessment by Provider, 4) Plan of Care/Service Plan, 5) Provider Qualifications, 6) Health & Welfare & Regulatory, and 7) Plan of Action/Corrective Action. All questions have closed responses so the Department will be able to monitor frequency of findings and types of corrective actions requested.

Tool or Measure Set: Virginia Family Satisfaction Survey
Data Source: Family survey
Respondent: Families of people with MR receiving Department of Mental Retardation services
Mode of Data Collection: Mail survey
Total Items/indicators: 26
HCBS MS Domains: Health and Welfare, Safety, Unmet Need, Qualified Providers, Timeliness, Access, Effectiveness
Populations: Families of children and adults with intellectual disabilities
Developer: Alexandria Community Services Board
Use history: Tool is used to help the Virginia Department of Medical Assistance identify problem areas in their waiver for people with MR/DD.
States Currently Using: Virginia
Psychometric Testing: Not specified
Summary: This tool prompts the person completing the survey to provide some demographic information on themselves as well as on the person with mental retardation. General satisfaction questions then ask the family member to rank using a 3-point response scale their impressions of the consumer's environment, availability of appropriate services, services that the person is receiving, problems with staffing changes. Respondents receive a prepaid envelope to return the survey.

Tool or Measure Set: Virginia Department of Medical Assistance Services Quality Management Review Tool-Waiver for People who are Technology Dependent
Data Source: Administrative Data, Provider Data
Respondent: N/A
Mode of Data Collection: N/A
Total Items/indicators: 49
HCBS MS Domains: Health and Welfare, Safety, Qualified Providers, Administrative Oversight, Effectiveness, Unmet Needs, Coordination of Long-Term Care and Other Services
Populations: Chronically ill/severely impaired technology assisted children and adults
Developer: Virginia Department of Medical Assistance Services
Use history: The Commonwealth used this tool for the first time in March 2007 to review two providers and 18 consumers. The goal is to eventually develop performance indicators based on the information collected from using this tool.
States Currently Using: Virginia
Psychometric Testing: Not specified, tool is still under development
Summary: The Medicaid agency developed this extensive Excel spreadsheet—and four similar tools for each of its other HCBS waivers—for reviewers to collect and organize their information during provider reviews. Sets of questions are categorized into seven focus areas that the Commonwealth plans to develop performance indicators for: 1) Financial Accountability, 2) Prescreening/Recipient Choice, 3) Assessment by Provider, 4) Plan of Care/Service Plan, 5) Provider Qualifications, 6) Health & Welfare & Regulatory, and 7) Plan of Action/Corrective Action. All questions have closed responses so the Department will be able to monitor frequency of findings and types of corrective actions requested.

Tool or Measure Set: Virginia Department of Medical Assistance Services Quality Management Review Tool-Persons with Developmental Disabilities Day Supports Waiver
Data Source: Administrative Data, Provider Data
Respondent: N/A
Mode of Data Collection: N/A
Total Items/indicators: 84
HCBS MS Domains: Health and Welfare, Safety, Qualified Providers, Administrative Oversight, Effectiveness, Unmet Needs, Coordination of Long-Term Care and Other Services
Populations: Adults and children with intellectual disabilities
Developer: Virginia Department of Medical Assistance Services
Use history: The Commonwealth used this tool for the first time in January 2007 to review one provider and two consumers. The goal is to eventually develop performance indicators based on the information collected from using this tool.
States Currently Using: Virginia
Psychometric Testing: Not specified, tool still under development
Summary: The Medicaid agency developed this extensive Excel spreadsheet—and four similar tools for each of its other HCBS waivers—for reviewers to collect and organize their information during provider reviews. Sets of questions are categorized into seven focus areas that the Commonwealth plans to develop performance indicators for: 1) Financial Accountability, 2) Prescreening/Recipient Choice, 3) Assessment by Provider, 4) Plan of Care/Service Plan, 5) Provider Qualifications, 6) Health & Welfare & Regulatory, and 7) Plan of Action/Corrective Action. All questions have closed responses so the Department will be able to monitor frequency of findings and types of corrective actions requested.

Tool or Measure Set: Virginia Department of Medical Assistance Services Quality Management Review Tool-Persons with Developmental Disabilities Waiver
Data Source: Administrative Data, Provider Data
Respondent: N/A
Mode of Data Collection: N/A
Total Items/indicators: 73
HCBS MS Domains: Health and Welfare, Safety, Qualified Providers, Administrative Oversight, Effectiveness, Unmet Needs, Coordination of Long-Term Care and Other Services
Populations: Adults and children with intellectual disabilities
Developer: Virginia Department of Medical Assistance Services
Use history: The Commonwealth is currently revising this tool to be more user friendly because of the large number of waiver services it needs to cover. The tool was used for the first time in January 2007 to review seven providers and 23 consumers and continues to be used while under revision. The goal is to eventually develop performance indicators based on the information collected from using this tool.
States Currently Using: Virginia
Psychometric Testing: Not specified, tool is still under development
Summary: The Medicaid agency developed this extensive Excel spreadsheet—and four similar tools for each of its other HCBS waivers—for reviewers to collect and organize their information during provider reviews. Sets of questions are categorized into seven focus areas that the Commonwealth plans to develop performance indicators for: 1) Financial Accountability, 2) Prescreening/Recipient Choice, 3) Assessment by Provider, 4) Plan of Care/Service Plan, 5) Provider Qualifications, 6) Health & Welfare & Regulatory, and 7) Plan of Action/Corrective Action. All questions have closed responses so the Department will be able to monitor frequency of findings and types of corrective actions requested.

Tool or Measure Set: Virginia Department of Medical Assistance Services Quality Management Review Tool-Persons with Mental Retardation Waiver
Data Source: Administrative Data, Provider Data
Respondent: N/A
Mode of Data Collection: N/A
Total Items/indicators: 73
HCBS MS Domains: Health and Welfare, Safety, Qualified Providers, Administrative Oversight, Effectiveness, Unmet Needs, Coordination of Long-Term Care and Other Services
Populations: Adults and children with intellectual disabilities
Developer: Virginia Department of Medical Assistance Services
Use history: The Commonwealth is currently revising this tool to be more user friendly because of the large number of waiver services it needs to cover. The tool was used for the first time in December 2006 to review 23 providers and 184 consumers and continues to be used while under revision. The goal is to eventually develop performance indicators based on the information collected from using this tool.
States Currently Using: Virginia
Psychometric Testing: Not specified, tool still under development
Summary: The Medicaid agency developed this extensive Excel spreadsheet—and four similar tools for each of its other HCBS waivers—for reviewers to collect and organize their information during provider reviews. Sets of questions are categorized into seven focus areas that the Commonwealth plans to develop performance indicators for: 1) Financial Accountability, 2) Prescreening/Recipient Choice, 3) Assessment by Provider, 4) Plan of Care/Service Plan, 5) Provider Qualifications, 6) Health & Welfare & Regulatory, and 7) Plan of Action/Corrective Action. All questions have closed responses so the Department will be able to monitor frequency of findings and types of corrective actions requested.

Return to Appendix III Contents

Washington

Tool or Measure Set: Washington Mental Health Adult Survey Instrument
Data Source: Consumer Survey
Respondent: Consumer
Mode of Data Collection: Computer assisted telephone interviewing (CATI)
Total Items/indicators: 32 close-ended items regarding program services, plus additional demographic and narrative items
HCBS MS Domains: Quality of Life, Access, Unmet Need, Effectiveness, Person Centeredness/Autonomy
Populations: Adults with mental illness
Developer: State of Washington, Department of Social and Health Services, Mental Health Division, in collaboration with the University of Washington
Use history: Administered to a sample of consumers who have received services in the last nine months. Survey is conducted every other year.
States Currently Using: Washington
Psychometric Testing: Not specified
Summary: Telephone interviewers from the University of Washington read a series of 32 statements to respondents, asking them to assess their level of agreement using a five-point scale. Items are grouped into those related to the experience of receiving mental health services and those assessing the direct result (outcome) of those services. Additional items examine standard demographics, as well as use of medical services, insurance, and employment history. Responses are rolled up for reporting as indicators, (e.g., clients' perception of service quality) on the State's website.

Tool or Measure Set: Washington Mental Health Performance Indicators
Data Source: Multiple: Selected consumer surveys, utilization and expenditure data
Respondent: N/A
Mode of Data Collection: N/A
Total Items/indicators: 77 indicators; not all apply to Medicaid recipients
HCBS MS Domains: Access, Effectiveness, Unmet Need, Efficiency
Populations: Adults and children receiving mental health services
Developer: Washington Department of Social and Health Services, Mental Health Division. The ongoing effort to develop performance indicators and publish the annual reports has been supported in part by funds provided by the U. S. Center for Mental Health Services, Substance Abuse and Mental Health Services Administration, through the State Indicator Pilot Grant and the State Mental Health Data Infrastructure Grant.
Use history: The Performance Indicator report was first published in Fiscal Year 2001 and has been published annually ever since.
States Currently Using: Washington
Psychometric Testing: Not specified
Summary: The Performance Indicators provide information on who receives services (Penetration Rates and Client Characteristic); what types of services are delivered (Community Outpatient, Community Inpatient, Crisis Only, Types of Services), what happens to people in services (Outcomes, Client Perceptions of Care) and how much services cost (Expenditures). Indicator results are published on an interactive website that allows users to see the data, formulas and data sources used to develop each indicator.

Tool or Measure Set: Washington Division of Developmental Disabilities Plan of Care Survey
Data Source: Consumer survey
Respondent: Service recipient, family/guardian, paid staff or friend/advocate
Mode of Data Collection: Mail
Total Items/indicators: 14 items
HCBS MS Domains: Unmet Need, Person Centeredness/Autonomy, Safety, Coordination of Long-Term Care and Other Services
Populations: Individuals with intellectual or other developmental disabilities who receive HCBS Waiver services.
Developer: Division of Developmental Disabilities, Washington Department of Social and Health Services
Use history: Survey conducted annually.
States Currently Using: Washington
Psychometric Testing: Not specified
Summary: After each Waiver participant's annual Plan of Care meeting, a survey is left with the individual/family to complete regarding their experiences at the Plan of Care meeting. Surveys are left with all 9,500 HCBS Waiver participants. This tool is available in English, Cambodian, Chinese, Korean, Laotian, Russian, Somali, Spanish, and Vietnamese. Items address participation in development of the Plan of Care, staff interaction, rights, and plan completeness.

Tool or Measure Set: Washington Division of Developmental Disabilities Supported Employment Cost Benefit Analysis
Data Source: Administrative payment data
Respondent: N/A
Mode of Data Collection: N/A
Total Items/indicators: Six: three averages and three ratios
HCBS MS Domains: Access, Efficiency, Effectiveness
Populations: Individuals with intellectual or other developmental disabilities who receive employment support services funded through the Division of Developmental Disabilities
Developer: Division of Developmental Disabilities, Washington Department of Social and Health Services
Use history: Reports run as needed
States Currently Using: Washington
Psychometric Testing: N/A
Summary: This is automated data report from the State's payment system. The report can be run for a single month or a larger unit of time. Client-level data are aggregated by region and State to examine hours worked, wages, State costs and the cost-benefit of State dollars.

Tool or Measure Set: Washington Division of Developmental Disabilities Modified National Core Indicator Consumer Surveys
Data Source: Consumer survey
Respondent: Consumer (for consumer survey)
Family member or other proxy (family surveys)
Mode of Data Collection: In-person interview (consumer survey)
Mail Surveys (Family Surveys)
Total Items/indicators: Additional items beyond the original NCI items (number varies by survey tool)
HCBS MS Domains: Health and Welfare, Person Centeredness/Autonomy
Populations: Individuals with intellectual or other developmental disabilities
Developer: Division of Developmental Disabilities, Washington Department of Social and Health Services-modification of National Core Indicators tools originally developed by the Human Services Research Institute in collaboration with the National Association of Directors of Developmental Disabilities Services
Use history: Used biannually with a sample of program participants
States Currently Using: Washington. A few NCI States are using the additional questions on case management.
Psychometric Testing: Original NCI items subjected to psychometric testing; State specific measures tested for construct validity, some items have been tested for criterion validity and interrater reliability.
Summary: Washington's Division of Developmental Disabilities has modified five of the NCI survey tools; three family surveys (adult, guardian and child) and the adult and child versions of the consumer surveys. The additional items address cultural sensitivity, staff turnover, schooling, individual providers, case managers, caregiving challenges, informal support, protective supervision, and future planning, among others. The NCI Consumer Survey has been adapted for use with minors. Questions about interviewer observations of the home setting were adapted from Pennsylvania's interview form. Surveys are available in English and Spanish, other languages are available as requested.

Return to Contents
Proceed to Next Section