Chapter 6. Overview of instruments Used in Evaluating Residential Care

The instruments included in this review reflect a variety of long-term care residential settings including assisted living, residential care, and continuing care retirement communities because assisted living is defined in several ways. Further, because assisted living encompasses many levels of care, instruments were also reviewed from nursing home and other long-term care settings to potentially inform the development of tools that could help consumers make decisions about assisted living facilities. Nearly all of the instruments included in this scan were developed as a result of research efforts; however, some were produced by commercial vendors who market an assisted living satisfaction instrument and still others were developed by consumer advocacy organizations. Most were developed in the United States.

Because there is such disparity in the services that are offered in assisted living, relying on one source of information to help consumers evaluate facilities is not sufficient. Consumers need information to help them determine if a facility offers services that can meet their needs in the near and long-term, and they need a clear description of the services provided. In addition, the consumer needs to be able to determine the quality of those services. To provide a comprehensive assessment of both the description of what is offered and how facility performance could be evaluated, the researchers reviewed the following categories of instruments:

• Consumer satisfaction tools that evaluate the most important aspects of satisfaction and quality of life from the perspective of the resident or family member or close friend of a resident. The majority of the instruments presented in Appendix A (PDF File, 700 KB; PDF Help) are consumer-reported.
• Observational instruments that allow an objective verification of services offered and the quality provided, similar to the private accreditation and State survey processes. These surveys tend to focus on easily observable features, such as the condition of the physical building and furnishings, cleanliness of rooms, and accessibility of the outdoor grounds, as well as resident behavior and interaction with staff. Observational instruments are included in Appendix A (PDF File, 700 KB; PDF Help).
• Provider-reported surveys that (1) list programs and services and (2) supply information on the ownership and financial status of the facility, the education and experience level of the administrator and staff in the facility, and broad demographic characteristics of the residents residing in the facility. Provider-reported instruments are also presented in Appendix A (PDF File, 700 KB; PDF Help).

Return to Contents

Consumer-Reported Instruments

Although the consumer-reported instruments are categorized by purpose, e.g., satisfaction of services or quality of life, many of the domains are interconnected and use similar items/questions to measure different domains. Ejaz, Straker, Fox, and Swami (2003) note: "Consumer satisfaction represents a subjective measure of quality of care but it affects overall quality of life." Accordingly, consumer-reported tools cannot be strictly defined as satisfaction (quality of services) or quality of life tools, nor do they have unique, independent domains.

For example, Kane, Kling, Bershadsky, et al., (2003) use three items on the residents' satisfaction with food at the facility to measure the domain of "enjoyment" in the quality of life tool for nursing homes. Similarly, the satisfaction tools use food service as a domain of satisfaction. Likewise, the administrative tools that are designed to measure the assisted living philosophical environment (Kane, Bershadsky, Kane, et al., 2004; Utz, 2003)—including building a sense of community, integrating residents, and promoting independence—draw a parallel to elements found in resident satisfaction such as involvement and choice.

Although quality of life measures can be used to differentiate facilities based on resident reported information (Kane, Bershadsky, Kane, et al., 2004) and hold promise as a measure for consumers who are deciding on an assisted living facility, they are not typically measured in evaluating assisted living facilities (Wilson, 2003). Three quality of life tools developed for long-term care are presented in Appendix A (PDF File, 700 KB; PDF Help). First, Kane, Kling, Bershadsky, et al., (2003) distinguished 11 domains of quality of life in long-term care: security, physical comfort, enjoyment, meaningful activities, relationships, functional competence, dignity, privacy, individuality, autonomy, and spiritual well being. Second, the Quality of Life Index (Ferrans and Powers, 1985) subscales include health and functioning, social and economic factors, psychological/spiritual factors, and family situation/support http://www.uic.edu/orgs/qli. Third, the Minimum Data Set Version 3.0 (in development) includes a quality of life section that measures 11 domains that are the same as those identified by Kane, Kling, Bershadsky, et al., (2003) with the exception of food enjoyment, which is more specific than the broader enjoyment domain of Kane et al. (Anderson, Connolly, Pratt, and Shapiro, 2003).

Return to Contents

Expert Observational Instruments

Some researchers (Aud, Rantz, Zwygart-Stauffacher, and Manion, 2004; Hawes, Rose, and Phillips, 1999; Hawes, Phillips, and Rose, 2000b; Hawes, Phillips, Rose, et al., 2003; Sloane, Mitchell, Weisman, et al., 2002) adapted observational tools for assisted living facilities from those originally designed for nursing homes. Much of the background needed for development of these observational instruments was based in part on work by Moos and Lemke (1984), whose seminal work conceived a framework for observation of physical features in geriatric housing. Rantz, Jensdottir, Hjaltadottir, et al., (2002) provide an example of an observational instrument from the nursing home industry, and it is the tool from which Aud, Rantz, Zwygart-Stauffacher, and Manion (2004) adapted their assisted living observational instrument. The observational tools examine visible indicators of the physical environment (lighting, odors, noise, furnishings), presentation of residents (grooming, services provided), and staff-resident interaction. While observational tools provide an objective third-party interpretation of the facility for consumer choice, large-scale implementation may be prohibitively expensive and has not yet been done.

Return to Contents

Provider-Reported Instruments

For instruments in this category, "provider" typically means the facility administrator, but it also can mean facility staff, such as for the National Nursing Home Survey (Jones, 2002; National Center for Health Statistics, 2004). Kane, Bershadsky, Kane, et al., (2004) developed a survey with the intent of determining how the assisted living setting affects the roles of the residents within the community. Other administrator tools, such as those developed by Utz (2003), Hawes, Phillips, and Rose (2000b), and the American Seniors Housing Association (2004), tend to assess more operational aspects of long-term care facilities, such as ownership status, interpretations of philosophical tenets of assisted living, services offered, facility policies, and financial performance, among others. Although information obtained directly from facility administrators or staff can influence consumer decisionmaking, the main goal in evaluating the provider-reported instruments was to identify services that are offered in the assisted living setting.

Return to Contents

Survey Methods

Mode of Administration

The majority of the instruments reviewed for this report collected information directly from the residents; however, several were designed to collect information from family members and administrators or through direct observation. The information was most often collected through self-administered (i.e., mail) questionnaires (SAQ), followed by in-person, interviewer administered surveys. Telephone surveys were used, although infrequently (Ejaz, Schur, and Fox, 2003; Hedrick, Guilhan, Chakpro, et al., 2005). There were also examples of surveys available in paper and pencil and Web-based formats. Incentives were not discussed in any of the studies.

Response Scales

More than 80 percent of the tools analyzed use Likert response scales (go to Appendix A [700 KB; PDF Help] for specific scales and reliability information, if available). The majority used a 1 to 5 point scale, though the range was 4 to 20 response options. Response categories included "poor to excellent," "not met expectations to far exceeded expectations," "strongly disagree to strongly agree," "satisfied to dissatisfied," and "never to always," "unlikely to likely," "worst to best", and "low to high." Other surveys offered mutually exclusive response options, for example, "yes/no," "agree/disagree" (Anderson, Connolly, Pratt, and Shapiro, 2003; Ryden, Gross, Savik, et al., 2000; Yee, Capitman, Leutz, and Sceigaj, 1999; Kruzich, Clinton, and Kelber, 1992). The nursing home survey developed by Castle (2004b) used a 10-item visual analogue response scale. A bipolar option such as yes/no or satisfied/unsatisfied arguably may be better in obtaining valid responses from the long-term care population, which is similarly advocated by Ejaz, Schur, and Fox (2003) and Yee, Dapitman, Leutz, and Sceigaj, (1999), and by Kane, Kling, Bershadsky, et al., (2003) in cases of severe cognitive impairment.

Psychometric Properties

The information about the psychometric properties of the scales included in this study is limited. In very few cases were we able to find full analyses using exploratory factor analyses, and no studies report confirmatory analyses. At best, when loadings from factor analyses are provided, the loadings are given for the questions included in the final questionnaire. Therefore, we cannot tell whether the dimensions shown are just the hypothesized dimensions or empirically based dimensions. In addition, most studies provided internal reliability information of scales and subscales. Reliability and validity information in the studies was limited and sporadic, as has been reported previously (Castle, 2004b). Details of information on reliability and validity are included in Appendix A for each instrument when available.

Return to Contents

Survey Summary

In developing instruments to help consumers evaluate facilities for choice, all three perspectives—consumer-reported, expert observations, and provider-reported—could provide valuable information for States, case managers, families, and importantly, potential assisted living residents.

Surveys used by State regulatory agencies and accreditation organizations were not included in this report. Tools designed to assess the physical and mental characteristics of residents (e.g., ability to perform ADLs) also were not a focus in this scan, although they may contribute to the analyses of data through case-mix adjustment. The Minimum Data Set is evaluated because of its use as part of the Resident Assessment Instrument and its inclusion of a resident-reported quality of life section in the forthcoming Version 3.0 (Anderson, Connolly, Pratt, and Shapiro, 2003). Also, some instruments reviewed for this report include questions that assess resident characteristics, although such questions were not the primary focus of the instrument. Appendix A (PDF File, 700 KB; PDF Help) provides a summary description of the instruments that were reviewed, including information on their source and purpose, mode of data collection, psychometric properties when available, response options, and item level detail for most of the instruments.

In Chapter 7 investigators take a closer look at the content covered in consumer-reported, observational, and provider-reported survey instruments used in the assisted living field and consider areas that are important to consumers, as identified in the previous chapter, that are addressed by these instruments. Content from surveys in other sectors of long-term care, such as nursing homes, residential care, and board and care homes, is also reviewed to supplement survey content.

Return to Contents
Proceed to Next Section