Two different and complementary approaches are used during the development of a CAHPS survey to assess its reliability and validity:
1. Cognitive testing, which bases its assessments on feedback from interviews with people who are asked to react to the survey questions.
2. Psychometric testing, which consists of analyses of data collected using the questionnaire.
Cognitive testing provides useful information about respondents' comprehension of the questions, their ability to answer the questions, and the adequacy of the response choices. It also helps identify words that can be used to describe health care providers accurately and consistently across a range of consumers (e.g., commercially insured, Medicaid, fee-for-service, managed care, lower socioeconomic status (SES), middle SES, low literacy, higher literacy) and explores whether key words and concepts work equally well in both English and Spanish.
Field tests and psychometric analyses provide information about the items' reliability and validity. Many existing questionnaires about health care have been tested primarily or exclusively using a psychometric approach, but the CAHPS team views the combination of cognitive and psychometric approaches as essential to producing the best possible survey instrument.
- Hays RD, Shaul JA, Williams VSL, Lubalin JS, Harris-Kojetin LD, Sweeny SF, Cleary PD. Psychometric properties of the CAHPS 1.0 survey measures. Consumer Assessment of Health Plans Study. Med Care. 1999 Mar;37(3 Suppl):MS22-31.
- Hargraves JL, Hays RD, Cleary PD. Psychometric properties of the Consumer Assessment of Health Plans Study (CAHPS) 2.0 adult core survey. Health Serv Res. 2003 Dec;38(6 Pt 1):1509-27.
- Morales LS, Weech-Maldonado R, Elliott MN, Weidmer B, Hays RD. Psychometric properties of the Spanish Consumer Assessment of Health Plans Survey (CAHPS). Hisp J Behav Sci. 2003;25(3):386-409.
- Solomon LS, Hays RD, Zaslavsky AM, Ding L, Cleary PD. Psychometric properties of a group-level Consumer Assessment of Health Plans Study (CAHPS) instrument. Med Care. 2005 Jan;43(1):53-60.
- Keller S, O'Malley AJ, Hays RD, Mathew RA, Zaslavsky AM, Hepner KA, Cleary PD. Methods used to streamline the CAHPS Hospital Survey. Health Serv Res. 2005 Dec;40(6 Pt 2):2057-77.
- Levine RE, Fowler FJ Jr, Brown JA. Role of cognitive testing in the development of the CAHPS Hospital Survey. Health Serv Res. 2005 Dec;40(6 Pt 2):2037-56.
- Carman KL, Short PF, Farley DO, Schanaier JA, Elliott DB, Gallagher PM. Epilogue: Early Lessons from CAHPS Demonstrations and Evaluations. Med Care. 1999 Mar;37(3 Suppl):MS97-105.