Development of the CAHPS Health Information Technology Item Set

The development process for the CAHPS Health Information Technology (HIT) Item Set included several steps:

Stakeholder meetings. In June 2006, AHRQ hosted a meeting with HIT and CAHPS survey experts, as well as stakeholders representing health care organizations and consumers. This group discussed the use of HIT by physician practices, the aspects of patient care likely to be affected by HIT, the areas that consumers would be best suited to comment on, and sources of related information, including potential items. Stakeholders convened again in September 2008 to provide feedback on the item set domains, options for field-testing, and issues with the adoption and use of the item set.

Learn more in this summary of the: HIT-CAHPS® Stakeholders Meeting, June 28, 2006 (PDF, 293 KB).

Literature review. The CAHPS team completed a literature review of interactions between HIT, health care providers, and patients.

Focus groups. The CAHPS team conducted focus groups with HIT users. The goals of the focus group sessions were to explore:

  • What health care functions are important to patients;
  • How patients view HIT’s role in facilitating those functions;
  • What patients believe they are able to report on regarding HIT; and
  • What patients regard as the most important roles for HIT in a physician practice.

Item inventory. Based on the expert meeting, literature review, and focus groups, the grantees developed an inventory of items, sorted them by domains, and set priorities for the domains. Those priorities determined which items were selected for cognitive testing.

Site interviews. The team conducted in-depth interviews with health plans and providers that use HIT. The interviews explored issues around electronic and personal health record architecture, functions, and enrollment.

Cognitive testing. The items were cognitively tested with consumers to ensure that they can accurately understand and interpret the questions and response options. After two rounds of cognitive testing, the team incorporated the findings into an item set that can be used for testing on a larger scale.

Field testing. The HIT Item Set was field tested in three health care organizations: Beth Israel Deaconess Medical Center in Massachusetts, Group Health Cooperative in Washington, and Kaiser Permanente in Southern California. Using data from those three sites, the CAHPS Team analyzed the reliability, validity, and other psychometric properties of the items and composite measures.

Composite label testing. The composite measure labels were tested with consumers to ensure that consumers understood the labels correctly.

Public release. In 2011, AHRQ released the CAHPS Health Information Technology Item Set for use with the original CAHPS Clinician & Group Survey. These supplemental items were revised slightly in 2012 and 2013 to align the questions and placement instructions with the 2.0 version of the Clinician & Group Surveys and related supplemental items. In 2017, the items and placement instructions were modified to fit with the 3.0 version of the Clinician & Group Survey.

The CAHPS Consortium would like to acknowledge the following organizations for their support of the development of this item set: Beth Israel Deaconess Medical Center, Group Health Cooperative, and Kaiser Permanente Southern California.

Page last reviewed August 2017
Page originally created October 2011
Internet Citation: Development of the CAHPS Health Information Technology Item Set. Content last reviewed August 2017. Agency for Healthcare Research and Quality, Rockville, MD.