Although only measures from the patient/family perspective were considered for inclusion in the accountability measure set, measures from the health care professional and system representative perspectives that may be appropriate for quality improvement purposes are reviewed here. These measures may be useful in guiding improvements if the patient/family perspective accountability measures suggest short-comings. Before being used for accountability purposes, methods of auditing the responses on these tools would be desired, to balance their reliance on self-assessment by health care professionals and system representatives. These tools may also be useful for research purposes, depending on the focus and scope of the research study.
Recognizing that the choice of measures involves many trade-offs, suggestions are offered here based on different measurement priorities anticipated by different types of measure users.
Pediatric Measures From Health Care Professional Perspective
If comprehensive measurement is of chief interest, consider the Family-Centered Care Self-Assessment Tool—Provider Version.
Atlas Measure Number: 11b
Description: This measure surveys health care professionals and staff about the provision of family-centered pediatric care in their practices. No information is available on typical completion times for the 105-item self-assessment instrument. No reliability or validity testing has been reported among the sources identified. Due to lack of psychometric testing, the measure developers do not recommend combining responses across items to develop mean or aggregate scores for groups of conceptually-related instrument items. Rather, they recommend reporting percent responses to individual items only. Therefore, while the instrument should be administered as a whole, responses on items most relevant to care coordination may be reported separately.6,7
Strengths for Quality Improvement Purposes: The measure is designed for use as a quality improvement tool. It provides comprehensive assessment of care coordination, with three or more items mapped to all but one Atlas activity sub-domain (Facilitate Transitions Across Settings).
Weaknesses for Quality Improvement Purposes: No information is available on validity or reliability of the measure. Although no information is available on typical completion times, the lengthy instrument may add a large measurement burden on busy clinicians.
For More Information: Additional information on the measure properties is summarized in the Atlas measure profile as well as in Appendix C of this document. A copy of the measure instrument may be obtained by visiting the Care Coordination Measures Atlas Appendix IV. To use the measure, written permission must be obtained from the measure developer. A user guide is also available on request from the measure developer. Contact information is available in the Atlas Appendix IV
If consistency with the pediatric patient/family perspective accountability measure is of chief interest, consider the Primary Care Assessment Tool—Provider Edition (PCAT-PE).
Atlas Measure Number: 17d
Description: This measure surveys physicians about care delivery in their practices. No information is available on typical completion times for the 153-item self-administered survey. The survey covers many aspects of primary care delivery, including but not limited to care coordination. No information is available on reliability or validity, although some testing has been performed on the closely related patient surveys (PCAT-CE and PCAT-AE).
Strengths for Quality Improvement Purposes: The survey is closely related to the PCAT-CE, which will improve comparability of responses across the two instruments. Results of the provider version survey may be particularly useful in guiding improvement initiatives that aim to improve performance on the related child version of the survey (used for accountability purposes).
Weaknesses for Quality Improvement Purposes: The measure's focus extends beyond care coordination to include many aspects of primary care (39 of 153 measure items do not map to any Atlas care coordination activity domain), which increases measurement burden on busy clinicians. However, if measurement burden is less of a focus, this may be viewed as a strength because it will provide more comprehensive information about practice characteristics. The instrument does not measure two Atlas activity domains, (Establish Accountability or Negotiate Responsibility, and Create a Proactive Plan of Care), nor the sub-domain (Facilitate Transitions as Coordination Needs Change) so it will not be useful for guiding improvement initiatives related to these coordination mechanisms.
For More Information: Additional information on the measure properties is summarized in the Atlas measure profile and Appendix C of this document. A copy of the measure and user guide may be obtained by contacting the measure developer. Contact information is available from Appendix IV of the Atlas.
If validity/reliability is of chief interest, none of the health care professional measures identified through the inclusion, exclusion, and prioritization process had well established reliability and validity. For more information about the selection criteria as well as detailed methodology, refer to Appendix B.
If feasibility is of chief interest, there is insufficient information about the measurement burden on available measures to make a recommendation based on feasibility.
If an alternative to survey-based measurement is of interest: consider the Care Coordination Measurement Tool (CCMT).
Atlas Measure Number: 5
Description: This tool relies on self-report by health care professionals to collect detailed information about coordination activities carried out in a clinic. Specifically, it collects information on actions, resource use, outcomes, and time associated with individual patient encounters that included an element of care coordination. The measure was designed for use by clinicians, but there is no report of its impact on clinical work flow or completion rates. No information was identified regarding validity or reliability.
Potential for Accountability Purposes: If methods of auditing or validating self-reported information were developed, the CCMT might be useful as an accountability measure. However, note that validity and reliability remain unknown at this time.
Strengths for Quality Improvement Purposes: The tool may be useful for quality improvement purposes by providing detailed information about the types of coordination-related activities performed in a clinic.
Weaknesses for Quality Improvement Purposes: Measurement burden may be a concern and some time is likely required to train clinicians in the use of the tool. In one study, use of the instrument required 2-hour training sessions prior to data collection and ongoing technical support throughout the data collection period. Generally, practice personnel were able to incorporate accurate use of CCMT into clinical workflow after 1 week of using the instrument. No information is available on validity and reliability of the tool. It would be particularly important to assess inter-rater reliability among various users of the tool in a particular clinic.
For More Information: Additional information on the measure properties is summarized in the Atlas measure profile and Appendix C of this document. To use the measure, written consent must be provided by the measure developer. A copy of the measure and contact information for the measure developer are available in Appendix IV of the Atlas.
Adult Measures From the Health Care Professional Perspective
After applying the inclusion and exclusion criteria, only one adult measure is available from this perspective for primary care practice evaluation: the Primary Care Assessment Tool—Provider Edition (PCAT-PE). For more information about the selection criteria as well as detailed methodology, refer to Appendix B.
Atlas Measure Number: 17d
Description: This measure surveys physicians about care delivery in their practices. No information is available on typical completion times for the 153-item self-administered survey. The survey covers many aspects of primary care delivery, including but not limited to care coordination. No information is available on reliability or validity, although some testing has been performed on the closely related patient surveys (PCAT-CE and PCAT-AE).
Strengths for Quality Improvement Purposes: The survey is closely related to the PCAT-AE, which will improve comparability of responses across the two instruments. Results of the provider version survey may be particularly useful in guiding improvement initiatives that aim to improve performance on the related adult version of the survey. (the PCAT-AE is a measurement option for accountability purposes for the patient/family perspective.)
Weaknesses for Quality Improvement Purposes: The measure's focus extends beyond care coordination to include many aspects of primary care (39 of 153 measure items do not map to any Atlas care coordination activity domain), which increases measurement burden on clinicians. However, if measurement burden is less of an interest, this may be viewed as a strength because it will provide more comprehensive information about practice characteristics. The instrument does not measure two Atlas activity domains, (Establish Accountability or Negotiate Responsibility and Create a Proactive Plan of Care) nor the sub-domain (Facilitate Transitions as Coordination Needs Change), so it will not be useful for guiding improvement initiatives related to these coordination mechanisms.
For More Information: Additional information on the measure properties is summarized in the Atlas measure profile and Appendix C of this document. A copy of the measure and user guide may be obtained by contacting the measure developer. Contact information is available in Appendix IV of the Atlas.
Pediatric Measures From the System Representative Perspective
If validity/reliability is of chief interest, consider the Medical Home Index (MHI).
Atlas Measure Number: 16a
Description: This 25-item instrument asks system representatives (administrators or clinicians reflecting on system characteristics) to assess the level of progress towards providing a medical home for a particular practice. Some items require both a physician and non-physician staff member's perspective. Both pediatric and adult versions of the instrument exist. Typical completion time for the measure is 30-to-45 minutes. The measure has strong reliability and validity, including an association between higher total MHI score (indicating better adherence to medical home model) and lower hospitalization rates. The measure is strongly focused on care coordination; all items map to an Atlas care coordination activity domain. It covers all activity domains except Monitor, Follow-up, and Respond to Change.
Strengths for Quality Improvement Purposes: The MHI is designed and has been used for quality improvement purposes.11 The instrument is designed to stimulate self-reflection and to guide improvement by specifying more advanced levels of medical home implementation.
Weaknesses for Quality Improvement Purposes: The measure includes only one or two items for most activity domains. It does not measure the Monitor, Follow-up, and Respond to Change domain.
For More Information: Additional information on the measure properties is summarized in the Atlas measure profile and Appendix C of this document. Permission to use the instrument requires written consent from the measure developer. A user guide is also available on request from the measure developer. A copy of the measure and contact information for the measure developer are available in Appendix IV of the Atlas.
If feasibility is of chief interest, consider the Medical Home Index (MHI). Typical completion time is 30 to 45 minutes. A 10-item short version of this survey is also available (Atlas measure # 16b) which offers reduced measurement burden, albeit at the expense of comprehensive domain coverage. See measure details above.
If in-depth measurement is of chief interest, consider the Primary Care Assessment Tool—Facility Expanded Edition (PCAT-FE). Although it does not cover every Atlas activity domain, it does include at least three items for most domains, offering a more comprehensive assessment of care coordination mechanisms.
Atlas Measure Number: 17c
Description: This measure surveys practice administrators (or other representatives of a facility) about care delivery in their practices. No information is available on typical completion times for the 153-item self-administered survey. The survey covers many aspects of primary care delivery, including but not limited to care coordination. No information is available on reliability or validity, although some testing has been performed on the closely related patient surveys (PCAT-CE and PCAT-AE).
Strengths for Quality Improvement Purposes: The survey is closely related to the PCAT-CE, which will improve comparability of responses across the two instruments. Results of the provider version survey may be particularly useful in guiding improvement initiatives that aim to improve performance on the related child version of the survey (used for accountability purposes). The measure includes at least three items for most domains, offering a more in-depth assessment and more detail that may be useful for guiding improvement initiatives.
Weaknesses for Quality Improvement Purposes: The measure's focus extends beyond care coordination to include many aspects of primary care (39 of 153 measure items do not map to any Atlas care coordination domain), which increases measurement burden. However, if measurement burden is less of an interest, this may be viewed as a strength because it will provide more comprehensive information about practice characteristics. The instrument does not measure two Atlas activity domains, Establish Accountability or Negotiate Responsibility, and Create a Proactive Plan of Care nor the sub-domain Facilitate Transitions as Coordination Needs Change, so it will not be useful for guiding improvement initiatives related to these coordination mechanisms.
For More Information: Additional information on the measure properties is summarized in the Atlas measure profile and Appendix C of this document. A copy of the measure and user guide may be obtained by contacting the measure developer. Contact information is available in Appendix IV of the Atlas.
If consistency with the pediatric patient/family perspective measure is desirable, consider the Primary Care Assessment Tool—Facility Expanded Edition (PCAT-FE). See measure details above.
Adult Measures From the System Representative Perspective
If a focus on care coordination is of chief interest, consider the Medical Home Index (MHI). It is strongly focused on care coordination, with all items mapped to an Atlas care coordination domain.
Atlas Measure Number: 16a
Description: This 25-item instrument asks system representatives (administrators or clinicians reflecting on system characteristics) to assess the level of progress towards providing a medical home for a particular practice. Some items require both a physician and non-physician staff member's perspective. Both pediatric and adult versions of the instrument exist. Typical completion time for the measure is 30 to 45 minutes. The measure has strong reliability and validity, including an association between higher total MHI score (indicating better adherence to medical home model) and lower hospitalization rates.12 The measure is strongly focused on care coordination; all items map to an Atlas care coordination domain and it covers all activity domains except Monitor, Follow-up, and Respond to Change.
Strengths for Quality Improvement Purposes: The MHI is designed and has been used for quality improvement purposes.11 The instrument is designed to stimulate self-reflection and to guide improvement by specifying more advanced levels of medical home implementation.
Weaknesses for Quality Improvement Purposes: The measure includes only one or two items for most activity domains. It does not measure the Monitor, Follow-up and Respond to Change domain.
For More Information: Additional information on the measure properties is summarized in the Atlas measure profile and Appendix C of this document. Permission to use the instrument requires written consent from the measure developer. A user guide is also available on request from the measure developer. A copy of the measure and contact information for the measure developer are available in Appendix IV of the Atlas.
If validity and/or reliability are of chief interest, consider the Assessment of Chronic Illness Care (ACIC). This measure has well-established validity and demonstrated links to outcomes, including attributable risk of coronary heart disease, hemoglobin A1c values, cholesterol levels and cardiac events among patients with diabetes and heart disease. It also has been shown to be sensitive to system improvements made during quality improvement initiatives.
Atlas Measure Number: 1
Description: This 34-item instrument is designed to be completed by a team of representatives (health care professionals and/or system representatives) from a practice. It assesses the level of support for chronic illness care. No information is available about typical completion times; this likely depends on the method of administration (e.g., team members discuss and reach consensus together or complete separately and average scores). The measure has very strong validity. Higher total and sub-scale scores, indicating better support for chronic illness, have been associated with lower attributable risk of coronary heart disease and lower hemoglobin A1c values among patients with diabetes.13 Quality improvement initiatives that were associated with increases in ACIC scores were also associated with improvements in composite measures of cholesterol levels and hemoglobin A1c scores among diabetics and cholesterol levels and cardiac events among patients with heart disease.13-15
Strengths for Quality Improvement Purposes: The ACIC is designed and has been used for quality improvement purposes.16,17 The instrument specifies more advanced levels of support for chronic illness care, which may help guide improvement initiatives. It has been shown to be sensitive to system improvements made during quality improvement initiatives.
Weaknesses for Quality Improvement Purposes: The measure is not focused exclusively on care coordination (9 of 34 items do not map to an Atlas activity care coordination domain). It does not provide information about the transition of care.
For More Information: Additional information on the measure properties is summarized in the Atlas measure profile and Appendix C of this document. No permission is needed by the measure developer for non-commercial quality improvement work or research. However, if you intend to use it for other purposes you must obtain written permission from the Group Health Cooperative through its MacColl Center. A copy of the measure and contact information are available in Appendix IV of the Atlas.
If feasibility is of chief interest, consider the Medical Home Index (MHI). Although no information is available on typical completion times for the ACIC, the shorter length of the MHI, combined with its usual completion time of 30 to 45 minutes, suggest that it may offer a lower measurement burden than the ACIC. In addition, a 10-item short version of this instrument is also available (Atlas measure #16b), which offers reduced measurement burden, albeit at the expense of comprehensive domain coverage. See measure details above.
Table 4 summarizes the measures recommended as companion measure options for quality improvement uses for the health care professional and system representative perspectives. These measures are not recommended for accountability purposes.