Prospects for Care Coordination Measurement Using Electronic Data Sources

Opportunities for Future Measurement of Care Coordination with Electronic Data

Although panelists identified many challenges associated with using data from health IT systems for care coordination measurement, they also identified several promising opportunities. In reviewing these opportunities, we differentiate between near-term opportunities—those that panelists expected could potentially be implemented in at least a pilot phase in the next 2-to-3 years—from long-term opportunities, which might be 3 or more years in the future. These timeframes are not concrete and are based on what panelists estimated, as well as whether or not the foundation for making use of a given health IT tool for measurement is in place at the time of this report.

Near-term Opportunities

In this section, we discuss measure concepts that panelists suggested would be feasible to develop and at least pilot test, if not fully implement, over the next 2-to-3 years. We also discuss specific obstacles that must be overcome to implement such measures.

Aligning Measures With Meaningful Use Data Elements

The measurement strategy recognized by almost all panelists as the most promising in the near-term is developing care coordination measures using data elements from the Meaningful Use Health IT incentive program measures.

The Meaningful Use incentive program, offered by CMS, has already garnered much attention in the health care industry, and in particular among users and potential adopters of EHR systems. The influence of the Meaningful Use program on implementation of health IT is expected to grow over the coming years, as more hospitals and outpatient providers begin participating in the program.


The expected widespread participation in Meaningful Use is likely to drive further standardization within the industry, at least with respect to data elements required for the Meaningful Use measures. Tables 1, 2, and 3 list Stage I and Stage II Meaningful Use measures that contain data elements that may be useful for care coordination measurement. These data elements should become widely available, most in a structured format, within EHR systems as participation in the Meaningful Use program grows. Although the data elements identified here may be used in a wide variety of ways, we highlight some of the ways in which they are most likely to be useful for measuring care coordination processes. Specifically, we note elements that are likely to be useful for calculating numerators or identifying denominator populations and point to specific care coordination activity domains from the Care Coordination Measures Atlas framework that the data element might help measure.3

The Stage I Meaningful Use measures are divided into two sets: core measures that must be met, and menu set measures, from which participants may choose five measures in order to fulfill the Stage I requirements. Because they are required for all participants in the Meaningful Use program during their first year of participation, data elements from the Stage I core measures will be the most widely available the soonest (Table 1).x

In addition to the core measures, during Stage I, participants in the Meaningful Use program must also choose five measures to report from among the Stage I menu set. Thus, data elements from these menu set measures may not be as widely available as those from the core set, but many likely will still be in use (Table 2).

Stage I of Meaningful Use also requires reporting of Clinical Quality Measures using EHR data. These measures focus on performance of specific clinical processes, such as mammography screening, asthma assessment, and appropriate therapy for colorectal and breast cancer. Their focus on specific diseases and conditions and clinical processes makes them less useful for assessing coordination processes. However, the expected widespread availability of elements required for these measures within EHR systems as structured data offers another set of data elements available to review for care coordination measurement development and testing. Guidance for calculating the Clinical Quality Measures, all of which predate the Meaningful Use program, is available using the NQF Quality Data Model format, which seeks to enable quality measurement using electronic data (Appendix D).

Data elements listed in Table 3 are likely to be widely available as more Meaningful Use participants reach Stage II. Although these measures currently are recommendations, it is expected that the final set of measures will be very similar to these recommendations. Note that the ONC Health IT Policy Committee recommended that implementation of Stage II Meaningful Use be delayed until 2014 for those sites that attest to Stage I Meaningful Use in 2011. Thus, although specification of Stage II Meaningful Use elements will be available much sooner (final rule to be released in June 2012), actual implementation and availability of these data elements will not generally be available until late 2013 and early 2014. Specifications should be sufficient to begin measure development prior to the full implementation of Stage II Meaningful Use, but empirical testing may be limited to advanced sites until 2014, when wider implementation is underway.

The Meaningful Use measures will enable further care coordination measurement using data from EHRs to the extent that the Meaningful Use incentive program stimulates capture of structured data for all patients. Capturing structured data for only a percent of the patient population, for example the minimum thresholds specified in the Stage I and Stage II measures, would be insufficient to facilitate broader quality measurement. Likewise, health care entities may meet the Meaningful Use requirements of some measures by demonstrating use of some structured data elements without structuring all elements of a particular data field, which would be insufficient for broader quality measurement. For example, in Stage I, only one entry on an active medication list is required to be structured, but to be useful as a data element for broader quality measurement, all entries on a medication list should be recorded as structured data. However, it is expected that improvements in the design and use of EHRs in order to fulfill the Meaningful Use measures will stimulate usage beyond the minimum standards established by the Stage I and II measures.

The Meaningful Use measures provide a starting place for developing new measures of care coordination processes that rely on EHR data. Further development is needed for some data elements, in particular specification of information to be included in and coding systems used for clinical summaries, summary of care records, care summaries, and a plan of care. Such specifications will likely emerge, and open up new possibilities for measurement efforts, as the Meaningful Use measures are widely adopted and implemented. For example, specifications are currently under development through the Transitions of Care Initiative to facilitate transfer of summary of care records, including standard definitions of data elements and which elements must at a minimum be exchanged. Additional information is available in Appendix D.

Measuring Information Transfer

Several additional near-term measurement opportunities identified by panelists focused on confirming transmission of information during care transitions. For all these measure concepts, although evidence of information transfer alone is insufficient to establish that care is well coordinated, it provides insight into one important step in care coordination.

One suggested measure concept involves using CCD/CCR messages to confirm that information was transferred from the discharging hospital to the primary care clinic at the time of patient discharge. Presence of a CCD/CCR message from the hospital within a primary care clinic EHR would indicate a basic level of information transfer. If more detail were desired about the kinds of information transmitted, for example, a medication list or problem list, the section headings within a CCD/CCR message could be used to confirm that that information was included with the transmission. These measure ideas would not attempt to evaluate the contents of CCD/CCR messages, but rather confirm that certain categories of information were transmitted between particular participants in a patient's care (e.g., hospital and primary care provider) at key care transitions. Health information exchanges that contain a data repository could be used to identify CCD/CCR messages, or EHR data could be used to examine CCD/CCR messages received.

Given that use of CCD/CCR messages is still evolving, panelists also suggested looking for any information about outside care within a primary care EHR as evidence of information transfer and a basic level of coordination. For example, panelists suggested that, if a patient is known to have visited the emergency room but the primary care EHR contains no information on that visit within a certain amount of time, this indicates poor coordination. Such measurement would require clear specification of the kinds of outside information of interest, such as discharge summaries, clinical summaries, transition of care records, CCD/CCR messages, or lab or imaging results from an outside system. Given the heterogeneity in how such information is stored within EHR systems, each clinic would likely need to develop their own algorithm for identifying it. However, broadening the scope of information format beyond CCD/CCR messages may be more feasible in the current health IT environment.

An important limitation in such measures of information transfer noted by some panelists is that they fail to measure whether or how information is used, an important consideration in understanding if care coordination has been achieved. One measure concept suggested to address this was use of audit files to evaluate whether information transferred from other settings is viewed by primary care providers. In order to comply with privacy regulations, EHRs typically contain an auditing feature that tracks which users access which information at particular times. These audit files could be used to confirm that someone within a primary care practice opened a discharge summary within a certain amount of time after it was received, or that a clinical summary from a specialist visit was viewed prior to or at the time of a follow-up visit. Although they suggested measurement using audit files is feasible, our panelists were not aware of any efforts to use them for quality measurement to date, and emphasized that methods would need to be developed to extract the necessary information from audit files (likely requiring IT system administration assistance) and to assess the accuracy of information within audit files for quality measurement purposes. One potential problem brought up by a panelist was accuracy of information about the origin of transitions documents (provenance), such as discharge summaries, that are integrated into an EHR. The provenance of documents forwarded through multiple systems or providers would be important to understand.

In all the measures suggested that focus on information transfer at the time of patient transitions, claims data would be needed to identify those transitions. For example, claims data would be needed to identify when patients were discharged from the hospital or readmitted or when they visited an outpatient specialty provider. Health information exchanges, when they include a data repository and incorporate claims data, would be a particularly useful source for such measures because they include clinical information from EHRs and claims data in a single source. All-payer claims databases are also particularly attractive for this purpose because they contain data from all health care settings, an essential tool for identifying a wide range of patient transitions. However, linking APCD data with EHRs is currently only possible within the limited number of States that collect identified patient information within their APCD. Privacy issues would need to be resolved before such linkage could be achieved. For patients with commercial insurance, payer files would also likely contain sufficient information to identify most care transitions. However, transitions to and from services not covered by the payer, such as behavioral health services for payers with mental health benefit carve-outs, would not be reflected in single-payer data.

A further consideration for any measures that focus on transmission of information to or from the primary care clinic or a patient-centered medical home is the ability to attribute a patient to a particular primary care provider or home. Panelists emphasized that most EHRs do not identify a PCP for patients because they use an encounter-based model that does not easily capture those kinds of longitudinal care concepts. But panelists with experience attributing patients to PCPs emphasized that this is not an insurmountable obstacle to such quality measure concepts. One panelist with experience administering an HIE provided information about the algorithm developed to attribute patients to PCPs within the exchanges' data repository and reported that the method had a high degree of accuracy (approximately 90% agreement between the algorithm and primary care providers themselves). Thus, although attention must be paid to this issue during measure development, it is unlikely to hamper use of such measures. Another informant suggested that, in the long-term, issues of attribution would be less problematic if future EHR standards would encourage or require capture of longitudinal care concepts, such as designation of a PCP or medical home.

Using Claims Data for Measurement

Panelists suggested several ways in which claims data, particularly from all-payer claims databases, could be used to evaluate care coordination. Because claims data focus on services received (events) rather than processes of care, these measure suggestions focused on intermediate outcomes that might be indicative of poor coordination.

One concept suggested is presence of follow-up appointments within an expected time frame of a particular event, such as discharge from the hospital or performance of a particular surgical procedure. Another concept suggested was evaluation of redundant testing. For example, claims for the same imaging study or lab test from different facilities or ordered by different providers within close proximity to one another might indicate failure to share test results across settings. Any such measures would likely need to be limited to particular tests within specific instances, such as repetition of a particular imaging study within a set timeframe surrounding an inpatient admission for a specific diagnosis, or a particular lab test performed at both a primary care and specialty clinic during a timeframe within which the results would not be expected to change. One panelist noted that an important limitation in such measures is ambiguity in claims data about which provider or institution performs a test vs. which bills for the test. This issue would need to be investigated during measure development.

All-payer claims databases would be particularly useful for such measures because they aggregate claims for care received in most settings of the health care system. Health information exchanges with a data repository and that include claims data might also be used for such measures. In both cases, completeness of the data source must be carefully considered. For example, an HIE that contains claims data from only 50% of ambulatory care providers would not be sufficient to confirm that a follow-up appointment occurred.

If these measures relied on only claims data, they would be limited by lack of clinical context. Panelists emphasized that the ability to link claims data to clinical information from within EHRs, whether by linking EHRs with APCDs or through an HIE that contains claims data, would greatly enhance such measures. The additional clinical context was seen as particularly important for ensuring face validity among clinicians. Issues of patient matching would need to be resolved in attempting to link claims with clinical data, but these challenges have already been addressed in some HIEs. Privacy concerns and other regulatory hurdles that limit use of identified patient data pose a greater challenge in the near-term, but panelists agreed that these barriers are also likely to be overcome in the next few years as the need for linked data is more widely recognized.

Panelists also discussed measures related to what care coordination is expected to achieve or influence positively. For example, other ideas suggested for claims-based care coordination measurement included hospital readmissions, adherence to guidelines for episodes of care where well-established and fairly standard processes of care exist, and adherence with guidelines for pharmacotherapy of certain chronic conditions. Such measures would not directly capture coordination processes, but rather would provide an indirect view of coordination through events potentially related to adequacy or inadequacy of coordination and thus might be considered proxy measures of coordination processes. The focus of this report, however, is on electronic data source opportunities for measuring coordination processes directly, so further detail on these ideas from panelists is not covered in this report.


Summary of Near-term Measure Opportunities

Table 4 summarizes the measure opportunities identified by panelists as likely feasible in the next 2-to-3 years. A common thread among most near-term measurement possibilities is a focus on the transfer of information. Panelists agreed that potential exists to measure documentation and transmission of some kinds of information that is likely to be useful in coordinating care, but that other dynamic processes of care coordination, such as interpersonal communication or some supports for self-management, are not likely to be measurable with health IT data sources because they are not well documented or easily captured as part of care encounters.

Although these measures represent the most promising opportunities identified from our panel review, they still require some additional efforts. To implement several of these measure concepts, methods must first be developed to link clinical and claims data and the reliability and accuracy of any such linkage examined. Because many of the data elements needed for these suggested measures have not been used in prior quality measurement efforts (to our knowledge), the validity and quality of the specific data elements used would also need to be investigated. For example, it would be necessary to examine whether information needed from claims data is routinely collected by all payers and what level of specificity for document types, provenance, and viewer are typically included within EHR audit files. The accuracy of data automatically extracted from EHRs must also be assessed, for example, by comparison with manual chart review. The published literature may contain some information on validity, reliability, and accuracy of some data elements or sources, but in the absence of published literature, new investigations would be required. Finally, as with all measure development efforts, measures must be carefully specified with clear definitions of numerator, denominator, and exclusions. In some cases, risk adjustment would also be needed. Clinical input is essential during such development, as well as evidence from published literature and evidence-based guidelines, when available.

Return to Contents

Long-term Opportunities

Because the health IT field is developing rapidly and care coordination measurement is still in its infancy, estimates of long-term prospects for measuring care coordination using health IT data are likely to change within the next few years. Nevertheless, our discussions with panelists revealed a number of possible avenues for such measurement which, though likely not feasible in the near-term, present a promising possibility as both fields further develop. These approaches are necessarily less well defined than the near-term opportunities identified in the previous section.

During one of the group calls, panelists discussed the challenge in assessing whether care is coordinated as an intermediate outcome rather than measuring discrete actions or processes that are believed to be important for coordinating care, but may not individually be sufficient to achieve coordinated care. Evidence that data are being linked across sites or across providers was suggested as one potential structural indicator related to care coordination. Although panelists debated where to look for such evidence of linkage, there was some agreement that aggregated information from multiple sources should be located within whatever entity is primarily responsible for coordinating care, whether that is in the EHR of a primary care provider, long-term care facility, or the registry of another responsible entity such as an insurance provider. However, there was no agreement among panelists about where within an EHR this information might be found or how it might be structured. Overall, using aggregation of clinical information from multiple settings as evidence of care coordination will likely require further development of interoperability infrastructure, the evolution of EHRs and how data are recorded within them, and further conceptual development around what constitutes coordinated care. Dovetailing efforts to improve coordination and simultaneously record and measure related activities is part of the promise of health IT, which to date has been focused more on clinical processes (e.g., preventive screening) and less on management processes (e.g., patient and information flow) that support clinical activity.

Another idea suggested as an indicator of poor coordination is lack of documentation in a coordinating practice's EHR (e.g., of a primary care provider or medical home) regarding health care utilization in other settings. The specific example was provided of a patient with multiple visits to outside specialists about which no information can be found in the PCP's EHR. This concept could also be applied to evaluate a PCP's awareness of patients' hospital admissions, ED visits, or behavioral health visits. The denominator of such a measure would be based on knowledge about what health care services individual patients have used, requiring claims data or other information on health care utilization from across settings.

A limited application of this kind of measure is likely to be feasible in some cases within the next few years (see near-term opportunities section), but broader application is likely more distant. For example, to be applicable to a broad patient population, such measures would require use of APCD data for the denominator, rather than claims data from a single payer or from a particular HIE. (Only mature HIEs that include a data repository and incorporate claims data would be sufficient for use in the denominator). Currently, the ability to link APCD data with outside data sources, such as EHRs, is possible in only a limited number of States that collect identified data. Issues of patient matching patients across claims and EHR data would pose a challenge, but one panelist noted that matching is done regularly by many health plans to enable patient outreach by clinicians.

Furthermore, given the wide variation in how outside information is integrated into current EHR systems, near-term measurement would likely be restricted to specific information types and formats, such as a CCD/CCR message or a hospital discharge summary. Given that there is currently no usual location where outside information is stored within EHRs, and the possibility that information might be present within text-based notes that could not be assessed through automatic data extraction, confirming the absence of information about outside health care utilization will pose a significant challenge. As industry standards evolve for incorporating discharge summaries, clinical summaries, outside lab and imaging results, pharmacy data, and transition of care documents, it may be feasible to capture a wider range of data integration (and lack thereof) within such measures.

One panelist noted that linking EHRs to patient registries would offer another potentially rich data source for quality measurement. This not only would improve care at the point of care—for example, information on immunizations obtained at a public health fair might be available to the primary care physician—but could also provide more comprehensive data for quality measurement. However, panelists emphasized that near-term measurement using patient registries, with or without linking to EHRs, is not likely. They noted that although particular registries contain some data elements that would be useful for care coordination measurement, the lack of standard design or data elements included in various registries makes it impractical to design care coordination quality measures around registry data at this time. Looking forward, an ongoing project funded by AHRQ aims to develop a Registry of Patient Registries (RoPR), similar to, which would provide searchable information about the focus, content, design, and stewardship of many patient registries in the country (go to Appendix D for more information). This project is still in development, but in the future may help facilitate integration of registry data with other data sources, such as EHRs, by making it easier for users to identify relevant registries, and encouraging use of standardized core data elements across registries.

Return to Contents

Summary of Measurement Opportunities

Panelists recognized that the demands for quality measurement have grown substantially in recent years and noted the challenge in keeping up with quality measurement initiatives that are not always synchronized. This has put considerable measurement burden on clinicians and systems administrators, and, increasingly, health IT vendors, who must keep pace with a dynamic measurement field in order to facilitate measurement using their products. Panelists encouraged harmonizing measurement efforts as much as possible to ease this burden. Aligning new measures of care coordination with data elements required for Meaningful Use measures is one example of this strategy. In the field of care coordination specifically, supporting research investigating how coordination activities relate to key outcomes would also help reduce measurement burden by focusing measurement efforts around a limited number of processes or concepts known to be important. Finally, one panelist summed up his recommendations in this way: "Don't ask for too many measures, don't make them too complex, incrementally increase the kinds of data needed for quality measures."

x. Eligible providers, hospitals, and critical access hospitals may begin participating in Meaningful Use in 2011.

Return to Contents
Proceed to Next Section

Page last reviewed July 2018
Page originally created September 2012
Internet Citation: Opportunities for Future Measurement of Care Coordination with Electronic Data. Content last reviewed July 2018. Agency for Healthcare Research and Quality, Rockville, MD.
Back To Top