Reasonable expectations for consumer reporting
This report presents the draft consensus recommendations from the Technical Expert Panel (TEP) and illustrates potential reporting systems described by these recommendations. As discussed above, while reporting of patient safety events is often associated with hospitals-based incidents, these recommendations are applicable to patient safety events in all health care settings. TEP members indicated that the consumer perspective may be particularly important for patient safety events that occur during health care transitions (i.e., when care for a patient transfers from one provider or health care organization to another). Transitions are thought to be high-risk intervals for patient safety events, and often no single health care provider or organizations has complete information on or "ownership" of the transition. As such, consumers may be able to provide much more information on events occurring during transitions.
A broader question discussed by certain TEP members and external reviewers focused on what are reasonable expectations for accomplishments from consumer reporting, and whether development of consumer reporting systems would be worth the required investment. Some level of doubt was expressed regarding the potential impacts of consumer reporting systems on improving patient safety. Multiple reporting systems have been developed and implemented over the past decade, and their value is still uncertain. Reviewers commented that consumer reporting systems may not be able to contribute substantially to information on the incidence, types, or causes of events. For example, errors resulting in patient harms may involve multiple health care providers, systems, and interactions among humans and technologies. Descriptions of events provided by consumers may not provide useful information to understand the activities and system features (often substantially removed from the consumer) that enabled a patient safety event to occur. As such, it may be difficult for consumer reporting systems to design meaningful improvements in patient safety (part of the system purpose included in Recommendation 2.1) if the factors precipitating patients safety events are distant from or even invisible to the experience of consumers. Reviewers also commented that vigilant health care professionals and systems will attempt to both prevent patient safety events from occurring and understand the causes of events that do occur, regardless of whether consumer reporting is involved.
However, several reasons can be provided to substantiate the importance of consumer reporting systems. First, not all patient safety events are known (or knowable) in the absence of consumer reporting; many events may not be noticed or detected if they are not reported by a consumer. This may particularly apply to patient safety events that occur in outpatient settings, where there is likely to be less surveillance than in hospitals. Also, even if an event is identified without consumer reporting, consumers are likely to be able to provide additional important information. TEP members agreed that the consumer perspective is a unique source of information for understanding the contributing factors associated with patient safety events, the response of health care providers and systems to these events, and the subsequent impacts of events on patients and their families. In many cases, information from health care professionals may not be sufficient to understand a patient safety event. As illustrated in the discussion of health care transitions above, there are instances in which consumers are likely to have more complete information regarding a patient safety event than will any individual health care provider or entity. Further, even if consumers do not provide additional useful information regarding events, providing consumers with the opportunity to report events allows them to be active participants in the pursuit of improvements in patient safety. Beyond positive impacts of this role for consumers, involvement of consumers in patient safety event reporting may increase the level of vigilance among health care providers and organizations, and may increase motivation to produce system change and, thereby, improvements in patient safety.
Issues Related to the TEP Recommendations
There are a number of limitations, implementation issues, and policy considerations related to the TEP recommendations. These are discussed below.
Because of resource constraints and attempts to minimize the already considerable time members of the TEP graciously devoted to this project, we were unable to explore all possible areas for recommended consumer reporting system design features. In addition, although the TEP consisted of a diverse group of individuals with experience in a range of relevant areas, it is not possible to include individuals with all types of appropriate backgrounds and expertise on a single panel. This limitation was addressed by providing TEP members additional information from consumer focus groups, stakeholder interviews, and an environmental scan and literature review. Information from the consumer focus groups in particular provided additional perspectives and viewpoints for TEP members in developing recommendations for key design features.
There are several areas in which the TEP chose not to develop explicit recommendations. For example, although TEP members discussed the types of organizations that would be most appropriate to operate consumer reporting systems, members elected not to recommend specific organization types, and instead specified characteristics of such organizations. Similarly, although TEP members discussed several potential financial models for consumer reporting systems, the TEP chose not to present a recommended approach to financing systems.
Throughout the course of this project, there has been an expected tension between the goal to develop design features for ideal consumer reporting systems and the knowledge of real-world barriers and limitations in the design and operation of such systems. To some extent, this has reflected an interest by TEP members in issues related to implementation of the consumer reporting systems that they were designing. Implementation of consumer reporting systems and related activities that go beyond specifying recommendations for system design features are outside of the scope of this project. However, for future work, it will be critical to consider these issues.
One implementation issue strongly emphasized by the TEP is the need for pilot testing. Determination of many design features of consumer reporting systems will depend on the scope, key activities, available resources/personnel, and other factors specific to each system; pilot testing will be needed to assess how best to implement system design features given these factors. Other system design features, such as ways to maximize consumer reporting and provide timely and meaningful feedback, are not well understood; pilot testing will be needed to collect empirical data and explore potential options to develop best practices. Pilot testing will need to assess not just whether these system design features are feasible, but whether they are useful. Pilot testing will also yield insights with regard to the number and volume of consumer reports, which will have implications for cost and feasibility of full system implementation. TEP members also emphasized the need for pilot testing in multiple local/regional settings, potentially including settings that cross geographic or political borders; a system cannot credibly or efficiently start with national implementation. It will be important for systems to have sufficient resources to allow appropriate pilot testing.
Other topics related to system implementation that were raised by TEP members include the following:
- Before a system is initiated, it will be important to develop a monitoring and evaluation program to be able to assess the system's costs and benefits and thus provide long-term justification for the system. How will the efficiency, costs, and benefits of a system be demonstrated? What sorts of outcomes are needed from a system (or a pilot test of a system) to demonstrate that it is worth the expenditures? What evidence will be needed to garner support at local, State, and national levels for the system?
- More attention will be needed regarding specifics for consumer awareness campaigns. In particular, it will be important to convey to consumers the need to report near misses/close calls and successful error recovery events. That is, why did some initial problems/errors not result in patient harm? What actions/insights by staff or consumers made this possible?
- Training will be needed for system personnel, particularly regarding making submitted data uniform. How will staff be selected? How will training be designed and accomplished?
- Decisions regarding system functions/activities that occur at the "system hub" versus those that occur locally will need to be specified.
- Consumer reporting systems will need to monitor changes in relevant laws and regulations, and to dynamically alter their functions with respect to such changes. An example of this could be passage of "no fault" laws related to patient safety events.
The TEP also recognized that certain design features are critical for consumer reporting systems to have specified at initial implementation; other features, while equally important, can be implemented at a later time. The TEP agreed that the following recommendations were critical features for consumer reporting systems to have specified at the first stage of implementation:
- Purpose and Goals (Recommendation 2.1).
- Level of Operation (Recommendation 2.2).
- Type of Organization (Recommendation 4.1).
- Anonymity (Recommendation 4.5).
- Feedback (Recommendation 6.3).
While all of the recommended design features are important for the final system, many of the other design features do not need to be present or fully specified at initial system implementation. For example, consumer reporting systems may chose to initially collect reports only for specific types of patient safety events (Recommendation 1.1) from a select group of individuals (Recommendation 1.2) using a limited number reporting modalities (Recommendation 4.3). Systems may have limited analytic capabilities (Recommendation 3.2) and few or no linkages (Recommendation 3.1) at initial implementation. Following system implementation, these design features could then develop towards the recommendations specified by the TEP.
A variety of salient policy considerations arise from the TEP recommendations. We have described a number of considerations below, categorized in three groups:
Areas of rapidly-changing technologies and practices. For several of the recommendations presented in this report, the TEP indicated that design features were important or even critical, but elected not to specify the exact nature of these design features. In many instances, this reflected the realization that technologies or accepted practices are changing rapidly; if specified too fully (or perhaps at all), design features in these areas would be obsolete by the time a system is operational. As such, the TEP recommended that full specification of such features would need to wait until system implementation. Policy considerations related to these rapidly changing areas include the following:
- Data sharing and interoperability. The TEP emphasized the importance of linkages. That is, consumer reporting systems should share information with other organizations to address patient safety events indicated in reports; provide assistance to reporting individuals that goes beyond the scope of services offered by consumer reporting systems; and enhance efforts to broadly enhance patient safety. The TEP also emphasized data sharing in comments on linking to quality and patient safety improvement efforts, stating that consumer reporting systems should not replace existing patient safety event reporting systems but should work with such systems. However, implementation of these recommendations would need to address multiple technical considerations. Organizations choosing to share data may not have compatible systems, and thus would be unable to integrate information from other sources. Interoperability is a substantial issue in health information technology today, posing a barrier to the sharing of medical information between sites of care such as hospitals, outpatient physician practices, urgent care centers, pharmacies, and skilled nursing facilities. A number of organization, including government agencies (such as the Office of the National Coordinator for Health Information Technology) and private sector groups (such as the American Medical Informatics Association), are attempting to develop standard protocols and specifications to provide greater interoperability of health care data. It will be important for members of the patient safety event reporting systems community to participate in such discussions, so that plans and decisions regarding interoperability take into account the needs and goals of reporting systems.
Confidentiality. While allowing for anonymous reporting, the TEP expressed strong support for confidential reporting, which would allow collection of additional information from and permit feedback to be provided to reporting individuals. Confidentiality of patient safety event reports, which may include protected health information (PHI), is crucial for building trust in systems and maximizing reporting. A number of statues and regulations already exist, such as the Health Insurance Portability and Accountability Act (HIPAA), which address the security and privacy of individual health-related information. However, ensuring "well-designed confidentiality" (a term used by a TEP member) involves numerous challenges. Three areas in particular in which policies are needed to maintain confidentiality are data transmission/storage; access to confidential information; and legal considerations.
- Data transmission/storage: Confidential information transmitted to a system's central operating facility or shared with other systems or organizations may be at risk of being intercepted and divulged. Similarly, information stored within a consumer reporting system may be vulnerable if adequate security procedures and safeguards are not employed. Current standards regarding data safeguards for medical data (i.e., data collected by and provided to licensed clinicians and health care facilities) may not apply to information submitted to consumer reporting systems. To ensure appropriate confidentiality of patient safety event reports and thereby encourage consumers to provide confidential rather than anonymous reports, it may be necessary to develop regulations governing the minimum permissible level of computer security for reporting systems.
- Access to confidential information: Beyond issues in transmitting and storing confidential information, a separate consideration is limiting access to this information at consumer reporting systems. Not all personnel working for or otherwise affiliated with a consumer reporting system will need access to confidential-level data. To better protect confidentiality, policies will need to be established to determine which personnel are allowed to access these data and how access is prevented for unauthorized personnel. This may involve passwords, data encryption, and secure severs (e.g., computers that can be accessed only at the physical location of a consumer reporting system, not remotely via the Internet). Tracking capabilities may also be needed, to assess which reporting system personnel accessed confidential information and when it was accessed; regular audits to identify any inappropriate access of confidential information could then be performed. In addition, policies may be needed to specify training for reporting system personnel related to data confidentiality and security issues.
- Legal considerations: Beyond the issues discussed above related to maintaining confidentiality, there are also concerns regarding possible legal actions. Private individuals or government agencies could obtain legal orders (e.g., subpoenas) requiring the release of information from consumer reporting systems despite assurances of such information's confidentiality. In addition to discouraging future confidential consumer reports of patient safety events, release of confidential information could jeopardize the future medical care for involved individuals and potentially even result in legal challenges (e.g., defamation suits) with associated economic damages. For whistleblowers (individuals who submit reports on patient safety events at health care facilities where they work), the release of confidential information could result in loss of employment and substantial damage to future career options. Patient Safety Organizations (PSOs) have been granted protections related to such legal actions. Similar policies, government regulations, or laws may be needed to clarify the nature of information submitted to patient safety event reporting systems, and to protect the confidentiality of such information.
- Public reporting. The TEP strongly supported the concept of public reporting, and specified several purposes for public reporting, but electing not to specify the format or design of such reporting due to ongoing rapid changes in this area. However, a related issue is the response of regulatory agencies to such information. Although there are numerous caveats regarding the use of publicly reported patient safety event information (due in large part to incomplete information on both the numerators and denominators related to such events), there is an inherent desire to compare rates of patient safety events across different facilities or clinicians. There would likely be concerns (justified or not) that facilities or clinicians associated with higher rates of patient safety events are potentially negligent in some manner. Should local, State, or Federal government regulatory agencies or private sector accreditation organizations respond to such information? Should these agencies/organizations be required to respond? As the debate on the use of publicly reported patient safety information evolves, policies regarding the rights of facilities and clinicians identified in such information versus the obligations of regulatory agencies and accreditation bodies will need to be developed.
Coordination and collaboration among organizations/agencies involved in patient safety. Other policy considering resulting from the TEP recommendations are related to coordination and collaboration among different organizations involved in collecting information on and attempting to prevent or address patient safety events. Policy considerations in this area include the following:
- Coordination among government agencies and consumer reporting systems. A particularly salient policy consideration relates to integrating or coordinating the work of consumer reporting systems with existing systems and activities of AHRQ and other Federal agencies. There may be issues with other Federal agencies that currently have consumer reporting systems or may consider such a system to be within their mission—for example, the Food and Drug Administration and the Centers for Disease Control and Prevention. In addition, many States have their own reporting systems. There is a need to be aware of whether such a system would be considered to impinge on the "turf" of other government agencies. Further, the continued development of consumer reporting systems may suggest potential policy changes for government agencies. For example, a TEP member suggested that the Medicare Conditions of Participation (for health care facilities) require institutions to have a formal process for the receipt of information from consumer reporting systems and incorporation of such reports into their quality and patient safety improvement efforts. AHRQ may find it useful to convene a policy summit bringing together representatives from State and Federal agencies with overlapping interests to clarify the boundaries of consumer reporting systems and address the issues raised above. An external reviewer also commented that the framework of recommended design features described in this report will be useful only if AHRQ and other agencies link these recommendations to specific, funded activities where they are implemented and tested.
- Information sharing with health care facility-based reporting systems. The TEP emphasized that consumer reporting systems should not replace other reporting systems, but should work with these other systems. In particular, the importance of sharing information with reporting systems based at health care facilities (as permitted by confidentiality restrictions specified by individuals submitting reports) was highlighted, to encourage learning by health care professionals and increase opportunities to improve patient safety. However, this type of collaboration cannot occur unless facility-based reporting systems are in operation and are specifically tasked with accepting reports from consumer systems. Although many (perhaps most) health care facilities have existing procedures for submitting information on patient safety events, requirements for such reporting systems vary widely by State and by type of facility (e.g., hospitals, urgent care centers, skilled nursing facilities, outpatient physician practices). Furthermore, there are no uniform requirements regarding whether facility-based systems accept reports from outside systems or respond to such reports. To encourage collaborations among reporting systems for addressing patient safety issues, policies are needed to facilitate development of facility-based systems that can review and act on information from consumer reporting systems.
The TEP also recommended that consumer reporting systems request information from health care facilities on Root Cause Analyses (RCAs) when such analyses have been performed for submitted events. However, there are no uniform requirements for health care facilities to share RCAs or other analyses when performed; in some jurisdictions, there may be barriers to sharing such information even when requested by the involved patients. Policies are needed to remove these barriers and encourage or require sharing of analyses and other relevant information from health care facilities with consumer reporting systems. Furthermore, as few individuals in hospitals or other health care environments have been trained to perform RCAs, these analyses may have limited usefulness.
- Ownership/operation of consumer reporting systems. The TEP chose not to specify the specific type or types of organizations to own and operate consumer reporting systems. Rather, the TEP provided recommendations as to the characteristics of such organizations. These characteristics include independence; fiduciary responsibility to the public; neutrality; and transparency. Parallel to the process created for the development and operation of PSOs, State or national legislation may be needed to specify the requirements, operations, and protections for consumer reporting systems. Legislation could specify who (individuals or organizations) are allowed to operate consumer reporting systems (to maintain independence); the required level of oversight by neutral bodies with consumer representation; mandatory reports by such systems to provide transparency regarding organizational priorities, actions, and finances; and protections for information submitted to consumer reporting systems (potentially addressing legal consideration and whistleblower issues mentioned by the TEP). Legislation could also address funding of consumer reporting systems, potentially using the models highlighted by the TEP (e.g., the Pennsylvania State reporting system). It will also be important to distinguish the roles of PSOs and consumer reporting systems, although this may or may not be a component of legislation.
The idea of consumer reporting systems for patient safety events holds great promise. Turning that idea into a reality will require significant political will, policy coordination, and resource investment. The design features presented in this report provide an actionable foundation for the necessary next steps of implementation and operation.