Peter J. Pronovost, Jill A. Marsteller, Albert W. Wu, Christine G. Holzmueller, David A. Thompson, Lisa H. Lubomski, Lori A. Paine, Deborah B. Hobson, Melinda D. Sawyer, Rhonda M. Wyskiel, Hanan Aboumatar, Dale M. Needham, Christine A. Goeschel, Bradford D. Winters, Julius C. Pham, Adam Sapirstein, Mark Romig, Pedro A. Mendez-Tellez, Ayse P. Gurses, Michael A. Rosen, Sallie J. Weaver, J. Matthew Austin, Asad Latif, Sean M. Berenholtz
Abstract
This paper describes our journey to advance the science and practice of patient safety and quality improvement. The journey began with efforts to identify hazards through an incident reporting system called the Intensive Care Unit Safety Reporting System. We quickly found that identifying hazards was merely a first step. We also needed to investigate and learn from these hazards to prevent patient harm. Therefore, we developed the Comprehensive Unit-based Safety Program (CUSP) to identify and learn from local defects and improve teamwork and safety culture. Teams across many units faced common problems, such as healthcare-acquired infections, for which there is empiric evidence on prevention practices, but the evidence is unreliably applied. This discovery led us to develop a model to translate research into practice (TRIP). We combined TRIP with CUSP for the Keystone ICU Project design, with the goal of improving care for adult patients in Michigan intensive care units (ICUs). The resulting dramatic and sustained reductions in central line-associated bloodstream infections (CLABSIs) in Michigan led to a national initiative to reduce CLABSIs across the United States. Applying the perspectives from different academic disciplines helped us learn how this national effort succeeded, an approach we also used to study cardiac surgery-related errors. Still, the CLABSI effort addressed one type of harm, while patients are at risk for over a dozen and care systems relying more on the heroism of clinicians than on safe design. Current efforts include building a quality management infrastructure to support improvement work and defining the skills, resources, and accountability needed at every level of a health system. We are also partnering with patients, their loved ones, and others to eliminate all harms, optimize patient experience and outcomes, and reduce waste. In this trans-disciplinary systems approach, we hope to reduce all harms, improve productivity, and enhance joy for clinicians.
Introduction
Before the renowned To Err is Human report,1 there were isolated studies digging into the problem of medical mistakes.2 The Institute of Medicine (IOM) report brought urgency and attention to the problem and advanced the theory that system flaws were more to blame than the failures of caregivers.1 The IOM and other prominent agencies looked to clinicians and researchers for answers, but the science was barely a bud on a branch. The 2013 report, Making Health Care Safer II, from the Agency for Healthcare Research and Quality (AHRQ), shows how much the science has matured and how much we still have to learn.3 This paper reflects on our research group's journey to advance the science of patient safety and quality improvement. We approached the work organically. The work evolved and often germinated new ideas, projects, or interventions. We grafted experiences and models from other disciplines onto our work, resulting in stronger inferences and robust interventions. All of these contributed to improving the quality of care and patient outcomes.
Reflecting back, we helped advance the science by identifying hazards, establishing a culture of safety, reducing a preventable harm, and moving from one type of harm to all harms (Table 1). Throughout our efforts, we had profound respect for the wisdom of health care workers, especially caregivers; sought to integrate researchers with operational safety practitioners; and used the frontlines of clinical care as our laboratory to harness the wisdom of clinicians, test tools and interventions, measure performance and evaluate success, and acquire new knowledge. We assembled interdisciplinary teams and used transdisciplinary research (team science), in which different disciplines work on common problems through a common conceptual model, to bring a comprehensive perspective to our work.4
Table 1. Journey advancing the science of patient safety and quality improvement
| Year Started | Description and Funding Source |
|---|---|
| 1999 |
|
| 2001 |
|
| 2002 |
|
| 2003 |
|
| 2007 |
|
| 2008 |
|
| 2010 |
|
| 2011 |
|
| 2012 |
|
Note: a fractal comprises smaller parts that are similar or identical to the whole object and connected to support and form the whole object.
Identifying Hazards
In 2001, researchers from the Johns Hopkins University, Quality & Safety Research Group, were awarded a grant from AHRQ to build and use an incident reporting system to identify hazards in a cohort of intensive care units (ICUs). This demonstration project was called the Intensive Care Unit Safety Reporting System (ICUSRS).5,6 First, we had to develop a system that staff would use to report incidents. We researched existing reporting systems, including the Australian Incident Monitoring Study,7 to determine how they encouraged self-reporting and what data they found useful. We learned that the aviation safety reporting system (ASRS)8 was successful because it collected adverse event and near-miss data and used the data to fix systems rather than blame individuals. We included these elements in the ICUSRS. Our project identified factors contributing to incidents and established a network of units to share de-identified data, learn from reports and experiences, and improve care.9–12
The ICUSRS was Web-based and accessible from any computer for convenience and privacy (home or work). It was anonymous and confidential (to protect users from perceived or actual punitive actions or liability), voluntary, and integrated with existing reporting systems to avoid duplicate reporting.13 We collected quantitative and qualitative data to obtain a comprehensive account of the incident. The system went live in July 2002 in three ICUs at the Johns Hopkins Hospital (JHH) and ramped up to 23 ICUs across the United States. We partnered with the Society for Critical Care Medicine, which identified and recruited a geographically diverse group of hospitals and ICUs (adult and pediatric, including medical, surgical, cardiac, and trauma services). Each participating unit assembled an interdisciplinary team to encourage reporting, disseminate feedback, and manage improvements.
The information gleaned from this project benefited the participating ICUs and provided our agenda for further research and intervention. The aggregate data helped sites recognize hazards and change care processes. For example, one site found a large cluster of medication errors when a pharmacist was not present on the unit. The data helped us recognize common hazards, such as inadequate training/education and teamwork.9–11,14
As we worked with the participating sites, we discovered that local culture played a major role in whether or not staff reported incidents and/or teams used their data to improve safety. We also found that simply identifying hazards was not enough. We all needed to learn from them to mitigate the risks that would otherwise harm future patients. These insights led to the creation of the Comprehensive Unit-based Safety Program (CUSP).
Establishing a Culture of Safety
Our work in ICUs helped us realize that culture is an integral part of safety and quality improvement. At the same time, a Safety Committee was created at JHH in response to the IOM report.1 The committee created CUSP to build a culture of safety throughout the hospital.15–20 CUSP was based on available evidence and expert advice and evolved through trial and adaptation.21 The committee studied commercial aviation's success with improving and managing errors and high reliability organizations and found industries where employees shared a common commitment to safety. When we started at JHH, frontline clinicians were unsure about the hospital's commitment to safety, and hospital leaders were far removed from the frontlines of care. Recognizing that safety programs must empower frontline staff and provide them with resources to identify, and mitigate risks and that performance and culture vary widely among units, the safety committee designed CUSP as a unit-level intervention to reach all staff and connect them to hospital management.
The original CUSP measured safety culture (pre- and post-intervention), educated staff on the science of safety, partnered units with a hospital executive,18 and had staff identify safety concerns, implement improvement efforts and document results, and share their learning across the organization.17 The program was piloted in two ICUs in 2001 as an eight-step program. It continued to evolve as it was implemented and spread to other clinical areas at JHH,21 and it now comprises five steps.22 CUSP built a safety infrastructure within units, changed unit culture, and became the platform on which to organize and implement many safety or quality improvement efforts. For example, we were accumulating reports of errors and near misses through the ICUSRS project without a formal process for staff to prioritize their greatest safety concerns and mitigate these risks. To rectify this, we developed a feasible yet scientifically sound root cause analysis tool, called "learning from defects," that staff could use to immediately investigate a defect,23 and we made this a step in CUSP.
Our defect investigations and sentinel event reports from The Joint Commission repeatedly noted poor communication and teamwork as major contributing factors to all types of adverse events. Teamwork and communication errors quickly became a catch-all term, comprising many types of errors. Thus, we reviewed the literature and looked at specific ways to improve communication and teamwork. We already had a checklist to coordinate and effectively communicate a patient's daily goals during rounds. This checklist was developed in 2001, when an ICU attending physician (PJP) realized that only 10 percent of the nurses or physicians in an adult surgical ICU at JHH understood the goals for their patients at the end of rounds. The checklist was pilot tested in the same ICU, increasing nurse and physician comprehension of daily patient goals to 95 percent.15 The daily goals checklist became a repository for some safety practices. For example, one item reminds clinicians to remove unnecessary central lines, which is part of the Centers for Disease Control and Prevention (CDC) guidelines for managing existing lines. The checklist was a great success and was modified to fit other clinical areas at JHH,24,25 and it is now a staple of interdisciplinary rounds and team communication in ICUs across the United States and much of the world.
To help clinicians address their specific teamwork concerns, we gradually assembled a menu of tools from which a CUSP can choose to improve teamwork, communication, or culture, and we made this a program step.15,16,19,20,24–27 With all of these tools, we identified the need; searched the evidence to see what, if any, interventions were successful; developed tools; and pilot tested them for content validity and feasibility.
Reducing One Preventable Harm: Local, National, Global
While CUSP helps units improve their safety culture and learn from local mistakes, we recognized that a different method was needed to mitigate common causes of harm, harms with empiric evidence-based practices that were common and standardized enough to be measured as rates. To reduce these types of harms (e.g., bloodstream infection, ventilator-associated pneumonia, surgical site infection [SSI], deep venous thrombosis, and decubitus ulcer), we developed a model for translating research into practice (TRIP).28 The model was designed for collaborative groups to improve performance over a large cohort of units. Some work is most efficiently conducted by a coordinated scientific body. For instance, it would be ineffective and inefficient for individual units to review evidence on their own and develop performance measures. The TRIP model includes summarizing evidence-based practices into simple, unambiguous checklists; identifying barriers to implementing those practices;29 measuring performance (processes and outcomes, if possible); ensuring that all patients reliably receive the checklist items; and encouraging units to locally modify how they implement the checklists.
Our efforts to prevent one type of patient harm—central line-associated bloodstream infection (CLABSI)—began in 1999 at JHH. A convergence of the CDC-sponsored clinical guidelines, JHH epidemiology and infection control efforts, and awareness of high infection rates from central lines—lines we inserted daily—inspired us to reduce these rates. The guidelines recommended effective and feasible clinical practices, yet these practices were not routinely reaching patients. We sought to bridge this gap.
The project to prevent CLABSIs occurred over a 3-year period.30 It included a mandatory training module (which taught clinicians the infection prevention practices), a central line cart, and a line insertion checklist. Five of the checklist practices related to catheter insertion (wash hands, clean patient skin with chlorhexidine, use full barrier precautions during insertion, avoid femoral vein site, maintain sterile field during insertion), and one described catheter maintenance.
Despite knowledge and a desire to comply with the checklist, physicians faced a major barrier—lack of necessary supplies. We followed the principles of safe design (part of the CUSP training in the science of safety) and created the line cart because clinicians in our surgical ICU had to gather catheter insertion supplies from eight places to comply with the practices. The cart increased checklist compliance rates from 30 percent to 75 percent, but we were still far from 100 percent. Infection rates were cut in half. Therefore, we compiled a checklist of these best practices, which standardized the insertion process and offered an independent check by the bedside nurse to ensure that physicians complied with these practices. When nurses piloted the checklist, physicians resisted being questioned about their practices and often ignored nurses (a warning that to improve safety we must also improve culture). Therefore, we educated all staff about the consequences of suboptimal practices. We empowered nurses to stop non-emergent procedures if a practice was ignored and instructed them to page the ICU attending physician if a resident was noncompliant. With the nursing intervention, compliance increased from 75 percent to 95 percent, and infection rates fell further. Finally, ICU and infection prevention staff investigated every infection as a defect, using a modified version of the original learning from defects tool,23 to identify ways to prevent the infection. The success of the overall intervention shifted health care from the philosophy that bloodstream infections were inevitable, to the realization that most infections are preventable. Moreover, it identified opportunities to further reduce infections by taking the checklist to the operating room and developing a checklist to improve catheter maintenance practices.
Through the CLABSI project,31 we established an improvement model that seemed to work and made sense and empowered clinicians: identify an outcome to improve, summarize the evidence to identify interventions that improve the outcome, query frontline staff to identify barriers to complying with the interventions, standardize the process to reliably implement the interventions (e.g., checklist), educate staff about the evidence, have staff implement the interventions in the most seamless way possible in their work area, and evaluate whether interventions were used and improved the outcome. We continue to use this model today to translate evidence into practice.28
In 2002, when AHRQ announced a second Request for Proposals (RFP) to improve patient safety, the director of the Keystone Center for Patient Safety and Quality, part of the Michigan Health & Hospital Association (MHA), approached us about partnering on a grant to improve care in Michigan ICUs. Historically, many quality improvement projects had poor data quality, often lacking standardized definitions and data collection tools, and usually missing volumes of data that exceeded available data, thus limiting the ability to draw inferences.32,33
We designed a cohort collaborative in which the Hopkins group provided the technical science (interventions, evidence, data collection tools, and analysis), and the Keystone Center provided project management, recruitment of hospitals, and interaction with improvement teams.34 To support the large number of participating hospitals and ICUs, we used a theory of change that was practical, yet based on the diffusion of innovation and behavior change.35,36 Our implementation model included four E's (engage, educate, execute, evaluation) and targeted the three levels of a hospital required to improve care (executive leaders, unit team leaders, and frontline staff).28
The project included interventions with evidence supporting their use, including CUSP, daily goals and interdisciplinary rounds, ICU physician staffing,37,38 our checklist of infection control prevention practices for CLABSI, and another checklist we developed to prevent ventilator-associated pneumonia. CUSP was implemented first, and in parallel with the four E's model,28 to prepare staff to implement the other interventions. Over 100 ICUs participated in the AHRQ-funded Keystone ICU project from September 2003 to September 2005. The design was a prospective cohort study using a multiple time-series analysis. Our original intent was to conduct a cluster-randomized design in which hospitals would be randomized to receive the intervention early versus late, but few hospitals wanted to be randomized to the control group. Most hospitals believed the intervention would be effective and wanted to implement it. In response, we changed our design to a multiple time series and achieved significant reductions in CLABSIs, though the design lacked a concurrent control group.39,40
We established an association between the CUSP/CLABSI interventions and reduced CLABSIs in the Keystone project. A next step, scientifically, was to test this association in a randomized clinical trial to evaluate the effectiveness of the interventions. With a Robert Wood Johnson Foundation grant, we conducted a phased, cluster-randomized trial in 45 hospitals from two Adventist health systems, from 2007 to 2008.41 In the Adventist collaborative, we added two E's to our organizational change model. Expand encouraged teams to spread the program to other units, and endure reminded teams to make the intervention a part of routine practice. We also placed greater emphasis on catheter maintenance. The Adventist hospitals decreased their CLABSI rate by 81 percent, an even greater impact than the 66 percent rate reduction achieved in the Keystone project.39 Importantly, the Adventist collaborative established a causal relationship between the intervention and reduced CLABSIs.
Next, our team recognized that to gain support to spread this intervention broadly, we needed to demonstrate not only that it prevents infections, but that it can be sustained, save lives and money, and be disseminated to other States and types of harm. Sustainability demonstrates whether a successful intervention can become routine practice and engrained in the culture. In a followup analysis, low CLABSI rates were sustained.39 Moreover, the program improved the safety climate,42 sustained reductions in VAP,43 decreased mortality rates,44 and averted $1.1 million in annual costs per hospital.45 Despite comparable mortality rates, by the end of the study the mortality rate of a Medicare patient admitted to any Michigan ICU was 10 percent lower than for similar patients in the 11 surrounding States.44 The intervention was disseminated to Rhode Island, with similar reductions in CLABSI.46 The return on AHRQ's $500,000 yearly investment in this 2-year project was unprecedented and led the agency to award additional funding to spread the collaborative project to other States across the United States.
With support from AHRQ, the Price Family Foundation, and the Sandler Foundation for the Jewish Community Endowment Fund, in 2008 we partnered with the American Hospital Association and its research arm, the Health Research Education Trust (HRET), and with the Michigan Keystone Center (MHA), State hospital associations, and many other organizations to implement the collaborative in every State, the District of Columbia, and Puerto Rico.47
Hawaii and Connecticut soon replicated the Michigan results.48-50 Overall, 1,100 hospitals and 1,500 ICUs reduced their infection rates by 41 percent. Moreover, these hospitals achieved a mean rate of one infection per 1,000 catheter-days, a rate deemed impossible before the Michigan work. In Hawaii, the interventions were spread to non-ICUs, and the State leader for the project, from the Healthcare Association in Hawaii, worked with sites to develop tools to reduce rates even further.48 Also, beginning in 2008 with a partnership with the World Health Organization, the intervention termed "Matching Michigan" was spread to nearly all hospitals in England and Spain and to a sample of hospitals in Peru, Pakistan, and the United Arab Emirates (UAE). The intervention was associated with significant reductions in CLABSIs in Spain, Peru, and Pakistan. The UAE effort is ongoing.
While linking teams in clinical communities51 explains a large part of the success in reducing CLABSIs, with AHRQ funding we are applying CUSP methods to reduce ventilator-associated complications and surgical complications, other preventable causes of mortality. We realized we needed to link care teams across different service lines within the hospital to improve safety during patient handoffs and remove silos of care. In most hospitals, CUSP teams have had limited interactions with other care areas. Thus, with AHRQ funding, we sought to build on our work of identifying hazards in cardiac surgery27,52–55 and our work with CUSP/CLABSI by linking CUSP teams within a cardiac surgery product line. We are working with 15 hospitals to create cardiac surgery operating room, ICU, and floor CUSP teams. These teams will link with similar teams at other hospitals and with CUSP teams in different care areas within their own hospital. Although our analyses are still underway, these teams have demonstrated significant reductions in CLABSI, VAP, and SSI and increased teamwork among units.
Beyond One Harm to All Harms
The national CLABSI program provided lessons regarding what it takes to eliminate additional harms: ensure safety through the safe design of systems rather than the heroism of clinicians, build a fractal quality management infrastructure, influence peer norms through peer-to-peer review, and create more valid outcome measures. Despite these successes, the CLABSI work addressed one type of harm, while individual patients are at risk for over a dozen. Harms extend beyond physical harm to harm from disrespectful and undignified care.56 Yet most hospitals work on only one or two harms, largely because the methods to reduce harm are too burdensome. With generous support from the Gordon and Betty Moore Foundation, we are developing a systems engineering approach with a goal to eliminate all recognized harms (EMERGE Project). For example, ICU patients are at risk for CLABSI, ventilator-associated complications, deep venous thrombosis and pulmonary embolus, decubitus ulcers, delirium and physical deconditioning, and disrespectful care that does not meet their needs, among other harms. There is a checklist to prevent each of these harms, and when added up, there are nearly 200 interventions required each day to prevent all the harms. Yet there is no information technology (IT) tool that lists these harms and therapies. In the EMERGE project, we will work to create an integrated IT platform to ensure that patients receive all 200 interventions. This approach also applies to outpatients who are similarly at risk for multiple harms.
In addition to improving safety, this approach could significantly improve productivity and reduce health care costs. Health care is the only industry that invested heavily in IT but nearly flatlined in productivity. Today's ICU is likely less safe and productive than it was 30 years ago. It is packed with more devices and alarms, and none communicate. False alarm rates are extremely high in critical care,57 ranging from 85 percent to 99 percent in one study, and alarm fatigue is a documented problem.58 In this project, 18 different disciplines are using team science4 and working with patients, their families, and the private sector to eliminate preventable harm, optimize patient outcomes and experience, and reduce waste.
We also recognized that most devices are designed with little input from clinicians, creating human-machine interface problems and harming patients. With AHRQ funding, we partnered with human factors and systems engineers to evaluate the usability of a ubiquitous medical device, medication infusion pumps. In this study, we determined what users needed to safely operate infusion pumps; the next step is to encourage manufacturers to use our findingsa to design safer pumps. The opportunities in systems engineering and medical device redesign are enormous.
In addition to systems engineering, we learned that an infrastructure of defined skills, resources, and accountabilities was needed at every level of the health care system to manage and support the improvement work. In addition, each higher level should regularly meet with staff from each lower level to support peer learning and accountability. Improvement of patient safety and quality requires this fractal infrastructure. This quality management infrastructure is grossly underdeveloped in health care; without it, progress will remain slow and arduous.
Informed by the work of the World Association of Nuclear Operators, we are developing tools for peer-to-peer review, in which one provider organization evaluates another, focusing on an outcome (CLABSI), a geographic area (ICU or operating room), or an overall program (quality and safety).59 Lacking regulatory authority, these reviews focus on learning rather than judging, using validated tools and clinicians, being confidential yet probing.
Finally, health care has too few valid outcome measures, limiting the ability to develop programs to reduce other harms besides CLABSI. No U.S. entity is charged with developing measures, reporting performance data, or housing the data. A health care entity similar to the Securities and Exchange Commission would offer hope that one day, the quality committee of a health system's board could function like an audit committee, with clear goals and valid measures, with skilled staff at all levels of the organization monitoring performance, and with clear accountability and performance monitoring.
a. Unpublished data, Julius C. Pham, January 2, 2014.
Reflection
In reflecting on our journey thus far, we have learned several things. First, funding was essential to advance the science. Prominent organizations such as the Agency for Healthcare Research and Quality, the Robert Wood Johnson Foundation, the World Health Organization, the Price Family Foundation, and the Sandler Foundation for the Jewish Community Endowment Fund believed in the work and supported our efforts. Medical error is the third leading cause of death, yet Government research funding remains disproportionate to the magnitude of the problem. Second, professional norms are the roots. Clinicians must drive the work and be linked to clinical communities through intrinsic motivation and be energized by peer-to-peer learning to implement the work. Third, all disciplines must be involved to offer a comprehensive perspective on the problems and the solutions. Fourth, science must guide us, and the measures used must be valid. Fifth, we need skilled staff, resources, and accountability at every level to connect the work vertically in the organization and horizontally to clinical communities, clinicians, and other groups. Sixth, health care needs more measures of patient outcomes and costs and an organization similar to the Securities and Exchange Commission to coordinate measure development.60
Acknowledgments
Information presented in this paper was drawn from our work on the following projects: Intensive Care Unit Safety Reporting System (ICUSRS), grant HS11902 from the Agency for Healthcare Research and Quality (AHRQ); Michigan Keystone ICU Project, AHRQ grant HS14246; National On the CUSP: Stop BSI, AHRQ contract HHSA290200600021l; Cardiac Surgery CUSP Program, AHRQ grant HS29934; Medication Infusion Pump Usability Study, AHRQ grant HS20460; Adventist Health System, Robert Wood Johnson, project 58292; and EMERGE, Gordon & Betty Moore, project 3186.01. The findings and conclusions in this document are those of the authors, who are responsible for its content, and do not necessarily represent the views of AHRQ. No statement in this report should be construed as an official position of AHRQ or the U.S. Department of Health and Human Services.
Authors' Affiliations
Johns Hopkins Medicine Armstrong Institute for Patient Safety & Quality (PJP, JAM, AWW, CGH, DAT, LHL, LAP, DBH, MDS, RMW, HA, DMN, CAG, BDW, JCP, AS, APG, MAR, SJW, JMA, AL, SMB), School of Medicine, (PJP, AWW, DAT, DMN, CAG, BDW, JCP, AS, MR, PAM, APG, MAR, SJW, JMA, AL, SMB), Bloomberg School of Public Health (PJP, JAM, AWW, DAT, CAG, APG, SMB), and School of Nursing (PJP, DAT, CAG), The Johns Hopkins University, Baltimore, MD. The Johns Hopkins Hospital, Baltimore, MD (LAP, DBH, MDS, RMW).
Address correspondence to: Peter J. Pronovost, Armstrong Institute for Patient Safety and Quality, Johns Hopkins Medicine, 750 E. Pratt Street, 15th floor, Baltimore, MD 21202; Email ppronovo@jhmi.edu.
References
1. Kohn L, Corrigan J, Donaldson M. To Err is Human: Building a Safer Health System. Washington: National Academies Press, 1999.
2. Wu AW, Folkman S, McPhee SJ, Lo B. Do house officers learn from their mistakes? JAMA 1991 Apr 24;265(16):2089-94. PMID: 2013929.
3. Making Health Care Safer II: An updated critical analysis of the evidence for patient safety practices. www.ahrq.gov/research/findings/evidence-based-reports/ptsafetyuptp.html. Accessed December 16, 2013.
4. Adler NE, Stewart J. Using team science to address health disparities: MacArthur network as case example. Ann N Y Acad Sci 2010 Feb;1186:252-60. PMID: 20201877.
5. Holzmueller CG, Pronovost PJ, Dickman F, et al. Creating the web-based intensive care unit safety reporting system. J Am Med Inform Assoc 2005 Mar-Apr;12(2):130-9. PMID: 15561794.
6. Wu AW, Holzmueller CG, Lubomski L, et al. Development of the ICU safety reporting system. J Patient Safety 2005 Mar;1(1):23-32.
7. Beckmann U, West LF, Groombridge GJ, et al. The Australian Incident Monitoring Study in Intensive Care: AIMS-ICU. The development and evaluation of an incident reporting system in intensive care. Anaesth Intensive Care 1996 Jun;24(3):314-9. PMID: 8805885.
8. Aviation Safety Reporting System. Program briefing. http://asrs.arc.nasa.gov/overview/summary.html. Accessed March 20, 2013.
9. Pronovost PJ, Thompson DA, Holzmueller CG, et al. Toward learning from patient safety reporting systems. J Crit Care 2006 Dec;21(4):305-15. PMID: 17175416.
10. Sinopoli DJ, Needham DM, Thompson DA, et al. Intensive care unit safety incidents for medical versus surgical patients: a prospective multicenter study. J Crit Care 2007 Sep;22(3):177-83. PMID: 17869966.
11. Needham DM, Sinopoli DJ, Thompson DA, et al. A system factors analysis of "line, tube, and drain" incidents in the intensive care unit. Crit Care Med 2005 Aug;33(8):1701-7. PMID: 16096444.
12. Lubomski L, Pronovost PJ, Thompson DA, et al. Building a better incident reporting system: perspectives from a multisite project. J Clin Outcomes Mgmt 2004 May;11(5):275-80.
13. Thompson DA, Lubomski L, Holzmueller CG, et al. Integrating the intensive care unit safety reporting system with existing incident reporting systems. Jt Comm J Qual Patient Saf 2005 Oct;31(10):585-93. PMID: 16294671.
14. Needham DM, Thompson DA, Holzmueller CG, et al. A system factors analysis of airway events from the Intensive Care Unit Safety Reporting System (ICUSRS). Crit Care Med 2004 Nov;32(11):2227-33. PMID: 15640634.
15. Pronovost P, Berenholtz S, Dorman T, et al. Improving communication in the ICU using daily goals. J Crit Care 2003 Jun;18(2):71-5. PMID: 12800116.
16. Thompson DA, Holzmueller CG, Hunt, D, et al. A morning briefing: Setting the stage for a clinically and operationally good day. Jt Comm J Qual Patient Saf 2005 Aug;31(8):476-9. PMID: 16156196.
17. Pronovost P, Weast B, Rosenstein B, et al. Implementing and validating a comprehensive unit-based safety program. J Pat Safety 2005 Mar;1(1):33-40.
18. Pronovost PJ, Weast B, Bishop K, et al. Senior executive adopt-a-work unit: a model for safety improvement. Jt Comm J Qual Saf 2004 Feb;30(2):59-68. PMID: 14986336.
19. Makary MA, Holzmueller CG, Thompson DA, et al. Operating room briefings: working on the same page. Jt Comm J Qual Patient Saf 2006 Jul;32(6):351-5. PMID: 16776390.
20. Makary MA, Holzmueller CG, Sexton JB, et al. Operating room debriefings. Jt Comm J Qual Patient Saf 2006 Jul;32(7):407-10. PMID: 16884128.
21. Timmel J, Kent PS, Holzmueller CG, et al. Impact of the Comprehensive Unit-based Safety Program (CUSP) on safety culture in a surgical inpatient unit. Jt Comm J Qual Patient Saf 2010 Jun;36(6):252-60. PMID: 20564886.
22. Paine LA, Rosenstein BJ, Sexton JB, et al. Assessing and improving safety culture throughout an academic medical centre: A prospective cohort study. Qual Saf Health Care 2010 Dec;19(6):547-54. PMID: 21127113.
23. Pronovost PJ, Holzmueller CG, Martinez E, et al. A practical tool to learn from defects in patient care. Jt Comm J Qual Patient Saf 2006 Feb;32(2):102-8. PMID: 16568924.
24. Schwartz JM, Nelson KL, Saliski M, et al. The daily goals communication sheet: a simple and novel tool for improved communication and care. Jt Comm J Qual Patient Saf 2008 Oct;34(10):608-13. PMID: 18947121.
25. Holzmueller CG, Timmel J, Kent PS, et al. Implementing a team-based daily goals sheet in a non-ICU setting. Jt Comm J Qual Patient Saf 2009 Jul;35(7):384-8. PMID: 19634807.
26. Sexton JB, Paine LA, Manfuso J, et al. A check-up for safety culture in "my patient care area". Jt Comm J Qual Patient Saf 2007 Nov;33(11):699-703. PMID: 18074719.
27. Thompson DA, Holzmueller CG, Lubomski L, et al. View the world through a different lens: shadowing another provider. Jt Comm J Qual Patient Saf 2008 Oct;34(10):614-8. PMID: 18947122.
28. Pronovost PJ, Berenholtz SM, Needham DM. Translating evidence into practice: a model for large scale knowledge translation. BMJ 2008 Oct;337:963-5. PMID: 18838424.
29. Gurses AP, Murphy DJ, Martinez EA, et al. A practical tool to identify and eliminate barriers to compliance with evidence-based guidelines. Jt Comm J Qual Patient Saf 2009 Oct;35(10):526-32. PMID: 19886092.
30. Berenholtz SM, Pronovost PJ, Lipsett PA, et al. Eliminating catheter-related bloodstream infections in the intensive care unit. Crit Care Med 2004 Oct;32(10):2014-20. PMID: 15483409.
31. Berenholtz SM, Milanovich S, Faircloth A, et al. Improving care for the ventilated patient. Jt Comm J Qual Saf 2004 Apr;30(4):195-204. PMID: 15085785.
32. Berenholtz SM, Needham DM, Lubomski LH, et al. Improving the quality of quality improvement projects. Jt Comm J Qual Patient Saf 2010;36(10):468-73. PMID: 21548508.
33. Needham DM, Sinopoli DJ, Dinglas VD, et al. Improving data quality control in quality improvement projects. Int J Qual Health Care 2009 Apr;21(2):145-50. PMID: 19218334.
34. Pronovost PJ, Berenholtz SM, Goeschel C, et al. Improving patient safety in intensive care units in Michigan. J Crit Care 2008 Jun;23(2):207-21. PMID: 18538214.
35. Greenhalgh T, Robert G, Macfarlane F, et al. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q 2004;82(4):581-629. PMID: 15595944.
36. Michie S, Johnston M, Abraham C, et al. Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care 2005 Feb;14(1):26-33. PMID: 15692000.
37. Pronovost PJ, Needham DM, Waters H, et al. Intensive care unit physician staffing: financial modeling of the Leapfrog standard. Crit Care Med 2006 Mar;34(3 Suppl):S18-24. PMID: 16477199.
38. Pronovost PJ, Angus DC, Dorman T, et al. Physician staffing patterns and clinical outcomes in critically ill patients: a systematic review. JAMA 2002 Nov 6;288(17):2151-62. PMID: 12413375.
39. Pronovost PJ, Goeschel CA, Colantuoni E, et al. Sustaining reductions in catheter related bloodstream infections in Michigan intensive care units: observational study. BMJ 2010 Feb 4;340:c309. PMID: 20133365.
40. Pronovost P, Needham D, Berenholtz S, et al. An intervention to decrease catheter-related bloodstream infections in the ICU. N Engl J Med 2006 Dec 28;355(26):2725-32. PMID: 17192537.
41. Marsteller JA, Sexton JB, Hsu YJ, et al. A multicenter, phased, cluster-randomized controlled trial to reduce central line-associated bloodstream infections in intensive care units. Crit Care Med 2012 Nov;40:2933-9. PMID: 22890251.
42. Sexton JB, Berenholtz SM, Goeschel CA, et al. Assessing and improving safety climate in a large cohort of intensive care units. Crit Care Med 2011 May;39(5):934-9. PMID: 21297460.
43. Berenholtz SM, Pham JC, Thompson DA, et al. Collaborative cohort study of an intervention to reduce ventilator-associated pneumonia in the intensive care unit. Infect Control Hosp Epidemiol 2011 Apr;32(4):305-14. PMID: 21460481.
44. Lipitz-Snyderman A, Steinwachs D, Needham DM, et al. Impact of a statewide intensive care unit quality improvement initiative on hospital mortality and length of stay: retrospective comparative analysis. BMJ 2011 Jan 28;342:d219. PMID: 21282262.
45. Waters HR, Korn R, Jr, Colantuoni E, et al. The business case for quality: economic analysis of the Michigan Keystone Patient Safety Program in ICUs. Am J Med Qual 2011 Sep-Oct;26(5):333-9. PMID: 21856956.
46. DePalo VA, McNicoll L, Cornell M, et al. The Rhode Island ICU collaborative: a model for reducing central line-associated bloodstream infection and ventilator-associated pneumonia statewide. Qual Saf Health Care 2010 Dec;19(6):555-61. PMID: 21127114.
47. Sawyer M, Weeks K, Goeschel CA, et al. Using evidence, rigorous measurement, and collaboration to eliminate central catheter-associated bloodstream infections. Crit Care Med 2010 Aug;38(8 Suppl):S292-8. PMID: 20647786.
48. Lin DM, Weeks K, Holzmueller CG, et al. Maintaining and sustaining the On the CUSP: Stop BSI model in Hawaii. Jt Comm J Qual Patient Saf 2013 Feb;39(2):51-60. PMID: 23427476.
49. Lin DM, Weeks K, Bauer L, et al. Eradicating central line-associated bloodstream infections statewide: the Hawaii experience. Am J Med Qual 2012 Mar-Apr;27(2):124-9. PMID: 21918016.
50. Hong AL, Sawyer MD, Shore A, et al. Decreasing central line-associated bloodstream infections in Connecticut intensive care units. J Healthc Qual Epub 2013 Jan 24. PMID: 23347278.
51. Dixon-Woods M, Bosk CL, Aveling E, et al. Explaining Michigan: developing an ex post theory of a quality improvement program. Milbank Q 2011 Jun;89(2):167-205.
52. Martinez EA, Marsteller JA, Thompson DA, et al. The Society of Cardiovascular Anesthesiologists' FOCUS initiative: Locating Errors through Networked Surveillance (LENS) project vision. Anesth Analg 2010 Feb 1;110(2):307-11. PMID: 20081130.
53. Martinez EA, Shore A, Colantuoni E, et al. Cardiac surgery errors: results from the UK National Reporting and Learning System. Int J Qual Health Care 2011 Apr;23(2):151-8. PMID: 21224272.
54. Martinez EA, Thompson DA, Errett NA, et al. Review article: High stakes and high risk: a focused qualitative review of hazards during cardiac surgery. Anesth Analg 2011 May;112(5):1061-74. PMID: 21372272.
55. Gurses AP, Kim G, Martinez EA, et al. Identifying and categorising patient safety hazards in cardiovascular operating rooms using an interdisciplinary approach: a multisite study. BMJ Qual Saf 2012 Oct;21(10):810-8. PMID: 22562873.
56. Pronovost PJ, Bo-Linn GW. Preventing patient harms through systems of care. JAMA 2012 Aug 22;308(8):769-70. PMID: 22910751.
57. Edworthy J. Medical audible alarms: a review. J Am Med Inform Assoc 2013 May 1;20(3):584-9. PMID: 23100127.
58. Talley LB, Hooper J, Jacobs B, et al. Cardiopulmonary monitors and clinically significant events in critically ill children. Biomed Instrum Technol 2011 Spring;Suppl:38-45. PMID: 21599480.
59. Hudson DW, Holzmueller CG, Pronovost PJ, et al. Toward improving patient safety through voluntary peer-to-peer assessment. Am J Med Qual 2012 May-Jun;27(3):201-9. PMID: 22202557.
60. Mathews SC, Pronovost PJ. Commentary: Establishing safety and quality as core values: a hospital road map. Am J Med Qual 2012 Jul-Aug;27(4):348-9. PMID: 22205770.
