U.S. flag

An official website of the Department of Health and Human Services

SHARE:

linkedInFacebookXEmail AHRQPrintShare

R18 Closed Loop Diagnostics: AHRQ R18 Patient Safety Learning Laboratories

Grant Number:
R18HS027282
Principal Investigator(s):
Russell S. Phillips, Harvard Medical School
James C. Benneyan, Northeastern University
Gordon D. Schiff, Harvard Medical School
Topics:
Patient Safety Learning Labs
Medical Errors
Grant Period:
September 30, 2022 – September 29, 2024
Goals/Purpose:

The proposed research sought to address diagnostic errors due to failures in follow-up. More explicitly the research focused on efforts to “close the loop” on the public health concern of diagnostic errors by employing system engineering methods in three diverse healthcare practices to create a system that provides highly reliable follow-up of diagnostic tests, referrals, and symptom evolution. Results are planned to be disseminated broadly to improve diagnostic safety nationally.

Specific Aims:
  1. Design, develop, and refine highly reliable “closed loop” systems for diagnostic tests and referrals that ensure these occur within clinically- and patient-important timeframes.
  2. Design, develop, and refine a highly reliable “closed loop” symptom monitoring system to ensure clinicians receive information about evolving symptoms and physical findings of concern.
  3. Ensure broader generalizability of results of Aims 1 and 2 by ensuring these new processes are effective in a community health center in an underserved community, a large telemedicine system, and in simulations representing a range of other health system settings and populations.

Publications

Expand All Collapse All

Analysis of Radiology Report Recommendation Characteristics and Rate of Recommended Action Performance

Citation:
White T, Aronson MD, Sternberg SB, Shafiq U, Berkowitz SJ, Benneyan J, Phillips RS, Schiff GD. Analysis of Radiology Report Recommendation Characteristics and Rate of Recommended Action Performance. JAMA Netw Open. 2022 Jul 1;5(7):e2222549. doi: 10.1001/jamanetworkopen.2022.22549. PMID: 35867062; PMCID: PMC9308057.
Summary:
This study assessed the completion rates of actionable radiology recommendations in primary care. Radiology reports were redesigned to include a dedicated field for recommendations, categorized by the action, time frame, and contingency language. Of 598 reports reviewed, 87.4% of recommendations were completed on time, but 31 cases posed significant clinical risks due to non-completion. Factors improving completion included shorter time frames, no contingency language, and direct communication between radiologists and primary care physicians. The findings highlight the need for more reliable systems to ensure follow-through on radiology recommendations.
Link to PubMed

Completion Rates and Timeliness of Diagnostic Colonoscopies for Rectal Bleeding in Primary Care

Citation:
Campbell KA, Sternberg SB, Benneyan J, Flier SN, Amat M, Salant T, Nambara K, Fernandez L, Feuerstein J, Shafiq U, Phillips RS, Aronson MD, Schiff GD. Completion Rates and Timeliness of Diagnostic Colonoscopies for Rectal Bleeding in Primary Care. J Gen Intern Med. 2024 May;39(6):985-991. doi: 10.1007/s11606-023-08513-9. Epub 2023 Nov 8. PMID: 37940753; PMCID: PMC11074061.
Summary:
This study examines the timeliness of diagnostic colonoscopies for patients with rectal bleeding in primary care. Of 770 patients, 22.3% (172) did not receive timely follow-up. Common process failures included no procedure ordered (36%), procedures not scheduled (26%), and scheduled procedures that were cancelled or missed (18%). The COVID-19 pandemic did not significantly impact these rates. The authors stress the need for improved systems to ensure timely ordering, scheduling, and completion of colonoscopies to enhance care for patients with rectal bleeding.
Link to PubMed

Low Rate of Completion of Recommended Tests and Referrals in an Academic Primary Care Practice with Resident Trainees

Citation:
Amat MJ, Anderson TS, Shafiq U, Sternberg SB, Salant T, Fernandez L, Schiff GD, Aronson MD, Benneyan JC, Singer SJ, Graham KL, Phillips RS. Low Rate of Completion of Recommended Tests and Referrals in an Academic Primary Care Practice with Resident Trainees. Jt Comm J Qual Patient Saf 2024 Mar;50(3):177-184. doi: 10.1016/j.jcjq.2023.10.005. Epub 2023 Oct 24. PMID: 37996308.
Summary:
This study examines "failure to close the diagnostic loop" in a resident primary care clinic compared to a faculty practice. Data from 12,282 patient visits across two academic centers (2018-2021) showed that tests ordered by residents had lower completion rates (52.9%) than those ordered by faculty (58.4%). In an adjusted analysis, resident orders were less likely to be completed (odds ratio 0.88). The findings highlight the challenge of ensuring timely diagnostic follow-up, especially in residency training programs where loop closure rates are lower. This represents a significant safety concern in primary care.
Link to PubMed

Closing the Loop: Re-engineering the Assessment and Tracking of Symptoms in Primary Care

Citation:
Phillips RS, Benneyan J, Bargal B, Schiff GD. Closing the Loop: Re-engineering the Assessment and Tracking of Symptoms in Primary Care. J Gen Intern Med. 2023 Mar;38(4):1054-1058. doi: 10.1007/s11606-022-07886-7. Epub 2022 Nov 22. PMID: 36414802; PMCID: PMC10039145.
Summary:
Reliable systems to track symptom progression are essential for accurate diagnosis and follow-up, especially in primary care. Primary care clinicians need efficient tools that simplify tracking without adding to their workload. Some symptoms require urgent attention, while others can be monitored over time. However, many clinicians lack tools to reliably track symptoms. This commentary outlines approaches to developing and implementing these systems in primary care.
Link to PubMed

Analysis of Radiology Report Recommendation Characteristics and Rate of Recommended Action Performance

Citation:
White T, Aronson MD, Sternberg SB, Shafiq U, Berkowitz SJ, Benneyan J, Phillips RS, Schiff GD. Analysis of Radiology Report Recommendation Characteristics and Rate of Recommended Action Performance. JAMA Netw Open. 2022 Jul 1;5(7):e2222549. doi: 10.1001/jamanetworkopen.2022.22549. PMID: 35867062; PMCID: PMC9308057.
Summary:
This study assessed the completion rates of actionable radiology recommendations in primary care. Radiology reports were redesigned to include a dedicated field for recommendations, categorized by the action, time frame, and contingency language. Of 598 reports reviewed, 87.4% of recommendations were completed on time, but 31 cases posed significant clinical risks due to non-completion. Factors improving completion included shorter time frames, no contingency language, and direct communication between radiologists and primary care physicians. The findings highlight the need for more reliable systems to ensure follow-through on radiology recommendations.
Link to PubMed

Systems engineering analysis of diagnostic referral closed-loop processes

Citation:
Nehls N, Yap TS, Salant T, Aronson M, Schiff G, Olbricht S, Reddy S, Sternberg SB, Anderson TS, Phillips RS, Benneyan JC. Systems engineering analysis of diagnostic referral closed-loop processes. BMJ Open Qual. 2021 Nov;10(4):e001603. doi: 10.1136/bmjoq-2021-001603. PMID: 34844935; PMCID: PMC8634018.
Summary:
This study examines the challenges in completing diagnostic referrals, a major patient safety issue with high failure rates (65%-73%). Using dermatology referrals as a case study, the research conducted a systems engineering (SE) analysis of referral processes in two primary care practices. The analysis identified significant workflow variations, reliability issues, and common failures, with only 21% of activities deemed value-added. Most processes relied on low-reliability methods like reminders and workarounds. The study suggests that improving diagnostic referrals requires redesigning processes with high-reliability strategies, rather than incremental changes.
Link to PubMed

Systems Analysis of a Dedicated Ambulatory Respiratory Unit for Seeing and Ensuring Follow-up of Patients With COVID-19 Symptoms

Citation:
Benneyan JC, White T, Nehls N, Yap TS, Aronson M, Sternberg S, Anderson T, Goyal K, Lindenberg J, Kim H, Cohen M, Phillips RS, Schiff GD. Systems Analysis of a Dedicated Ambulatory Respiratory Unit for Seeing and Ensuring Follow-up of Patients With COVID-19 Symptoms. J Ambul Care Manage. 2021 Oct-Dec 01;44(4):293-303. doi: 10.1097/JAC.0000000000000390. PMID: 34319924; PMCID: PMC8386384.
Summary:
The COVID-19 pandemic prompted the redesign of ambulatory care workflows to manage surge volumes and ensure timely care. A dedicated COVID-19 unit was established with standardized workflows, electronic communication, and efficient follow-up processes, achieving over 85% follow-up completion within 24 hours and no infections. Challenges included role confusion, staffing bottlenecks, and patient reluctance to visit or communicate with phone screeners. The study highlights successful innovations as well as areas for improvement.
Link to PubMed