Understanding the AI Wave: Foundational Knowledge for Improving Diagnosis and Beyond

As AI tools become more deeply integrated into clinical workflows, from ambient documentation to diagnostic support, healthcare providers are increasingly being asked to interpret and act upon model outputs. Given the risks of bias, hallucinations, and lack of transparency described above, this shift places clinicians in a challenging position: they must remain vigilant for potential errors, but often with limited insight into how or why the AI reached a particular conclusion.

This evolution in clinical work reflects a broader change in the role of the provider from decision-maker to AI reviewer. Rather than generating diagnoses or drafting communications from scratch, providers are now reviewing AI-generated outputs and deciding whether to accept, reject, or revise them. Whether an AI tool is flagging lung cancer on a radiograph, suggesting that a patient is at risk of sepsis, or drafting a patient message, the clinician becomes the final filter. Ideally, this human-in-the-loop model offers the best of both worlds: the efficiency of AI combined with the judgment and contextual awareness of human expertise. However, this approach relies on the notion that humans will be effective and consistent reviewers of AI-generated content—an assumption that does not always hold true.⁴⁶ Just as AI models are prone to hallucinations or biased reasoning, humans are susceptible to their own cognitive biases that can undermine their ability to critically evaluate AI outputs. Understanding these psychological dynamics is essential as healthcare systems prepare clinicians for new roles in AI-assisted care.

Although the use of AI in healthcare is relatively new, the field of human-automation interaction has long studied how people respond to automated systems. This body of research offers important insights into how clinicians may interact with AI, including four particularly relevant cognitive phenomena:

• Automation complacency—the tendency to monitor automated systems poorly,⁴⁷ especially when those systems are highly reliable.⁴⁸ If AI models rarely make errors, clinicians may become less vigilant over time and fail to catch the few that do occur—an effect known as vigilance decrement.
• Functional fixedness—a cognitive trap in which individuals struggle to think beyond the most obvious solution.⁴⁹ If an AI model suggests a plausible diagnosis, clinicians may be less likely to consider less common alternatives, even when warranted.
• Confirmation bias—the inclination to favor information that aligns with one’s preexisting beliefs.⁵⁰ If an AI output reinforces what the clinician already suspects, it may be scrutinized less thoroughly.
• Automation bias—the tendency to over-rely on automation,⁵¹ particularly under conditions of time pressure or high workload.⁵² Ironically, because AI tools are often introduced to reduce cognitive burden, in order to reap those benefits, clinicians may be less careful in reviewing AI-generated outputs.

In addition to cognitive bias, over-dependence on AI tools may lead to long-term consequences for clinician performance. One concern is deskilling, in which a clinician’s ability to perform a task independently declines with prolonged reliance on automation. For example, a radiologist using AI to assist with diagnoses may gradually lose confidence or proficiency in interpreting studies without algorithmic input. Another consideration is the generation effect—the phenomenon that information is better retained if it is generated from one’s own mind rather than simply read. Clinicians who rely on ambient digital scribes for documentation may have poorer recall of patient details than if they had written the notes themselves. Just as the realization of cognitive biases helped explain errors in traditional diagnostic reasoning,⁵³ familiar biases may give rise to new forms of error in the context of the human-AI team.

As AI models continue to improve and expand in clinical utility, their potential to enhance care grows. However, realizing that potential depends not just on advancing the technology, but also on fostering an effective human–AI partnership. This will require understanding both the capabilities and limitations of AI, as well as addressing the human factors that shape how clinicians interact with, trust, and rely on these systems.