Clinician Trust in AI Is Not an Attitude Problem
There is a recurring frustration in healthcare AI circles. A model gets built, validated, deployed. It performs well on the benchmark. And then clinicians don't use it, or use it inconsistently, or override it…
There is a recurring frustration in healthcare AI circles. A model gets built, validated, deployed. It performs well on the benchmark. And then clinicians don't use it, or use it inconsistently, or override it constantly. The engineering team calls this a "trust problem" and starts thinking about explainability features or change management programmes.
This framing is wrong, and it causes real harm to how these systems get designed.
Clinicians who are sceptical of AI tools are not being irrational. In most cases, they are responding entirely reasonably to the information they have. The problem is not their attitude toward AI. The problem is that the systems being built are not yet trustworthy enough to deserve the trust being asked of them.
What Trust Actually Requires
Trust, in any professional context, is built on three things: competence, consistency, and transparency.
Competence means the system does what it claims to do, in the conditions where it will actually be used. A model that performs well on a held-out test set from a different hospital, a different population, or a different time period has not demonstrated competence in your clinical environment. It has demonstrated competence somewhere else.
Consistency means the system behaves predictably. A clinician who works with a tool every day builds a mental model of when to rely on it and when to be sceptical. If the system's behaviour shifts, that model breaks. They have to start from scratch, with no indication that anything changed.
Transparency means the system can account for itself. Not necessarily in the form of a detailed feature attribution for every prediction, but in the sense that a clinician can form a reasonable understanding of what the system is doing and why. "The model said so" is not a clinical justification. It has never been, and it shouldn't be.
Most deployed clinical AI systems are strong on competence in controlled conditions, weak on consistency over time, and poor on transparency in practice. Scepticism is the appropriate response to that combination.
The Explainability Detour
The field spent several years treating explainability as the solution to the trust problem. If clinicians could see which features drove a prediction, they would understand the system and therefore trust it.
This turned out to be more complicated. Explanation interfaces add cognitive load to workflows that are already stretched. Feature attributions are often hard to interpret without statistical training. And in some cases, seeing the explanation makes clinicians less likely to follow a correct recommendation, not more, because the highlighted features don't match their clinical intuition.
Explainability is valuable. It is not the same thing as trustworthiness. A system can be fully explainable and still be wrong in ways that matter. A system can be hard to explain in detail and still be reliable enough to trust in practice.
The distinction worth drawing is between explanation, which is a feature, and accountability, which is a property of the system as a whole. Clinicians want to know that if the system makes a mistake, there is a mechanism to detect it, correct it, and prevent recurrence. That is an accountability question, not an explainability question, and it requires different engineering.
What Clinicians Are Actually Asking For
Research into clinician attitudes toward AI tools, including work in my own area, points consistently to a few things that matter more than explanation interfaces or accuracy metrics.
They want to know when the system is uncertain, not just what it recommends. A confident wrong answer is more dangerous than a hedged one. Systems that communicate their own uncertainty, and do so honestly, are more useful than systems that project confidence they don't have.
They want continuity. If a tool changes its behaviour, they want to know. Not a changelog buried in release notes, but a signal in the workflow itself. "This system has been updated since you last used it" is a small thing that would change the trust dynamic significantly.
They want a clear line of responsibility. When an AI recommendation is wrong and a patient is harmed, who is accountable? The clinician who followed it? The hospital that deployed it? The company that built it? This is unresolved in most jurisdictions and in most institutions. Clinicians are aware of this ambiguity, and it is a rational reason to be cautious.
Building for Trust Rather Than Adoption
The framing of "clinician trust" as a barrier to AI adoption puts the problem in the wrong place. It suggests that the goal is to get clinicians to trust the system, and that resistance is something to be overcome.
A better framing: trustworthiness is a design property, and adoption follows from it. Systems that are competent in the actual deployment environment, that behave consistently, that communicate uncertainty, that log their changes, and that sit within a clear accountability structure will be used. Not because clinicians have been convinced, but because the system deserves the confidence being placed in it.
This requires more than a better model. It requires thinking carefully about how the system behaves over time, how it is monitored, how changes to it are communicated, and who is responsible when it fails. These are engineering and governance questions as much as they are machine learning questions.
The technology is capable enough. The question now is whether the systems built around it are.
Kavishwa Wendakoon is a doctoral researcher at the University of Oulu working on self-adaptive AI in clinical decision support.