Clinical decision support AI
Rule-based CDS fires when a condition is met and can always explain itself. AI-based CDS predicts — and the shift from 'because rule 14' to 'because the model said 0.83' changes what safety even means.
In one line
Classical CDS applies a rule someone wrote. AI-based CDS predicts from patterns nobody wrote down. Both can be right; only one can tell you why — and that difference decides how you govern it, not just how you build it.
Two different animals
Rule-based CDS is what the CDS Hooks lab and most live systems do: if creatinine is X and the drug is Y, warn. It's deterministic, auditable, and its failure mode is boring — a rule is wrong, you find it, you fix it. You can always answer "why did it fire?" because someone wrote the reason down.
AI-based CDS learns from data: sepsis risk scores, deterioration prediction, imaging triage, readmission risk. It finds signal a human never articulated. Its failure modes are not boring at all.
The failure that should be taught in every course
The Epic Sepsis Model is the case worth knowing, because it is not a straw man — it was a proprietary model deployed at hundreds of US hospitals, and clinicians trusted it.
External validation published in JAMA Internal Medicine (2021) found substantially worse real-world performance than advertised: poor discrimination, a large volume of alerts, and many sepsis cases missed. It had been widely deployed before anyone independently checked.
Three lessons, and none of them are about the algorithm:
- Vendor-reported performance is a marketing claim until externally validated. The model was not evil; it was untested where it was used.
- The prevalence of the thing you predict changes everything. A model with excellent AUC in one population produces mostly false alarms in another, because the maths of positive predictive value depends on how common the condition is. The maths didn't fail — the transfer assumption did.
- Alert fatigue is a clinical harm, not an annoyance. A model producing many low-value alerts trains clinicians to dismiss all alerts — including the true one, tomorrow.
Dataset shift — the quiet killer
A model is trained on the past. Care changes. A new protocol, a new lab analyser, a different patient mix after a nearby hospital closes, a pandemic — and the relationship the model learned no longer holds. Nothing announces this. The model keeps producing confident numbers, because that is all it can do.
This is why "we validated it at go-live" is not a safety argument. Rule-based CDS decays visibly (someone notices the guidance changed). AI-based CDS decays silently, which is why it needs monitoring as a permanent obligation, not a launch task.
Explainability, honestly
The demand that clinical AI "explain itself" is right in spirit and often confused in practice. A feature-importance chart is not an explanation of this patient's risk; it's a statement about the model. Many "explanations" are post-hoc reconstructions that sound satisfying and don't constrain the model's actual behaviour.
The honest position: prefer a simpler model you can interrogate when the performance difference is small, and where you must use an opaque one, invest in monitoring and accountability rather than in a story that makes people comfortable. Comfort is not safety.
Where the regulator draws the line
The FDA's CDS guidance turns on whether the clinician can independently review the basis for the recommendation. Software that shows its reasoning so a clinician can reach their own conclusion is treated differently from software that issues a directive the clinician is expected to follow. That distinction — support the decision versus make the decision — is not bureaucratic. It's the difference between a tool and a practitioner, and it determines whether the thing is a regulated medical device.
What good looks like
- A named clinical owner, not just a technical one.
- External or local validation before go-live, on your population.
- Monitoring after go-live, with a defined trigger for withdrawal.
- The alert burden measured — and treated as a cost with a budget.
- A route to challenge it. If no clinician has ever overridden the model, that is a red flag, not a success metric.
The AI is the easy part. Everything above is the job.