The Checkpoint That Caught What the Model Missed

The Global Signal

Duke Health's Sepsis Watch, developed by the Duke Institute for Health Innovation, is a predictive model that flags patients at elevated risk of sepsis before clinical symptoms are obvious, directly confirmed to predict sepsis a median of five hours before clinical presentation. Duke's own program documentation states the system "doubled" its three-hour sepsis-bundle compliance rate as reported quarterly to the Centers for Medicare and Medicaid Services; a peer-reviewed account in Annals of Emergency Medicine puts this specifically at roughly 28 percent (2016 through the third quarter of 2018) rising to roughly 63 percent (fourth quarter of 2018 through the first quarter of 2019), a figure this draft treats as corroborated by a named peer-reviewed source rather than independently re-confirmed against the original journal page, which returned an access error on this pass. A 2025 peer-reviewed external validation, published in npj Digital Medicine, tested the model across four additional health systems and reported strong discriminative performance, with area-under-curve scores ranging from 0.906 to 0.960.

A separate, independently developed system, Johns Hopkins' TREWS (Targeted Real-time Early Warning System), led by researcher Suchi Saria and published in Nature Medicine, was associated with patients being 20 percent less likely to die of sepsis across a deployment spanning five hospitals, more than 4,000 clinicians, and roughly 590,000 patients, identifying 9,805 sepsis cases at 82 percent sensitivity, compared with under 50 percent for the tools it replaced.

The Hidden Signal

Both systems make the same design choice, and it is the choice that separates them from the failure pattern this series has already documented elsewhere in healthcare: the model's flag is not permitted to trigger a treatment protocol on its own. Duke's Sepsis Watch requires a bedside nurse to review and confirm the flag before the sepsis bundle activates. Johns Hopkins' TREWS requires a clinician to acknowledge the alert within a defined window before the treatment pathway proceeds. Consider what the alternative would look like, the same mechanism this series has already shown in a different clinical context: a hospital builds a similarly capable sepsis-prediction model, deploys it under pressure to move quickly, and never explicitly decides whether a nurse's override is mandatory or merely available. Within a few review cycles, the confirmation step would very plausibly become a formality clinicians click through without genuinely reviewing, exactly the pattern a length-of-stay algorithm elsewhere in this series was allowed to become.

Why the Visible Metric Misleads

The compliance-rate improvement, from roughly 28 percent to roughly 63 percent at Duke, is the number most likely to be repeated in any summary of this program's success, and it is real and externally validated. It is not, on its own, proof that the decision-governance structure behind it is sound. A hospital could post a similar compliance improvement using a system with no mandatory human confirmation at all, simply because the model's predictions were accurate enough on average, and still be exactly as exposed as an unsupervised claims-denial algorithm the first time the model was wrong in a way nobody was positioned to catch. The fact that actually distinguishes Duke's and Johns Hopkins' systems from a decision-blindness failure is not in either program's topline compliance or mortality statistic; it is the specific, less publicized design requirement that a named clinician confirm the flag within a defined window before anything happens to the patient.

Implication for Operators

Any organization deploying a predictive model for a consequential clinical or operational decision should treat the mandatory human-confirmation step, who is required to confirm, within what window, built into the workflow rather than left optional, as the actual measure of whether the system is governed soundly, not the model's own accuracy statistic or a topline compliance number. A model can be highly accurate and still be one unsupervised deployment decision away from becoming exactly the kind of decision-blindness failure this series documents elsewhere.