Advanced machine learning in action: identification of intracranial hemorrhage on computed tomography scans of the head with clinical workflow integration

Abstract

Intracranial hemorrhage (ICH) requires prompt diagnosis to optimize patient outcomes. We hypothesized that machine learning algorithms could automatically analyze computed tomography (CT) of the head, prioritize radiology worklists and reduce time to diagnosis of ICH. 46,583 head CTs (~2 million images) acquired from 2007–2017 were collected from several facilities across Geisinger. A deep convolutional neural network was trained on 37,074 studies and subsequently evaluated on 9499 unseen studies. The predictive model was implemented prospectively for 3 months to re-prioritize “routine” head CT studies as “stat” on realtime radiology worklists if an ICH was detected. Time to diagnosis was compared between the re-prioritized “stat” and “routine” studies. A neuroradiologist blinded to the study reviewed false positive studies to determine whether the dictating radiologist overlooked ICH. The model achieved an area under the ROC curve of 0.846 (0.837–0.856). During implementation, 94 of 347 “routine” studies were re-prioritized to “stat”, and 60/94 had ICH identified by the radiologist. Five new cases of ICH were identified, and median time to diagnosis was significantly reduced (p < 0.0001) from 512 to 19 min. In particular, one outpatient with vague symptoms on anti-coagulation was found to have an ICH which was treated promptly with reversal of anticoagulation, resulting in a good clinical outcome. Of the 34 false positives, the blinded over-reader identified four probable ICH cases overlooked in original interpretation. In conclusion, an artificial intelligence algorithm can prioritize radiology worklists to reduce time to diagnosis of new outpatient ICH by 96% and may also identify subtle ICH overlooked by radiologists. This demonstrates the positive impact of advanced machine learning in radiology workflow optimization.

A computer program that automatically analyzes brain images from patients undergoing CT scans of the head can reliably flag those with signs of hemorrhage. A team of researchers from Geisinger in Danville, Pennsylvania, USA, trained and tested a machine-learning algorithm using 46,583 computed tomography imaging studies of the head. Subsequently, they implemented the model into routine care for 3 months to help prioritize radiology worklists. Of 347 routine cases, the computer identified 94 as having an intracranial hemorrhage, two-thirds of which were confirmed by a radiologist, including five among patients who had a new diagnosis of a brain bleed. The algorithm reduced the average time in which a radiologist diagnosed these patients from around 8.5 h to just 19 min, demonstrating the positive impact of incorporating artificial intelligence into radiology workflow.