Longitudinal deep neural networks for assessing metastatic brain cancer on a large open benchmark

Abstract

The detection and tracking of metastatic cancer over the lifetime of a patient remains a major challenge in clinical trials and real-world care. Advances in deep learning combined with massive datasets may enable the development of tools that can address this challenge. We present NYUMets-Brain, the world’s largest, longitudinal, real-world dataset of cancer consisting of the imaging, clinical follow-up, and medical management of 1,429 patients. Using this dataset we developed Segmentation-Through-Time, a deep neural network which explicitly utilizes the longitudinal structure of the data and obtained state-of-the-art results at small (<10 mm³) metastases detection and segmentation. We also demonstrate that the monthly rate of change of brain metastases over time are strongly predictive of overall survival (HR 1.27, 95%CI 1.18-1.38). We are releasing the dataset, codebase, and model weights for other cancer researchers to build upon these results and to serve as a public benchmark.

Cancer is a dynamic disease, with one of its deadly complications being metastatic brain tumors. Here, the authors present a large, multimodal, longitudinal dataset of metastatic cancer, assembled from real world data for cancer research and artificial intelligence (AI) model development. They train time-dependent AI models, and find that novel, dynamic biomarkers exist that are predictive of systemic disease control and overall survival.