Content area
Background
Sepsis survivors face substantial risks of late mortality following discharge, underscoring the critical need for early prediction and targeted interventions for this vulnerable population. Early identification of those at high risk of mortality following discharge may optimize healthcare resource allocation. We sought to feed common clinical available data to a deep learning algorithm for predicting short to long-term mortality in sepsis survivors.
Methods
This retrospective study, using a real-world database (MIMIC-IV database), screened adult critically ill patients with sepsis (as defined by Sepsis-3) admitted to the ICU with stays exceeding four days and who were discharged alive. Static features including patient characteristics, comorbidities, laboratory tests at ICU admission, and each dynamic SOFA component score over the first four days post-ICU admission were collected. We developed a deep learning-based combined model (DL-CMT) for post-discharge mortality prediction using multidimensional and time-series data. Comparisons were made with a multilayer perceptron and two machine learning models of random forest and eXtreme Gradient Boosting (XGBoost).
Results
7532 patients fulfilled the inclusion criteria, and the observed mortality rates were 30.7% at 28 days, 33.6% at 90 days, and 39.4% at one year post-ICU discharge. The proposed DL-CMT model achieved the best performance for mortality prediction at all the intervals, with area under the receiver operating characteristic curve of 0.95 (95% confidence interval [CI] 0.93–0.96), 0.92 (95% CI 0.90–0.94), and 0.90 (95% CI 0.87–0.92), respectively. Our model outperformed the multilayer perceptron, random forest, and XGBoost in all endpoints. Ablation experiments confirmed the model’s robustness, maintaining performance despite the absence of a specific physiological component.
Conclusions
Sepsis survivors have persistently high mortality risks post-discharge. The DL-CMT model, leveraging dynamic SOFA component scores and static features, demonstrated superior predictive performance for short to long-term mortality. This model has the potential to assist clinicians in optimizing post-discharge management and improving follow-up care.
Introduction
Sepsis, a life-threatening organ dysfunction due to a dysregulated host response to infections [1], remains a major global health challenge despite advancements in critical care and the implementation of standardized treatment protocols [2]. The progress in critical care medicine has reduced hospital mortality, thereby increasing the number of sepsis survivors (defined in this study as adult patients who survived to discharge following a ICU admission for sepsis) who are discharged with sequelae from critical illness [3]. Sepsis survivorship represents a critical, yet often overlooked, global health challenge [4, 5]. Those survivors may often face long-term health issues and reduced quality of life, particularly with late mortality or high readmission rates emerging as critical concerns [6].
The Sequential Organ Failure Assessment (SOFA) score, developed in 1994 and published in 1996 [7], is a widely used tool for assessing and monitoring organ dysfunction in critically ill patients, particularly, as a key component of the Sepsis-3 definition [1]. The SOFA scoring system assesses six organ systems: respiratory, cardiovascular, liver, coagulation, renal, and central nervous system (CNS). Each component of the SOFA score reflects the degree of dysfunction in a specific organ system, with scores ranging from 0 (normal function) to 4 (high degree of dysfunction). SOFA or quick SOFA (qSOFA) has been used for hospital mortality prediction in several studies [8, 9–10]. Despite the widespread use of the SOFA score, traditional static models using SOFA scores at a single time point may not fully capture the dynamic nature of organ dysfunction. Temporal progression of SOFA scores provides valuable insights into the patient’s condition, however often underutilized in conventional predictive models [8].
Recent advancements in deep learning, particularly recurrent neural networks (RNNs) and their variants such as Long Short-Term Memory (LSTM) networks [11], have shown great potential in handling complex, high-dimensional, and time-dependent data. These models are well-suited for processing time series data and capturing temporal dependencies, demonstrating promise in various medical applications, including predicting patient outcomes based on physiological data and electronic health records. Predicting long-term mortality for sepsis survivors remains a complex challenge due to the need to account for both static patient characteristics and dynamic clinical features over time. While survival analysis techniques, such as Random Survival Forest, Survival XGBoost, and advanced methods like DeepSurv or DeepHit, are well-suited for time-to-event modeling [12], they often require significant computational resources and may not align directly with clinical workflows. In this study, we adopted an interval-based approach, categorizing mortality risks at clinically actionable time points (28-day, 90-day, and 1-year), which facilitates targeted interventions and resource allocation. This method prioritizes practical utility in a clinical setting while maintaining robust predictive performance.
This study proposes a deep learning-based model for mortality prediction in sepsis survivors by combining dynamic sequential SOFA component scores with data from the first 24 h after ICU admission. Our approach leverages information on the course of the ICU treatment reflected by dynamic SOFA scores over the first few days and integrates multiple data sources to enhance predictive power. We aim to develop a model that learns from the progression of SOFA scores over time with basic admission information to provide feasible short to long term mortality predictions for sepsis survivors. By providing mortality risk assessments at actionable time points, this study may enable healthcare providers to prioritize follow-up care and optimize resource allocation for patients at high risk of mortality.
Methods
Design and study population
The data utilized in this retrospective study were obtained by accessing Medical Information Mart for Intensive Care-IV (MIMIC-IV, version 2.2) database [13]. The electronic health record dataset encompassing > 50,000 patients admitted to the intensive care units (ICU) at Beth Israel Deaconess Medical Center (BIDMC) in Boston, Massachusetts, from 2008 to 2019. The Institutional Review Board of BIDMC granted a waiver of informed consent and approved the sharing of research resources. Access to the database was authorized for one of the authors (JW) (Record ID: 45,699,941).
Adult critically ill patients diagnosed with sepsis (Sepsis-3 definition) [1] on ICU admission or within the first 24 h after admission and who were discharged alive from the ICU were included in this study. Considering that dynamic sequential subsystem SOFA scores within the first 4 days were used for model development, patients with ICU stays of less than 4 days or those without complete SOFA scores from the first to the fourth day were excluded. Additionally, for patients with multiple ICU admissions, only data from the last admission were used.
Data collection
Electronic health record data were extracted for each patient, including demographic characteristics, existing comorbidities upon ICU admission, hematological and biochemical tests, vital sign data and clinical severity scores during the first day after ICU admission, all serving as static baseline characteristics. If more than one value was available at sepsis diagnosis time points, the worst value was selected. For time series features, we extracted each component scoring of total SOFA scores according to the standard used in Sepsis-3 definition [1]. If multiple SOFA score assessments were available per 24-h period, the worst value (highest score) per day according to the preconcerted data processing approach was used for algorithm development. Specifically, this process targeted the acquisition of data across six principal dimensions: demographic characteristic, existing comorbidities at ICU admission, vital signs, clinical severity score in the first day, lab indexes including hematological and biochemical parameters, and six dynamic SOFA component scores over the first four days. The complete list of collected features included in this study for model development is shown in Additional file: Table S1.
The target variable was short to long term-mortality (28-day, 90-day, and one-year) after ICU discharge in sepsis survivors. Patient death was identified through death registration within one year from the patient’s last hospital discharge in MIMIC IV database. The occurrence of mortality was extracted as a binary event [0, 1].
Data preprocessing
During the feature processing stage, variables with a missing rate exceeding 20% were excluded, and the remaining missing values were filled using multiple imputation to reduce bias and improve accuracy. For categorical features, none of which had missing values in this current dataset, we built a dictionary for each feature and mapped them to one-hot vectors. For continuous numerical features, to facilitate data comparison and model training, we standardize the data to the same scale. Specifically, we calculate the mean and standard deviation and use z-transformation {} for normalization. For SOFA time series data, we extracted data to calculate SOFA from four periods—daily SOFA score in the first four days upon ICU admission. A total SOFA score was composed of six component scores—respiration, coagulation, liver, cardiovascular, CNS, and renal. Each subsystem score was standardized using z-transformation similarly to the standardization process for continuous numerical features. Given the target outcome’s positive-to-negative ratio of less than 1:4, no further imbalance adjustments were made.
Prediction model development
The target variable was 28-day mortality, 90-day mortality and one-year mortality after ICU discharge, which can be derived from death registration information in the MIMIC IV 2.2. The architecture of our proposed DL-based combined model with multidimension and time series (DL-CMT) for dynamic post-discharge mortality prediction is illustrated in Fig. 1. The DL-CMT comprised three distinct paths: numerical feature extraction, categorical feature embedding, and time series feature extraction.
[See PDF for image]
Fig. 1
Architecture of the proposed post-discharge mortality prediction model. Data processing workflow was performed according to the proposed DL-based combined model with multidimension and time series (DL-CMT). Conv, convolution; LSTM, long short-term memory; FC, fully connected. ReLu, rectified linear unit. SOFA, sequential organ failure assessment
In the continuous numerical feature extraction path, each feature was mapped to a vector, and a convolutional neural network (CNN) was used to capture the relationships between different features. Specifically, each feature was assigned a unique vector representation. Since these numerical values have been normalized, their values were treated as weights and performed a dot product with the corresponding feature vectors. Next, a one-dimensional multivariate CNN with convolutional kernels of sizes 5, 7, 9, and 11 was used for feature extraction to capture interactions between features at different granularities. Subsequently, the extracted features underwent a nonlinear transformation using the ReLU activation function, followed by a max-pooling layer to extract overall features, which were then concatenated and passed through a fully connected layer for numerical feature output.
The categorical feature embedding path involved a lookup table providing numerical embeddings for each category, as shown in Fig. 1. For each feature, the corresponding embedding was extracted from the lookup table, and the overall output for the categorical feature was computed through a fully connected layer.
For temporal series features, the SOFA component score, a dynamic feature required time-dependent analysis. The total SOFA score was divided into six different components, which we believe have distinct meanings, each initialized with a unique vector similar to the method used for numerical feature extraction. Then, a bidirectional long short-term memory network (BiLSTM) was used to acquire and encode temporal features. Features were extracted from the final state, and the overall output for temporal features was computed through a fully connected layer.
Finally, the outputs from the numerical features, categorical feature embeddings, and time-series features were concatenated into a single vector. The survival predictions for 28-day, 90-day, and 1-year were respectively output through three different fully connected layers. During the model updating process, the loss values of these three predictions were summed, and their total was used as the overall loss function to optimize the model (Additional file: Figure S1).
Data was randomly divided by patients into training, validation, and test sets in a ratio of roughly 8:1:1, with 80% of the data used for training, 10% for validation and 10% for test. The training set was used to train the proposed models. The validation set was used to compare trained models and identify the best model architecture and hyperparameters. All models were trained for 60 epochs and implemented based on Python 3.8 (https://www.python.org/), and the epoch yielding highest accuracy performance on the validation set was selected for performance comparison on the test set (Additional file: Figure S2).
Other models for comparison
To evaluate the effectiveness of the feature extraction model, we introduced a multilayer perceptron (MLP) approach as a basic deep learning model for comparison. The specific steps were as following: we removed the CNN for continuous numerical variables and LSTM for temporal variables (SOFA), retaining only the fully connected layers for feature extraction. We also compared our proposed DL-CMT with two common machine learning models: random forest (RF) and eXtreme Gradient Boosting (XGBoost).
Model performance evaluation
Mortality prediction (28-day, 90-day or 1-year mortality) was the evaluation task in this study, accessed using death registration information from the MIMIC IV database. Mortality prediction is a classification problem, and the area under the receiver operating characteristic curve (AUROC) was chosen as the primary comprehensive performance metric to ensure a fair comparison. To determine the optimal probability cut-off values for each prediction endpoint (28-day, 90-day, and 1-year mortality), the Youden Index was utilized. The Youden Index, defined as J = Sensitivity + Specificity – 1, identifies the threshold that maximizes the trade-off between sensitivity and specificity. The cut-off values were calculated based on the receiver operating characteristic (ROC) curve for each prediction interval, and the corresponding sensitivity and specificity metrics were reported. Additionally, accuracy, recall, F-1 score, area under the precision-recall curve (AUPRC), and confusion matrix were also included as metrics to reflect the prediction performance of the artificial intelligence algorithms.
Statistics
Continuous variables were presented as mean (standard deviation, SD) or median (25th percentile, 75th percentile) and compared using the two tailed Student’s t-test or the Mann–Whitney U test. Categorical variables were expressed as number (percentage) and assessed using Chi-square (χ2) test or Fisher's exact test. All statistical analyses were performed using the Python package NumPy. A two-sided P value ≤ 0.05 was considered statistically significant.
Results
Data description
A total of 73,181 ICU admissions were screened for sepsis selection according to the Sepsis-3 diagnosis criteria [1]. Of these 32,971 admissions diagnosed as sepsis, 7532 unique patients discharged alive met the inclusion criteria and were finally included in the analysis (Fig. 2). The total cohort was dived into training, validation, and test sets in a ratio of 8:1:1. Baseline characteristics of the study cohort are presented in Table 1 and Additional file: Table S2, with a mean age 64.4 years (SD: 16.4), and female constituting 42.8% (n = 3220). The most common comorbidity was congestive heart failure (34.2%), followed by diabetes (33.3%). The observed mortality rate in sepsis survivors reached up to 30.3% at 30 days, 33.6% at 90 days, and 39.4% at one year, respectively.
[See PDF for image]
Fig. 2
Flowchart of the patient selection process
Table 1. Patient characteristics of the sepsis survivor cohort and observed mortality
Feature | Training cohort (n = 6025) | Validation cohort (n = 754) | Test cohort (n = 753) | Overall population (n = 7532) |
|---|---|---|---|---|
Demographic characteristics | ||||
Female, n (%) | 2567 (42.6) | 327 (43.4) | 326 (43.3) | 3220 (42.8) |
Age (years) | 64.3 (16.3) | 64.4 (16.4) | 65.1 (16.6) | 64.4 (16.4) |
Height (cm) | 169.5(10.7) | 169.4 (10.5) | 169.3 (10.1) | 169.4 (10.6) |
Weight (kg) | 84.7 (26.9) | 84.4 (27.4) | 83.2 (24.4) | 84.5 (26.7) |
BMI (kg/cm2) | 29.8 (8.4) | 29.9 (8.9) | 29.6 (8.5) | 29.8 (8.5) |
Race, n (%) | ||||
White | 3736 (62.0) | 464 (61.5) | 456 (60.6) | 4656 (61.8) |
Unknown | 833 (13.8) | 121 (16.1) | 102 (13.5) | 1056 (14.0) |
Others | 1456 (24.2) | 169 (22.4) | 195 (25.9) | 1820 (24.2) |
Charlson comorbidity index | 5 (3, 7) | 5 (3, 7) | 5 (3, 7) | 5 (4, 8) |
Vital signs | ||||
Heart rate (/min) | 87.7 (76.5, 100.1) | 86.8 (75.6, 100.2) | 86.6 (75.1, 100.0) | 87.4 (76.3, 100.1) |
SBP (mmHg) | 113.0 (104.1, 125.1) | 114.0 (105.0, 125.0) | 111.0 (102.3, 125.8) | 113.0 (104.1, 125.2) |
DBP (mmHg) | 60.8 (54.7, 67.8) | 61.2 (54.7, 68.1) | 60.0 (53.5, 67.4) | 60.8 (69.8, 83.3) |
MBP (mmHg) | 75.9 (69.9, 83.2) | 76.3 (70.0, 83.9) | 75.1 (69.1, 83.2) | 75.8 (69.8, 83.3) |
Respiratory rate (/min) | 19.6 (17.1, 22.8) | 19.7 (17.1, 22.7) | 19.6 (17.2, 22.7) | 19.6 (17.1, 22.8) |
Temperature (℃) | 36.9 (36.6, 37.4) | 37.0 (36.6, 37.3) | 36.9 (36.6, 37.3) | 36.9 (36.6, 37.4) |
SpO2 (%) | 93.0 (89.0, 95.0) | 93.0 (90.0, 95.0) | 93.0 (90.0, 95.0) | 93.0 (89.0, 95.0) |
Comorbidity at admission, n (%) | ||||
Myocardial infarction | 1103 (18.3) | 132 (17.5) | 136 (18.1) | 1371 (18.2) |
Congestive heart failure | 2046 (34.0) | 256 (34.0) | 274 (36.4) | 2576 (34.2) |
Peripheral vascular disease | 767 (12.7) | 95 (12.6) | 110 (14.6) | 972 (12.9) |
Cerebrovascular disease | 1250 (20.8) | 149 (19.8) | 143 (19.0) | 1542 (20.5) |
Dementia | 187 (3.1) | 31 (4.1) | 23 (3.1) | 241 (3.2) |
Chronic pulmonary disease | 1687 (28.0) | 237 (31.4) | 229 (30.4) | 2153 (28.6) |
Rheumatic disease | 227 (3.8) | 20 (2.7) | 21 (2.8) | 268 (3.6) |
Peptic ulcer disease | 164 (2.7) | 26 (3.5) | 33 (4.4) | 223 (3.0) |
Liver disease | 1493 (24.8) | 196 (26.0) | 197 (26.2) | 1886 (25.0) |
Diabetes | 2014 (33.4) | 255 (33.8) | 242 (32.1) | 2511 (33.3) |
Hemiplegia or paraplegia | 462 (7.7) | 58 (7.7) | 55 (7.3) | 575 (7.6) |
Renal disease | 1357 (22.5) | 197 (26.1) | 166 (22.1) | 1720 (22.8) |
Malignancy | 773 (12.8) | 97 (12.9) | 104 (13.8) | 974 (12.9) |
Metastatic solid tumor | 342 (5.7) | 41 (5.4) | 41 (5.4) | 424 (5.6) |
Clinical severity score | ||||
GCS | 15 (13, 15) | 15 (13, 15) | 15 (13, 15) | 15 (13, 15) |
SIRS | 3 (2, 4) | 3 (2, 3) | 3 (2, 4) | 3 (2, 4) |
APS III | 51 (38, 68) | 52.5 (39, 69) | 52 (37, 67) | 51 (38, 68) |
LODS | 6 (4, 8) | 6 (4, 8) | 6 (4, 8) | 6 (4, 8) |
OASIS | 35 (30, 41) | 36 (31, 42) | 36 (30, 42) | 35 (30, 42) |
SAPS II | 41 (32, 51) | 41 (33, 51) | 41 (32, 51) | 41 (32, 51) |
Total SOFA at first day# | 6 (4, 9) | 6 (4, 9) | 6 (4, 9) | 6 (4, 9) |
Lab indexes | ||||
Hematocrit (%) | 29.2 (24.8, 34.4) | 29.0 (24.7, 34.8) | 28.6 (24.3, 33.5) | 29.1 (24.7, 34.4) |
Hemoglobin (g/dl) | 9.6 (8.1, 11.4) | 9.5 (8.0, 11.5) | 9.4 (8.0, 11.1) | 9.6 (8.1, 11.4) |
Platelets (10^9/L) | 168.5 (111, 237) | 169.0 (107.0, 235.0) | 167.0 (113.0, 244.0) | 168.0 (111.0, 237.0) |
WBC (10^9/L) | 14.4 (10.2, 19.6) | 14.1 (10.1, 19.8) | 14.5 (10.7, 19.7) | 14.4 (10.2, 19.6) |
Albumin (g/dL) | 3.0 (2.5, 3.5) | 2.9 (2.5, 3.5) | 3.0 (2.5, 3.5) | 3.0 (2.5, 3.5) |
Anion gap (mmol/L) | 16.0 (14.0, 20.0) | 16.5 (14.0, 20.0) | 16.0 (14.0, 19.0) | 16.0 (14.0, 20.0) |
Bicarbonate (mmol/L) | 21.0 (18.0, 24.0) | 21.0 (18.0, 24.0) | 21.0 (18.0, 24.0) | 21.0 (18.0, 24.0) |
Calcium (mmol/L) | 2.0 (1.9, 2.1) | 2.0 (1.9, 2.1) | 2.0 (1.9, 2.1) | 2.0 (1.9, 2.1) |
Chloride (mmol/L) | 106.0 (102.0, 110.0) | 106.0 (102.0, 110.0) | 106.0 (102.0, 110.0) | 106.0 (102.0, 110.0) |
BUN (mg/dL) | 24.0 (16.0, 41.0) | 27.0 (16.0, 43.0) | 23.0 (15.0, 40.0) | 24.0 (16.0, 41.0) |
Bilirubin (mg/dL) | 0.8 (0.4, 1.7) | 0.8 (0.4, 1.6) | 0.8 (0.4, 1.7) | 0.8 (0.4, 1.7) |
Creatinine (mg/dL) | 1.2 (0.8, 2.0) | 1.2 (0.8, 2.1) | 1.1 (0.8, 1.8) | 1.2 (0.8, 2.0) |
Glucose (mmol/L) | 7.5 (6.4, 9.2) | 7.4 (6.3, 8.9) | 7.5 (6.4, 9.1) | 7.5 (6.4, 9.1) |
Sodium (mmol/L) | 137.0 (134.0, 140) | 138.0 (134.0, 140.00 | 137.0 (134.0, 140) | 137.0 (134.0, 140) |
Potassium (mmol/L) | 3.8 (3.5, 4.2) | 3.9 (3.5, 4.3) | 3.9 (3.5, 4.3) | 3.8 (3.5, 4.2) |
INR | 1.4 (1.2, 1.7) | 1.4 (1.2, 1.7) | 1.4 (1.2, 1.8) | 1.4 (1.2, 1.7) |
PT (s) | 14.9 (12.9, 19.0) | 14.9 (12.9, 18.7) | 15.1 (13.1, 19.2) | 15.0 (12.9, 19.0) |
PTT (s) | 33.9 (28.6, 48.4) | 33.3 (28.3, 47.7) | 34.4 (29.0, 46.6) | 33.9 (28.6, 48.1) |
Lactate (mmol/L) | 2.3 (1.4, 4.0) | 2.2 (1.4, 3.7) | 2.2 (1.4, 4.0) | 2.3 (1.4, 4.0) |
Urine output (mL) | 1461.5 (850, 2325) | 1417 (830, 2125) | 1371 (856, 2300) | 1450 (850, 2305) |
Vital signs | ||||
Heart rate (/min) | 87.7 (76.5, 100.1) | 86.8 (75.6, 100.2) | 86.6 (75.1, 100.0) | 87.4 (76.3, 100.1) |
SBP (mmHg) | 113.0 (104.1, 125.1) | 114.0 (105.0, 125.0) | 111.0 (102.3, 125.8) | 113.0 (104.1, 125.2) |
DBP (mmHg) | 60.8 (54.7, 67.8) | 61.2 (54.7, 68.1) | 60.0 (53.5, 67.4) | 60.8 (69.8, 83.3) |
MBP (mmHg) | 75.9 (69.9, 83.2) | 76.3 (70.0, 83.9) | 75.1 (69.1, 83.2) | 75.8 (69.8, 83.3) |
Respiratory rate (/min) | 19.6 (17.1, 22.8) | 19.7 (17.1, 22.7) | 19.6 (17.2, 22.7) | 19.6 (17.1, 22.8) |
Temperature (℃) | 36.9 (36.6, 37.4) | 37.0 (36.6, 37.3) | 36.9 (36.6, 37.3) | 36.9 (36.6, 37.4) |
SpO2 (%) | 93.0 (89.0, 95.0) | 93.0 (90.0, 95.0) | 93.0 (90.0, 95.0) | 93.0 (89.0, 95.0) |
Follow-up mortality, n (%) | ||||
28-day | 1825 (30.3) | 224 (29.7) | 231 (30.7) | 2280 (30.3) |
90-day | 2026 (33.6) | 249 (33.0) | 255 (33.9) | 2530 (33.6) |
one-year | 2365 (39.3) | 303 (40.2) | 297 (39.4) | 2965 (39.4) |
Data were represented as frequency (%), the mean (standard deviation, SD) or median (25th percentile, 75th percentile) where appropriate. # Here, we only showed the total SOFA score at the first day as a baseline characteristic. BMI body mass index, SOFA sequential organ failure assessment, SBP systolic blood pressure, DBP diastolic blood pressure, MBP mean blood pressure, SAPS-II Simplified acute physiology score, GCS Glasgow coma scale, SIRS Systemic Inflammatory Response Syndrome, OASIS Oxford Acute Severity of Illness Score, APS III Acute Physiology Score III, LODS logistic organ dysfunction score, INR international normalized ratio, PT prothrombin time, PTT partial thromboplastin time, SpO2 saturation of peripheral oxygen,WBC white blood cell
Model performance
The proposed DL-CMT model outperformed all three comparison models as evaluated by AUROC (Table 2). As shown in Fig. 3A-C, DL-CMT yielded the highest performance with AUROC values of 0.95 (95% CI: 0.93, 0.96), 0.92 (95% CI: 0.90, 0.94), 0.90 (95% CI: 0.87, 0.92) for 28-day, 90-day, and 1-year mortality prediction, respectively. The optimal probability cut-off values for the DL-CMT model were determined to be 0.43, 0.37, and 0.41 for 28-day, 90-day, and 1-year mortality prediction (Additional file: Table S3), respectively. At these cut-off values, the sensitivity and specificity for predicting 28-day mortality were 87% and 89%, for 90-day mortality were 85% and 83%, and for 1-year mortality were 85% and 79%, respectively. We further evaluated the model performance using precision-recall (PR) curves, which revealed the interdependence of precision and recall in the model metrics and can more fully express the real performance of models, particularly when dealing with unbalanced classes. As seen in Fig. 3D–F, DL-CMT confirmed the model’s superior performance, achieving the highest AUPRC values of 0.90 (95% CI 0.86, 0.92), 0.87 (95% CI 0.84, 0.90), and 0.86 (95% CI 0.83, 0.89), consistent with Table 2.
Table 2. Performance of the proposed DL-CMT model compared with other models
28-day | 90-day | 1-year | ||||
|---|---|---|---|---|---|---|
AUROC (95% CI) | AUPRC (95% CI) | AUROC (95% CI) | AUPRC (95% CI) | AUROC (95% CI) | AUPRC (95% CI) | |
DL-CMT | 0.95 (0.93, 0.96) | 0.90 (0.86, 0.92) | 0.92 (0.90, 0.94) | 0.87 (0.84, 0.90) | 0.90 (0.87, 0.92) | 0.86 (0.83, 0.89) |
MLP | 0.91 (0.89, 0.93) | 0.82 (0.77, 0.86) | 0.89 (0.87, 0.91) | 0.82 (0.77, 0.86) | 0.85 (0.83, 0.88) | 0.80 (0.75, 0.84) |
RF | 0.90 (0.88, 0.92) | 0.80 (0.75, 0.85) | 0.89 (0.86, 0.91) | 0.80 (0.75, 0.84) | 0.87 (0.84, 0.89) | 0.81 (0.77, 0.85) |
XGBoost | 0.93 (0.91, 0.95) | 0.85 (0.80, 0.89) | 0.90 (0.89, 0.93) | 0.83 (0.80, 0.88) | 0.86 (0.84, 0.89) | 0.82 (0.78, 0.85) |
DL-CMT the proposed DL-based combined model with multidimension and time series (DL-CMT) for dynamic post-discharge mortality prediction; MLP multilayer perceptron; RF random forest; XGBoost eXtreme Gradient Boosting; AUROC area under the receiver operating characteristic curve, AUPRC area under the precision-recall curve, CI confidence interval
[See PDF for image]
Fig. 3
Performance characteristic curves of models for mortality prediction at different endpoints on the test set. A–C area under the receiver operating characteristics curve (AUROC) for the 28-day, 90-day, and 1-year mortality prediction, respectively; D–F area under the precision-recall curve (AUPRC) for the 28-day, 90-day, and 1-year mortality prediction, respectively. DL-CMT, the proposed DL-based combined model with multidimension and time series (DL-CMT) for dynamic post-discharge mortality prediction; MLP, multilayer perceptron; RF, fandom forest; XGBoost, eXtreme Gradient Boosting
Additionally, we employed various metrics to evaluate and compare the performance results among different models on the same test dataset, including accuracy, precision, recall, and F1-score (Table 3) and confusion matrix (Fig. 4). All these results indicated our proposed model have good predictive performance for short to long term mortality after discharge in sepsis survivors.
Table 3. Performance evaluation of prediction models for mortality
Prediction endpoint | Model | Metrics | |||
|---|---|---|---|---|---|
Accuracy | Precision | Recall | F1-score | ||
28-day | DL-CMT | 88.71 | 88.60 | 88.71 | 88.63 |
MLP | 84.33 | 83.98 | 84.33 | 83.95 | |
RF | 83.67 | 83.32 | 83.67 | 83.06 | |
XGBoost | 86.32 | 86.09 | 86.32 | 86.11 | |
90-day | DL-CMT | 86.19 | 86.10 | 86.19 | 86.13 |
MLP | 81.14 | 80.83 | 81.14 | 80.90 | |
RF | 81.14 | 80.80 | 81.14 | 80.60 | |
XGBoost | 84.20 | 83.98 | 84.20 | 84.00 | |
1-year | DL-CMT | 82.34 | 82.29 | 82.34 | 82.31 |
MLP | 78.49 | 78.46 | 78.49 | 78.47 | |
RF | 79.02 | 78.84 | 79.02 | 78.84 | |
XGBoost | 76.23 | 76.10 | 76.23 | 76.15 | |
Results were shown with value (%). The best performance results are in bold, and the proposed model is underlined. DL-CMT the proposed DL-based combined model with multidimension and time series (DL-CMT) for dynamic post-discharge mortality prediction; MLP multilayer perceptron; RF random forest, XGBoost eXtreme Gradient Boosting
[See PDF for image]
Fig. 4
Model performance using confusion matrix. Confusion matrix for the proposed DL-CMT, MLP, RF and XGBoost models for predicting (A) 28-day, (B) 90-day and (C) 1-year mortality, respectively. DL-CMT, the proposed DL-based combined model with multidimension and time series (DL-CMT) for dynamic post-discharge mortality prediction; MLP multilayer perceptron, RF fandom forest, XGBoost eXtreme Gradient Boosting
[See PDF for image]
Ablation study
An ablation study is conducted in order to gain insights into validate the functionality of each component of SOFA. Removing different subsystems resulted in performance decreases in accuracy, precision, recall, and F1-score, ranging from 0.93% to 1.33%, 0.6% to 1.27%, 0.93% to 1.33%, and 0.76% to 1.28% (Table 4), respectively. Similar trends were observed for 90-day mortality prediction. For one-year mortality prediction, excluding cardiovascular SOFA did lead to a small but negligible improvement on the predictive metrics. Otherwise, the proposed DL-CMT using dynamic total SOFA possessed slightly better predictive performance over those removing any other component score. However, all the performance differences were small. These experiment results indicated that even with the absence of information from specific physiological subsystems, the model can still maintain a relatively stable performance level overall, demonstrating a certain level of robustness and generalization ability.
Table 4. Performance evaluation of ablation study
DL-CMT | -CNS | -Respiration | -Liver | -Renal | -Coagulation | -Cardiovascular | ||
|---|---|---|---|---|---|---|---|---|
28-day | ||||||||
Accuracy | 88.71 | 87.78 | 87.52 | 87.78 | 87.38 | 87.65 | 87.65 | |
Precision | 88.60 | 87.60 | 87.33 | 88.00 | 87.58 | 87.47 | 87.48 | |
Recall | 88.71 | 87.78 | 87.52 | 87.78 | 87.38 | 87.65 | 87.65 | |
F1-score | 88.63 | 87.58 | 87.35 | 87.87 | 87.46 | 87.38 | 87.51 | |
90-day | ||||||||
Accuracy | 86.19 | 85.92 | 84.33 | 83.67 | 84.99 | 84.86 | 85.92 | |
Precision | 86.10 | 85.75 | 84.30 | 84.05 | 84.91 | 84.67 | 85.83 | |
Recall | 86.19 | 85.92 | 84.33 | 83.67 | 84.99 | 84.86 | 85.92 | |
F1-score | 86.13 | 85.74 | 84.31 | 83.80 | 84.94 | 84.57 | 85.87 | |
1-year | ||||||||
Accuracy | 82.34 | 80.74 | 80.61 | 81.01 | 79.81 | 81.94 | 83.00 | |
Precision | 82.29 | 80.60 | 80.52 | 81.10 | 79.73 | 81.82 | 82.90 | |
Recall | 82.34 | 80.74 | 80.61 | 81.01 | 79.81 | 81.94 | 83.00 | |
F1-score | 82.31 | 80.56 | 80.55 | 81.05 | 79.76 | 81.83 | 82.90 | |
Results were shown with value (%). DL-CMT the proposed DL-based combined model with multidimension and time series (DL-CMT) for dynamic post-discharge mortality prediction; MLP multilayer perceptron; RF fandom forest; XGBoost eXtreme Gradient Boosting. CNS, central nervous system
Discussion
This study demonstrated that adult sepsis survivors, as defined by Sepsis-3 criteria, face heightened risks of short to long-term post-discharge mortality. Our DL-CMT model, which incorporates dynamic SOFA subsystem scores and static patient features, showed significant predictive capability for 28-day, 90-day, and one-year mortality outcomes. Specifically, the proposed DL-CMT model achieved AUROC values of 0.95 (95% CI 0.93, 0.96), 0.92 (95% CI 0.90, 0.94), and 0.90 (95% CI 0.87, 0.92) for these respective time points, outperforming other models such as MLP, RF, and XGBoost.
Researches have documented long-term health issues faced by sepsis survivors, including physical, cognitive, and psychological impairments [6, 14, 15], collectively termed post-intensive care syndrome (PICS), which significantly impact quality of life and contribute to high late mortality. Previous studies primarily focused on predicting ICU or in-hospital mortality in sepsis cohort. This study extends this prediction to the post-discharge period, highlighting the need for improved sepsis survivor care. The one-year mortality rate observed in our cohort was consistent with earlier studies, which reported mortality rates ranging from 16.1% to 51.4% among adult sepsis survivors [16, 17, 18–19]. Of these reported, Shankar-Hari M et al. [19] found there were still 6% to 8% dying per year over the subsequent 5 years following a 15% first-year mortality. Our finding of a 39.4% one-year mortality rate aligned closely with recent data of 30.7% by Fleischmann-Struzek C. et al. [20], underscoring the ongoing risk faced by the survivor population.
Currently, few studies elucidate the mechanisms of long-term consequences of sepsis and how to improve health post-sepsis. In the study of Sabri Soussi et al. [21], sepsis survivors were classed into two subtype cohort by clinical and biological data available at the time of ICU discharge using an unsupervised analysis. Those survivors in subtype B, which had more impaired cardiovascular and kidney functions, hematological disorders and inflammation at ICU discharge, had significantly higher one-year mortality (36%). In another study of Sachin Yende et al. [22], two phenotypes of sepsis-survivors’ trajectories were identified using inflammation and immunosuppression biomarkers measured at five time points during and after hospitalization for sepsis for one year, with the results indicating that the hyperinflammation and immunosuppression phenotype was independently associated with higher one-year mortality when compared with the normal phenotype. In this study, our approach leveraged dynamic SOFA component scores within the first few days, capturing a global combination of organ function changes and integrating patient data at ICU admission to develop the DL-CMT model. This model accurately predicted 28-day, 90-day, and one-year mortality for sepsis survivors. This work conforms to the 2018 colloquium on sepsis survivorship sponsored by the International Sepsis Forum suggesting a research road map to include improving research methods (e.g. leveraging multimodal assessment) as one of the central foci among sepsis survivors [6].
To our knowledge, this is the first time that dynamic sequential SOFA scores were subdivided into six subsystem scores and further integrated into construct a deep learning-based prediction model for short to long-term mortality risk assessment in sepsis survivors. The SOFA score, evaluating six organ systems, is a well-established instrument for assessing organ dysfunction and predicting outcomes in critically ill patients [23]. Especially, the Sepsis-3 guidelines used the SOFA score as a measure of disease severity and a mortality risk stratification tool [1]. A SOFA score ≥ 2 reflected an overall mortality risk of approximately 10% in a general hospital population with suspected infection [1, 24]. Previous studies have utilized the SOFA score for predicting sepsis mortality and demonstrated that higher SOFA scores correlated with increased mortality [7, 9, 23, 25]. Houthooft et al. [26] performed a prediction for 5 days mortality of sepsis patients using SOFA variables and achieved an AUROC of 0.82 using the support vector machine model. Taylor et al. [27] predicted 28-day mortality of sepsis patients with an AUROC of 0.86 using random forest. The study from Macdonald et al. [28] proposed XGBoost model for 31-day mortality using 111 clinical and lab variables with an AUROC of 0.84. Additionally, the concept of the quick SOFA (qSOFA) score was introduced as a possible predictive tool for sepsis prognostication [9, 29, 30–31]. Two recent study reported qSOFA used for predicting 28-day mortality in sepsis population, with an AUROC value of 0.642 [32], and a prediction accuracy of 0.70 evaluated using C-statistic [33], respectively. It was also indicated that an increase in SOFA score of 2 or more had greater prognostic accuracy for in-hospital mortality than the qSOFA score [9]. However, these previous SOFA-based predictive tools often relied on static scores measured at a single time point or used the admission SOFA score or the worst SOFA score within the first 24 h to predict mortality. While these models provide valuable insights, the limitations of static SOFA-based models may be evident in their inability to utilize the temporal variations in organ function, thereby missing crucial information that could enhance predictive accuracy. Our study integrates dynamic SOFA subcomponent scores to capture temporal changes in organ dysfunction. This temporal approach provides a more detailed understanding of patient trajectories, particularly during the critical early phase of ICU admission.
In this study, our proposed DL-CMT model addressed these limitations by incorporating dynamic SOFA component scores to capture the temporal progression of each organ dysfunction. The addition of the time-series feature-mapping pathway (each dynamic SOFA component scores) improved the predictive ability of the model compared with models using only the mean and standard deviation of each input feature. As the results shown, the DL-CMT outperformed a MLP, RF, or XGBoost, demonstrating superior AUROC, AUPRC, accuracy, precision, recall, and F1-score values for multiple endpoints. This improvement likely results from the model’s ability to identify patterns in organ function changes over time, reflected by daily SOFA component scoring. A recent study [34] has adopted only SOFA component scores within the first 24 h to develop machine learning models for early predicting in-hospital mortality among sepsis patients, without considering other risk factors for death, the severity of the subject or comorbidities, which did not achieve the ideal predictive ability. For our study, except for the dynamic SOFA component scores, we integrated static patient features such as demographics, comorbidities, vital signs, and lab tests within the first 24 h to reflect the initial admission severity of illness, displaying good performance at multiple endpoint predictions. We believe that this multidimensional approach can enhance the model’s ability to predict outcomes by providing a holistic view of the patient’s health status.
The ablation study showed the impact of removing different SOFA components on the model performance (Table 4). The DL-CMT maintained stable performance when specific physiological component was excluded, underscoring robustness and potential applicability in diverse clinical settings where a specific SOFA component may not be available.
The superior algorithm performance of DL-CMT can be attributed to its ability to handle complex, high-dimensional, and time-dependent data through deep learning techniques. The LSTM network effectively captured temporal dependencies, which were crucial for accurate mortality prediction in sepsis survivors. The LSTM network is particularly well-suited for processing time-series data due to its ability to retain long-term dependencies and manage the vanishing gradient problem inherent in traditional recurrent neural network (RNN) [35]. This capability allowed the DL-CMT to effectively utilize the sequential nature of SOFA scores to predict outcomes over different time horizons. Additionally, the CNN was employed for feature extraction from continuous numerical data, adept at identifying spatial hierarchies by using convolutional layers that applied filters to capture local patterns [36]. This approach enabled the model to learn complex interactions among features, enhancing its predictive power. Combining LSTM and CNN architectures leveraged the strengths of both models to provide a comprehensive understanding of patient data, improving mortality prediction accuracy.
Limitations and future work
Despite the promising results, this study has several limitations. First, this research is a retrospective single-center study based on electronic medical records, which may introduce potential unavoidable selection bias. The dataset used in our study includes patients admitted between 2008 and 2019, which may not fully reflect the impact of more recent advancements in sepsis management and treatment protocol. Additionally, the population used for model training, validation and test was selected from the same large publicly available dataset, limiting the generalizability due to the absence of external validation. We are constructing a prospective study integrating with sepsis patient data collected in our hospital for external model validating to provide meaningful assistance to sepsis survivors.
While this study demonstrates the effectiveness of interval-based mortality predictions using the DL-CMT model, we acknowledge certain methodological limitations. One is the lack of direct comparisons to survival analysis models, such as Random Survival Forest or DeepSurv, which could offer additional insights, especially for time-to-event predictions. In this study, we focused on fixed-interval mortality prediction, which provides clinically actionable predictions for 28-day, 90-day, and 1-year post-discharge mortality. However, we acknowledge that survival models, particularly those that model the timing of death, could complement our approach and provide further granularity to long-term mortality predictions. In future work, we plan to explore survival analysis techniques to incorporate time-to-event prediction alongside our interval-based classification model and provide a more detailed understanding of both who is at risk and when adverse outcomes are likely to occur.
Another methodological limitation is that SHAP for model interpretability has not been employed in our study. Deep learning models, including ours, are often criticized for the “black box” nature. While we recognize that SHAP-based techniques could provide valuable insights, we believe that the current complexity of dynamic SOFA component scores, each with its own clinical indicators and time-dependent relationships, presents a challenge that these methods might not fully address in the context of our model. Therefore, future work needs to explore these techniques and enhance model interpretability, possibly by incorporating more advanced methods tailored for time-series data.
Conclusion
This study highlights the significant risks of post-discharge mortality faced by sepsis survivors. The proposed DL-CMT model, integrating dynamic SOFA component scores with static patient features, demonstrated superior predictive performance for short- and long-term mortality compared to traditional machine learning models such as MLP, RF, and XGBoost. Our approach provides actionable insights to assist clinicians in optimizing post-discharge management and follow-up care for this vulnerable population.
While this study focuses on interval-based predictions, future research could explore the incorporation of survival analysis methods to gain deeper insights into the timing of mortality events. Combining advanced survival techniques with dynamic SOFA scores may enhance the granularity of risk stratification and enable the development of personalized intervention strategies for sepsis survivors, ultimately improving long-term outcomes.
Acknowledgements
None.
Author contributions
JW, FHL, QY and WH conceived the study and model development. JW acquired the data. JW and TJ prepared and cleaned the raw data. DF, SW and QY performed the statistical analyses; JW, XL and FHL prepared the first draft. All authors read and approved the final manuscript.
Funding
This work was supported by the Project of Key Supported Disciplines by Shanghai Municipal Health Commission (2023ZDFC0204), the Research physician talent plan of Shanghai Pulmonary Hospital (LYRC202404), the National Natural Science Foundation of China cultivating program supported by Shanghai Pulmonary Hospital (fkzr25116), the Natural Science Foundation of Zhejiang Province, China (Q24H040021), the Science and Technology Programme of Quzhou, Zhejiang Province, China (2023K105), and the Medical and Healthcare Talent Research Initiation Fund Category G (KYQD 2022–29).
Data availability
MIMIC-IV is a public electronic health record dataset and the datasets presented in the current study are available in the MIMIC-IV database (https://physionet.org/content/mimiciv/2.2/). Nevertheless, it is available from the corresponding author on reasonable request. The data extraction codes can refer to the GitHub website (https://github.com/MIT-LCP/mimic-code).
Declarations
Ethics approval and consent to participate
This study was performed in line with the principles of the Declaration of Helsinki. Since MIMIC-IV only includes anonymized information, patients’ consent to participate was waived at the local institution (Beth Israel Deaconess Medical Center (BIDMC)). Moreover, since this dataset was later made publicly available, the Ethics Committee at Shanghai Pulmonary Hospital did not require further protocol approval.
Consent for publication
Not applicable.
Competing interests
The authors declare no competing interests.
Abbreviations
Sequential organ failure assessment
Intensive care unit
Deep learning
Convolutional neural network
Long short‐term memory
Recurrent neural network
DL-based combined model with multidimension and time series
Random forest
EXtreme gradient boosting
Body mass index
Area under the receiver operator characteristic curve
Area under the precision-recall curve
Multilayer perceptron
Medical information mart for intensive care
Simplified acute physiology score
Glasgow coma scale
Systemic inflammatory response syndrome
Oxford acute severity of illness score
Acute physiology score III
Logistic organ dysfunction score
International normalized ratio
Prothrombin time
Partial thromboplastin time
Saturation of peripheral oxygen
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
1. Singer, M; Deutschman, CS; Seymour, CW; Shankar-Hari, M; Annane, D; Bauer, M et al. The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA; 2016; 315,
2. Rudd, KE; Johnson, SC; Agesa, KM; Shackelford, KA; Tsoi, D; Kievlan, DR et al. Global, regional, and national sepsis incidence and mortality, 1990–2017: analysis for the global burden of disease study. Lancet; 2020; 395,
3. Shankar-Hari, M; Harrison, DA; Rubenfeld, GD; Rowan, K. Epidemiology of sepsis and septic shock in critical care units: comparison between sepsis-2 and sepsis-3 populations using a national critical care database. Br J Anaesth; 2017; 119,
4. Bracke, K; Moya, E; Corcoran, J; Malfait, I; Nutma, I; Hedjal, J et al. Sepsis survivors call for the development of a European sepsis plan. Lancet; 2024; 404,
5. Beane, A; Shankar-Hari, M. Long-term ill health in sepsis survivors: an ignored health-care challenge?. Lancet; 2024; 404,
6. Prescott, HC; Iwashyna, TJ; Blackwood, B; Calandra, T; Chlan, LL; Choong, K et al. Understanding and enhancing sepsis survivorship priorities for research and practice. Am J Respir Crit Care Med; 2019; 200,
7. Vincent, JL; Moreno, R; Takala, J; Willatts, S; De Mendonça, A; Bruining, H et al. The SOFA (sepsis-related organ failure assessment) score to describe organ dysfunction/failure on behalf of the working group on sepsis-related problems of the european society of intensive care medicine. Intens Care Med; 1996; 22,
8. Ferreira, FL; Bota, DP; Bross, A; Mélot, C; Vincent, JL. Serial evaluation of the SOFA score to predict outcome in critically ill patients. JAMA; 2001; 286,
9. Raith, EP; Udy, AA; Bailey, M; McGloughlin, S; MacIsaac, C; Bellomo, R et al. Prognostic accuracy of the SOFA Score, SIRS Criteria, and qSOFA Score for In-hospital mortality among adults with suspected infection admitted to the intensive care unit. JAMA; 2017; 317,
10. Gupta, T; Puskarich, MA; DeVos, E; Javed, A; Smotherman, C; Sterling, SA et al. Sequential organ failure assessment component score prediction of in-hospital mortality from sepsis. J Intensive Care Med; 2020; 35,
11. Hochreiter, S; Schmidhuber, J. Long short-term memory. Neural Comput; 1997; 9,
12. Wiegrebe, S; Kopper, P; Sonabend, R; Bischl, B; Bender, A. Deep learning for survival analysis: a review. Artif Intell Rev; 2024; 57,
13. Johnson, AEW; Bulgarelli, L; Shen, L; Gayles, A; Shammout, A; Horng, S et al. MIMIC-IV, a freely accessible electronic health record dataset. Sci Data; 2023; 10,
14. Prescott, HC; Angus, DC. Enhancing recovery from sepsis: a review. JAMA; 2018; 319,
15. Pandolfi, F; Brun-Buisson, C; Guillemot, D; Watier, L. Care pathways of sepsis survivors: sequelae, mortality and use of healthcare services in France, 2015–2018. Crit Care; 2023; 27,
16. Weycker, D; Akhras, KS; Edelsberg, J; Angus, DC; Oster, G. Long-term mortality and medical care charges in patients with severe sepsis. Crit Care Med; 2003; 31,
17. Braun, L; Riedel, AA; Cooper, LM. Severe sepsis in managed care: analysis of incidence, one-year mortality, and associated costs of care. J Manag Care Pharm; 2004; 10,
18. Shankar-Hari, M; Ambler, M; Mahalingasivam, V; Jones, A; Rowan, K; Rubenfeld, GD. Evidence for a causal link between sepsis and long-term mortality: a systematic review of epidemiologic studies. Crit Care; 2016; 20, 101.
19. Shankar-Hari, M; Harrison, DA; Ferrando-Vivas, P; Rubenfeld, GD; Rowan, K. Risk factors at index hospitalization associated with longer-term mortality in adult sepsis survivors. JAMA Netw Open; 2019; 2,
20. Fleischmann-Struzek, C; Rose, N; Freytag, A; Spoden, M; Prescott, HC; Schettler, A et al. Epidemiology and costs of postsepsis morbidity, nursing care dependency, and mortality in Germany, 2013 to 2017. JAMA Netw Open; 2021; 4,
21. Soussi, S; Sharma, D; Jüni, P; Lebovic, G; Brochard, L; Marshall, JC et al. Identifying clinical subtypes in sepsis-survivors with different one-year outcomes: a secondary latent class analysis of the FROG-ICU cohort. Crit Care; 2022; 26,
22. Yende, S; Kellum, JA; Talisa, VB; Peck Palmer, OM; Chang, CH; Filbin, MR et al. Long-term host immune response trajectories among hospitalized patients with sepsis. JAMA Netw Open; 2019; 2,
23. Seymour, CW; Liu, VX; Iwashyna, TJ; Brunkhorst, FM; Rea, TD; Scherag, A et al. Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA; 2016; 315,
24. Jones, AE; Trzeciak, S; Kline, JA. The sequential organ failure assessment score for predicting outcome in patients with severe sepsis and evidence of hypoperfusion at the time of emergency department presentation. Crit Care Med; 2009; 37,
25. Arnold, RC; Sherwin, R; Shapiro, NI; O'Connor, JL; Glaspey, L; Singh, S et al. Multicenter observational study of the development of progressive organ dysfunction and therapeutic interventions in normotensive sepsis patients in the emergency department. Acad Emerg Med; 2013; 20,
26. Houthooft, R; Ruyssinck, J; van der Herten, J; Stijven, S; Couckuyt, I; Gadeyne, B et al. Predictive modelling of survival and length of stay in critically ill patients using sequential organ failure scores. Artif Intell Med; 2015; 63,
27. Taylor, RA; Pare, JR; Venkatesh, AK; Mowafi, H; Melnick, ER; Fleischman, W et al. Prediction of in-hospital mortality in emergency department patients with sepsis: a local big data-driven. Mach Learn Approach Acad Emerg Med; 2016; 23,
28. Macdonald, SP; Arendts, G; Fatovich, DM; Brown, SG. Comparison of PIRO, SOFA, and MEDS scores for predicting mortality in emergency department patients with severe sepsis and septic shock. Acad Emerg Med; 2014; 21,
29. Do, SN; Dao, CX; Nguyen, TA; Nguyen, MH; Pham, DT; Nguyen, NT et al. Sequential organ failure assessment (SOFA) Score for predicting mortality in patients with sepsis in Vietnamese intensive care units: a multicentre, cross-sectional study. BMJ Open; 2023; 13,
30. Rudd, KE; Seymour, CW; Aluisio, AR; Augustin, ME; Bagenda, DS; Beane, A et al. Association of the quick sequential (sepsis-related) organ failure assessment (qSOFA) score with excess hospital mortality in adults with suspected infection in low- and middle-income countries. JAMA; 2018; 319,
31. Churpek, MM; Snyder, A; Han, X; Sokol, S; Pettit, N; Howell, MD et al. Quick sepsis-related organ failure assessment, systemic inflammatory response syndrome, and early warning scores for detecting clinical deterioration in infected patients outside the intensive care unit. Am J Respir Crit Care Med; 2017; 195,
32. Li, A; Ling, L; Qin, H; Arabi, YM; Myatra, SN; Egi, M et al. Prognostic evaluation of quick sequential organ failure assessment score in ICU patients with sepsis across different income settings. Crit Care; 2024; 28,
33. Blair, PW; Mehta, R; Oppong, CK; Tin, S; Ko, E; Tsalik, EL et al. Screening tools for predicting mortality of adults with suspected sepsis: an international sepsis cohort validation study. BMJ Open; 2023; 13,
34. Pan, X; Xie, J; Zhang, L; Wang, X; Zhang, S; Zhuang, Y et al. Evaluate prognostic accuracy of SOFA component score for mortality among adults with sepsis by machine learning method. BMC Infect Dis; 2023; 23,
35. Du Q, Gu W, Zhang L, Huang S-L, Attention-based LSTM-CNNs For Time-series Classification, in Proceedings of the 16th ACM Conference on Embedded Networked Sensor Systems. 2018, Association for Computing Machinery: Shenzhen, China. 410–1.
36. Kim, P. Convolutional Neural Network. MATLAB Deep Learning: With Machine Learning, Neural Networks and Artificial Intelligence; 2017; Berkeley, Apress: pp. 121-147.
© The Author(s) 2025. This work is published under http://creativecommons.org/licenses/by-nc-nd/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.