Machine Learning-Based Algorithms for Enhanced Prediction of Local Recurrence and Metastasis in Low Rectal Adenocarcinoma Using Imaging, Surgical, and Pathological Data

Abstract

(1) Background: Numerous variables could influence the risk of rectal cancer recurrence or metastasis, and machine learning (ML)-based algorithms can help us refine the risk stratification process of these patients and choose the best therapeutic approach. The aim of this study was to assess the predictive performance of 4 ML-based models for the prediction of local recurrence or distant metastasis in patients with locally advanced low rectal adenocarcinomas who underwent neoadjuvant chemoradiotherapy and surgical treatment; (2) Methods: Patients who were admitted at the first Oncologic Surgical Clinic from the Regional Institute of Oncology, Iasi, Romania were retrospectively included in this study between November 2019 and July 2023. Decision tree (DT), naïve Bayes (NB), support vector machine (SVM), and random forest (RF) were used to analyze imagistic, surgical, and pathological data retrieved from the medical files, and their predictive performance was assessed; (3) Results: The best predictive performance was achieved by RF when used to predict disease recurrence (accuracy: 90.85%) or distant metastasis (accuracy: 89.63%). RF was closely followed by SVM (accuracy for recurrence 87.8%; accuracy for metastasis: 87.2%) in terms of predictive performance. NB and DT achieved moderate predictive power for the evaluated outcomes; (4) Conclusions: Complex algorithms such as RF and SVM could be useful for improving the prediction of adverse oncological outcomes in patients with low rectal adenocarcinoma.

1. Introduction

Colorectal cancer (CRC) ranks third in terms of frequency of diagnosis and mortality among both men and women in the United States [1]. Nevertheless, it holds the second position in terms of overall cancer-related fatalities and is the primary factor for mortality in males under the age of 50 [1]. Over 50% of all cases and deaths can be attributed to modifiable risk factors, including smoking, a poor diet, excessive alcohol use, a lack of physical activity, and obesity [2].

The 5-year relative survival rate for CRC experienced an increase from 50% in the mid-1970s to 65% throughout the period of 2012–2018 [3,4]. The long-term benefits are a result of the early detection of CRC through routine clinical examinations, and today more precise staging is achieved through advancements in imaging techniques [5,6]. Furthermore, improvements in infection control and surgical techniques, as well as advancements in chemotherapy and radiation, have contributed to these long-term gains [7].

Colorectal cancer ranks as the third most commonly reported cancer in males in Romania, following lung cancer and prostate cancer [8]. This type of neoplasia holds the second position in the list of newly diagnosed cancer cases in women, following breast cancer [3]. In Romania, colorectal cancer is the second most common cause of death connected to cancer, resulting in 4302 deaths in 2020 [3].

Accurate appreciation of the risk of local recurrence and distant metastasis is crucial in determining the appropriateness of neoadjuvant or adjuvant therapy. The literature’s data indicated several clinical, imagistic, surgical, and histopathological risk factors for colorectal cancer recurrence and metastasis. One recent meta-analysis conducted by Xu et al. evaluated 34 risk factors for CRC metastasis and 17 for recurrence from 43 observational studies or meta-analyses [9].

Considering the CRC metastasis as an outcome, the authors outlined 12 risk factors that had a significant effect size [9]. These risk factors included lymphovascular invasion, tumor size greater than 1 cm, tumor budding, poor differentiation, muscularis propria invasion, and extramural vascular invasion (mrEMVI) detected on magnetic resonance imaging (MRI). When the authors investigated the impact of several risk factors for CRC local recurrence, they found a significant effect size of perineural invasion and anastomotic leakage [9].

Another meta-analysis investigated the impact of perioperative blood transfusions on colorectal cancer recurrence in patients undergoing curative resection [10]. The results from this study confirmed perioperative blood transfusions to be independent prognostic factors (odds ratio, OR: 1.42, 95% confidence interval, CI: 1.20–1.67) for the evaluated outcome. Moreover, the effect was observed regardless of timing and in a dose-dependent manner.

Gloor et al. conducted a prospective observational study that evaluated the surgical outcomes and recurrence rates in a cohort of patients with low rectal cancer treated by transanal total mesorectal excision [11]. Their results indicated that a positive distal margin and lymph nodes predicted a local recurrence and distant metastasis of CRC.

Moreover, a retrospective study that evaluated the long-term outcomes of patients with rectal adenocarcinoma who underwent curative surgery indicated that a distal margin ≤ 2 cm, extracapsular invasion of lymph node metastasis, tumor stenosis, and parietal invasion were independent risk factors for recurrence [12].

In recent years, artificial intelligence has gained more interest for its applicability in the prediction of disease occurrence, progression, and/or recurrence [13,14,15]. In the field of oncology, and specifically for the prediction of CRC local recurrence or distant metastasis, several machine learning (ML)-based algorithms or artificial neural networks (ANN) have been developed. Many of these models included the above-mentioned clinical, imagistic, surgical, or histopathological predictors.

Jiang and colleagues tested the predictive performance of an MRI deep learning model for the prediction of survival in patients with rectal cancer [16]. The primary variable used in this model was the segmented tumor volumes obtained from prior treatment T2-weighted MRI images. This model achieved an overall good predictive performance for the evaluated outcome and demonstrated good capacity as a risk stratification tool.

Another retrospective study tested the predictive performance of 4 ML-based models for the prediction of recurrence or metastasis of CRC at 1 year, 3 years, and 5 years milestones [17]. These models were represented by recursive feature elimination (RFE), synthetic minority oversampling technique (SMOTE), and support vector machine (SVM) and used clinical-pathological factors, radiomic features, or their combinations, as predictors. Their results indicated that the combined model performed best for the prediction of CRC metastasis or recurrence at 1 year (area under the curve, AUC = 0.887), 3 years (AUC = 0.813), and 5 years (AUC = 0.794) [17].

One recent study by Skrede et al. performed digital image analysis of more than 12 million histopathological tiles using 10 convolutional neural networks and determined a prognostic biomarker with a hazard ratio (HR) of 3.04 (95%CI: 2.07–4.47; p < 0.0001) for the poor prognosis of patients with colorectal cancer [18]. Moreover, Tsai et al. investigated the feasibility of 5 convolutional neural networks for histopathological image classification in patients with colorectal cancers and demonstrated good overall performance [19].

The identification of specific models that achieve the best balance between sensitivity and specificity is a constant challenge, and many efforts should be invested in testing and validation of valuable predictors and their combinations. The aim of this retrospective study was to assess the predictive performance of four ML-based models for the prediction of local recurrence or distant metastasis in patients with locally advanced low rectal adenocarcinomas who underwent neoadjuvant chemoradiotherapy (CRT) and surgical treatment.

2. Materials and Methods

This retrospective observational study was conducted at the first Oncologic Surgical Clinic from the Regional Institute of Oncology, Iasi, Romania, between November 2019 and July 2023. Ethical approval for this study was obtained from the Institutional Ethics Committees of the University of Medicine and Pharmacy ‘Grigore T. Popa’ (No. 23103/23 October 2019) and of the Regional Institute of Oncology (No. 245/3 July 2019).

The inclusion criteria comprised patients diagnosed with low rectal tumors (less than 8 cm from the anal verge), a histopathological examination that indicated adenocarcinoma type, who had a preoperative pelvic MRI for staging, who received neoadjuvant therapy and surgical treatment, as well as those who offered their informed consent for participating in this study.

The exclusion criteria comprised patients with rectal cancer who needed emergency surgery, other subtypes of rectal cancer, loss of follow-up, incomplete medical data, or a lack of informed consent.

The following data was recorded: demographic and clinical characteristics, preoperative MRI parameters, histopathological examination of pre-operative biopsies and post-operative specimens, type of surgery, need for blood transfusions, status of total mesolectal excision (TME), postsurgical evolution (local recurrence or distant metastasis), and survival. All patients underwent pelvic MRI examinations on the SIEMENS MAGNETOM Avanto I-class 1.5 Tesla machine (Siemens Healthcare GmbH, Erlangen, Germany).

The examination protocol included the visualization of the pelvis in all three planes:

• -. Sagittal—this plane is used to locate the tumor and to plan the axial and coronal sequences;

• -. Axial—the plane is angled perpendicular to the tumor to correctly visualize the extension of the tumor against the rectal wall, as well as the distance between the tumor and the mesorectal fascia (MRF);

• -. Coronal—the plane is angled parallel to the axis of the tumor, which is perpendicular to the axial series.

The sequences used are the following: multiplanar T2w and T1w which provided valuable morphological information due to the high resolution of anatomical structures. The T2w sequence was ≤ 3 mm thick. The preoperative parameters included: tumor size, location, distance from the anal verge, extramural venous invasion (EMVI) status (tumor invasion into veins beyond muscularis propria), and MRI CRM (mCRM) positivity (tumor within 1 mm of the mesorectal fascia on the scan) [20,21].

Following the elective surgical procedure, the Regional Oncologic Institute’s oncology-trained pathologists evaluated the intestinal specimens in accordance with standard practice. A positive pathological circumferential resection margin (pCRM) was considered to be a distance of less than 1 mm from the tumor cells to the cut specimen margin. Lymph node positivity was also recorded in this stage, and more than 5 positive lymph nodes were considered the cut-off for risk stratification.

Patients were segregated based on the primary outcomes: local recurrence (group 1, n = 14 patients) and distal metastasis (group 2, n = 24 patients). Patients who did not have local recurrence or distant metastasis were included in the control group (group 3, n = 126 patients).

In the first phase of our analysis, we used descriptive statistics and comparison of categorical variables (Pearson’s χ² test) or continuous variables (analysis of variance, ANOVA, followed by Bonferroni posthoc test) between groups.

In the second stage of the analysis, we used a Cox regression model with the Breslow method for ties to identify imaging, surgical, and pathological predictors for CRC recurrence or metastasis, and quantified their impact as a hazard ratio (HR) and 95% CI. A p-value less than 0.05 was considered statistically significant. These analyses were performed using STATA SE (version 17, 2023, StataCorp LLC, College Station, TX, USA).

In the third stage of the analysis, predictors which had a significant effect on the evaluated outcomes were included in 4 machine-learning-based models: decision tree (DT), naïve Bayes (NB), support vector machine (SVM), and random forest (RF). The database was divided into two sets: 70% for testing and 30% for training. Most machine-learning-based studies choose this configuration, especially for small datasets such as ours [13,14,15,22].

Additionally, a 5-fold cross-validation was performed. The predictive performance of these models was tested for the evaluated outcomes using a sensitivity analysis. The following parameters were reported: sensibility (Se), specificity (Sp), false positive rate (FPR), accuracy, AUC value, Matthews correlation coefficient, and F1 score.

Sensitivity is defined as the number of true positives relative to the total number of sick individuals in the population. It is the probability of a positive test given that the patient is truly positive [23]. Specificity is expressed as the number of true negatives relative to the total number of healthy individuals [23].

FPR was defined as 1- specificity, while accuracy was defined as the sum of true positives and true negatives relative to the total population [24,25]. Mathews coefficient is a correlation coefficient between the observed and predicted binary classifications, that ranges between −1 and +1 [26].

The F1 score was considered the harmonic mean of precision and recall [27]. The models were constructed and analyzed using Matlab (version R2023a, The MathWorks, Inc., Natick, MA, USA).

3. Results

A total of 164 patients with low rectal adenocarcinoma who underwent neoadjuvant therapy and surgical treatment were included in the study, and their clinical and paraclinical characteristics are presented in Table 1.

All examined groups were similar regarding their age (p = 0.88), gender distribution (p = 0.92), medium of living (p = 0.48), BMI (p = 0.96), and smoking habit (p = 0.86), and no statistically significant difference between groups was found considering these characteristics.

On the other hand, the univariate analysis indicated that all imagistic parameters were significantly different between groups (p < 0.001). Thus, patients who later developed local recurrence or CRC metastasis presented with significantly higher rates of mCRM and EMVI positivity, as well as a higher incidence of anteriorly located tumors at a distance of less than 4 cm from the anal verge.

Moreover, the proportion of Hartmann procedures and pelvic exenterations was significantly higher for patients from the first and second groups compared with controls (p = 0.04). The same groups experienced significantly higher rates of incomplete TME (p= 0.005) and blood transfusions (p < 0.001) compared with controls.

Regarding histopathological characteristics, both a high number of positive lymph nodes and positive pCRM were significantly more frequently encountered in the groups of patients who later developed local recurrence or CRC metastasis (p < 0.001).

Table 2 comprises the results from the Cox regression model, which used the local recurrence of CRC as the outcome, and significant parameters in the univariate analysis as predictors.

A positive pCRM (HR: 53.33, 95%CI: 11.37–249.97, p < 0.001), mrEMVI (HR: 28.56, 95%CI: 6.15–132.62, p < 0.001), and more than 5 positive lymph nodes (HR: 13.68, 95%CI: 3.98–46.97, p < 0.001) demonstrated the highest positive impact on the occurrence of CRC local recurrence.

Also, mCRM (p = 0.002), the anterior location of the tumor (p = 0.005), and an incomplete TME (p = 0.022) had a significant impact on the evaluated outcome’s occurrence, even though it was more reduced compared to the previous predictors.

On the other hand, a distance of less than 4 cm from the anal verge (p = 0.230) and the need for blood transfusions (p = 0.078) did not appear to have a significant positive impact on the evaluated outcome.

Table 3 comprises the results from a Cox regression model that used the distant metastasis of CRC as the outcome and significant parameters in the univariate analysis as predictors.

Our model showed that more than 5 positive lymph nodes (HR: 48.02, 95%CI: 16.74–137.70, p < 0.001), mrEMVI (HR: 23.76, 95%CI: 8.98–62.88, p < 0.001), mCRM (HR: 21.22, 95%CI: 7.06–63.77, p < 0.001), the anterior location of the tumor (HR: 13.40, 95%CI: 5.12–35.07, p < 0.001), a distance of less than 4 cm from the anal verge (HR: 7.43, 95%CI: 2.48–22.24, p < 0.001), and a positive pCRM (HR: 15.41, 95%CI: 6.26–37.93, p < 0.001) significantly increased the risk of CRC distant metastasis.

Also, incomplete TME (p = 0.002) and the need for blood transfusions (p = 0.026) had a smaller, but significant, impact on the risk of CRC metastasis.

Significant predictors from the Cox hazard regression were included in 4 machine-learning-based algorithms, and their predictive performance for CRC local recurrence and distant metastasis was calculated (Table 4). A flowchart with the study methodology is presented in Figure 1. Comparisons of the ROC curves corresponding to the evaluated models and outcomes are presented in Figure 2 and Figure 3.

RF achieved the best results in terms of prediction of CRC local recurrence: Se- 85.71%, Sp- 91.33%, FPR- 8%, and accuracy of 90.85%. This type of algorithm also best predicted CRC distant metastasis, with a Se of 87.5%, Sp of 90%, FPR of 1%, and an accuracy of 89.63%. RF was closely followed by SVM (accuracy for recurrence 87.8%; accuracy for metastasis: 87.2%) in terms of predictive performance. NB and DT achieved moderate predictive power for the evaluated outcomes.

The ROC comparisons outlined RF and SVM models as having the highest AUC value for the prediction of CRC recurrence (AUC: 0.885/AUC: 0.803) and distal metastasis (AUC: 0.887/AUC: 0.838) (Figure 2 and Figure 3). NB performed better than DT for prediction of the evaluated outcomes: AUC values for recurrence—0.715 versus 0.611, and AUC values for metastasis—0.761 versus 0.678.

4. Discussion

Regular screening and prompt diagnostics of colorectal cancer have reduced the incidence of tumors in the early stages. Moreover, it was observed that in 2019, 60% of newly reported cases were classified as advanced, compared to 52% in the mid-2000s and 57% in 1995, prior to the implementation of universal screening [28]. Additionally, there has been a trend toward an increase in left-sided tumors, including rectal cancer, which accounted for 27% of cases in 1995 and rose to 31% in 2019 [28]. This epidemiological context outlines the need to identify the best prognostic factors that are associated with a high risk of local recurrence or distant metastasis from locally advanced rectal cancers.

This study was focused on patients diagnosed with low rectal adenocarcinomas, which is the most frequent type of rectal cancer (accounting for approximately 96% of all CRCs.) [29]. Specifically, we identified risk factors that increase the odds of rectal cancer local recurrence or distant metastasis in this cohort of patients, and we included the significant predictors in four machine-learning models whose predictive performance was determined.

Our results indicated that both local recurrence and distant metastasis shared common significant predictors such as positivity for pCRM, mCRM, and mrEMVI, as well as more than 5 positive lymph nodes, an anterior location of the tumor, and an incomplete TME. Additionally, the following predictors increased the risk of distant metastasis: a distance of less than 4 cm from the anal verge, and the need for blood transfusions.

These predictors were also identified in the literature to increase the risk of adverse outcomes recurrence. For example, Ma et al. found in a prospective study on a cohort of 209 patients diagnosed with rectal cancer who underwent tumor resection that mCRM was an independent risk factor for local recurrence, with an HR of 3.49 (p = 0.003) [30]. Currently, there is no evidence supporting the utilization of re-radiation for a positive pCRM after undergoing preoperative radiotherapy and TME surgery. The presence of a positive circumferential resection margin following preoperative radiotherapy and surgery creates ambiguity regarding the function of post-operative radiotherapy [31]. Therefore, our center does not commonly practice it due to the heightened toxicity and uncertain survival benefits.

Another study investigated the influence of a pCRM in 3196 individuals diagnosed with rectal cancer who underwent complete TME, with or without radiotherapy [32]. Their results indicated that a positive mCRM was associated with an increased risk of tumor recurrence (HR: 4.18, 95%CI: 2.48–7.05) and distant metastasis (HR: 2.81, 95%CI: 1.93–4.09) in patients who underwent only TME. Moreover, the same parameter was also associated with an increased risk of local recurrence (HR: 0.53, 95%CI: 0.30–0.90) and distal metastasis (HR: 1.22, 95%CI: 0.86–1.61) in patients who underwent TME and radiotherapy [32].

A recent meta-analysis was conducted to assess the prognostic significance of magnetic resonance extramural vascular invasion (mrEMVI) in predicting unfavorable oncologic outcomes in patients who had neoadjuvant therapy followed by total mesorectal excision [33]. The results of this meta-analysis confirmed mrEMVI as an independent prognostic factor for recurrence, metastasis, and decreased disease-free survival. Moreover, the GEMCAD 0801 trial investigated the prognostic value of mrENVI and other predictors for adverse oncological outcomes in patients who underwent primary chemotherapy for rectal cancer [34]. A positive mrEMVI was associated with an increased risk of local recurrence in a 3-year time frame (HR: 9.220, 95%CI: 0.802–105.965).

Lymph node positivity was considered an important independent risk factor for CRC recurrence or metastasis, but the cut-off differs between studies. Peng et al. conducted a retrospective analysis on the prognostic value of LNR (ratio of metastatic to retrieved lymph nodes) in patients with node-positive rectal cancer, who were treated with curative anterior resection, over a 14-year period [35]. Their results indicated that the five-year local recurrence rate was significantly higher in patients with an LNR between 0.14 and 1 (3.6% in LNR < 0.14 versus 15.6% in LNR 0.14–1, p = 0.019). Another retrospective study aimed to identify the prognostic significance of lateral node metastases in patients with stage 3 or pT4 low rectal adenocarcinoma [36]. The results from this study indicated that patients with lateral node metastases had a significantly shorter postoperative survival (5-year survival rate of 42 versus 71.6%; p < 0·001) and an increased risk of local recurrence (44 versus 11.7%; p < 0·001) in comparison with controls.

Finally, the anterior location of the tumor [37,38], an incomplete TME [39,40], a distance of less than 4 cm from the anal verge [41,42], and the need for blood transfusions [43,44] were also cited in the literature as independent prognostic factors for the local recurrence and/or distant metastasis of rectal cancer. Thus, in the second stage of the analysis, all these significant predictors were included in the following ML-based models: DT, NB, SVM, and RF, whose predictive performance was calculated.

Our results indicated that the best predictive performance was achieved by RF when used to predict low rectal adenocarcinoma recurrence or distant metastasis, with corresponding accuracies of 90.85% and 89.63%. RF was closely followed by SVM (accuracy for recurrence: 87.8%; accuracy for metastasis: 87.2%) in terms of predictive performance. NB and DT achieved moderate predictive power for the evaluated outcomes. These results are in line with previously published literature, although data is scarce and variable when considering specific machine learning-based algorithms.

Jeon et al. investigated the predictive performance of 4 ML-based algorithms (logistic regression, LR, SVM, RF, and extreme gradient boosting, XGBoost) based on clinical and paraclinical data for the prediction of rectal cancer recurrence after curative resection [45]. Their results indicated that SVM achieved the best predictive performance (AUC: 0.831, Se: 69.2%, Sp: 81.4%, and accuracy: 79.8%) for the prediction of the evaluated outcome. RF, which achieved a Se of 73.1%, a Sp of 80.2%, an accuracy of 79.3%, and an AUC value of 0.826, closely followed this algorithm [45]. In contrast, the lowest AUC value was obtained for the XGBoost method (0.804), with a sensitivity, specificity, and accuracy of 30.8%, 92.8%, and 84.5%, respectively.

Xu et al. investigated the predictive performance of 4 ML-based algorithms (LR, DT, GradientBoosting, and lightGBM) for the prediction of the postoperative recurrence risk in patients diagnosed with stage IV colorectal cancer [46]. Their results indicated that the GradientBoosting model (AUC: 0.734) and the lightGBM model (F1_score: 0.974) performed better than LR (AUC: 0.692) and DT in the testing phase. On the other hand, a study that included only radiomic features in SVM or LR models indicated that LR models achieve superior predictive performance (accuracy: 80%, Se: 83%, Sp: 76%, and AUC: 0.87) [47].

Limitations and Future Work Recommendations

The results from this study should be interpreted considering the following limitations: the small cohort of patients, the limited time frame for patients’ follow-up, and the small number of parameters evaluated. On the other hand, this study has the advantage of testing various ML algorithms for the prediction of adverse oncological outcomes in low rectal cancer patients.

We hypothesize that further studies, on a larger cohort of patients, could include several panels of clinical and paraclinical parameters in several machine learning algorithms or convolutional neural networks in order to better establish their predictive performance. These approaches allow better image segmentation or feature discrimination, and allow the analysis of a large dataset, even with high rates of missing data [48,49].

Moreover, some studies have outlined the potential of deep learning techniques to include a wide range of parametric and non-parametric data and to create a bridge between multiple areas of research, with practical implications in the treatment planning of oncologic patients [50,51].

This paper outlined several clinical and paraclinical risk factors that were associated with an increased risk of rectal adenocarcinoma recurrence or metastasis. These risk factors could be further included in various risk stratification algorithms, either classical or based on machine learning.

Moreover, we tested four supervised machine learning algorithms, which included these predictors, for their predictive power. Their performance indicated the need for further refinement and/or inclusion of additional data, as their accuracies ranged between 76.22 and 90.85%.

This study tested the predictive performance of machine learning algorithms on a small cohort of patients. This study cohort had the particularity of including locally advanced rectal adenocarcinomas who underwent neoadjuvant therapy and surgical treatment, and this could be an explanation for the small number of cases included. The number of cases included in studies that evaluated the predictive performance of machine learning algorithms for rectal cancer recurrence or metastasis is variable, depending on numerous factors.

For example, Sluckin et al. investigated the predictive performance of a deep learning model based on imaging and clinical features in 196 patients diagnosed with rectal cancer from three tertiary centers in the Netherlands [52]. The authors showed that their model achieved an AUC value of 0.78 for the prediction of lateral recurrence and 0.80 for the prediction of ipsi-lateral local recurrences.

Another study evaluated the performance of deep learning radiomics for the prediction of distant metastasis in 235 patients with locally advanced rectal cancer from three hospitals and demonstrated that their model achieved an AUC value of 0.894 in the validation phase [53].

Moreover, Liang et al. tested the predictive performance of two machine learning algorithms, SVM and logistic regression, for the prediction of metachronous liver metastases in a cohort of 108 patients diagnosed with rectal cancer, and demonstrated the superior performance of the logistic regression algorithm, which achieved an AUC value of 0.87 for the evaluated outcome [47].

Machine learning-based algorithms and neural networks are increasingly used in the medical field, and we hypothesize that their inclusion in clinical practice could serve as useful instruments for risk stratification and prognosis in various clinical scenarios.

This study outlines the efficacy of machine learning algorithms that operate with clinical and paraclinical data for the prediction of rectal adenocarcinoma local recurrence and distant metastasis in a cohort of patients from Romania. We hypothesize that further prospective internal and external validation of our models could aid the update of our oncological and surgical perspective regarding the profile of high-risk patients for adverse oncological outcomes and prognosis.

5. Conclusions

Machine learning is a complex scientific field that has gained a lot of interest in personalized medicine. Finding the best formula for disease recurrence or metastasis prediction is a constant challenge, and numerous predictors should be carefully evaluated for their prognostic value before including them in specific formulas.

This retrospective study indicated that several clinical and paraclinical predictors have a significant impact on rectal cancer local recurrence or metastasis, and their inclusion in 4 ML-based algorithms indicated overall good predictive performance.

The highest predictive performance for the evaluated outcomes was achieved by RF, which was closely followed by SVM. NB and DT achieved moderate predictive power for the evaluated outcomes.

The inclusion of machine learning algorithms in clinical practice could serve as useful instruments for risk stratification and prognosis in various clinical scenarios.

Author Contributions

Conceptualization, C.-C.V., D.-V.S., D.B., S.-R.V., C.U.-L. and V.S.; methodology, I.-A.V., L.G., L.-M.B. and C.V.; software, I.-A.V.; validation, I.-A.V., L.G., L.-M.B. and C.V.; formal analysis, I.-A.V., L.G., L.-M.B. and C.V.; investigation, C.-C.V., D.-V.S., D.B., S.-R.V., C.U.-L. and V.S.; resources, I.-A.V., L.G., L.-M.B. and C.V.; data curation, I.-A.V., L.G., L.-M.B. and C.V.; writing—original draft preparation, C.-C.V., D.-V.S., D.B., S.-R.V., C.U.-L. and V.S.; writing—review and editing C.-C.V., D.-V.S., D.B., S.-R.V., C.U.-L. and V.S.; visualization, C.-C.V.; supervision, V.S.; project administration, V.S. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Ethics Committee of the University of Medicine and Pharmacy ‘Grigore T. Popa’ (No. 23103/23 October 2019) and of the Regional Institute of Oncology (No 245/3 July 2019).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The datasets are available from the correspondent authors upon a reasonable request due to local policies.

Conflicts of Interest

The authors declare no conflicts of interest.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Figure 1. Flowchart with the study methodology.

Figure 2. Comparison of ROC curves corresponding to 4 models used for the prediction of CRC local recurrence.

Figure 3. Comparison of ROC curves corresponding to 4 models used for the prediction of CRC distant metastasis.

Table 1

Demographic and clinical characteristics of the evaluated groups.

Variable	Local Recurrence (Group 1, n = 14 Patients)	Distant Metastasis (Group 2, n = 24 Patients)	Control Group (Group 3, n = 126 Patients)	p Value
Age, years (mean ± SD)	68.57 ± 8.77	65.87 ± 9.24	63.06 ± 9.26	0.88
Gender (n/%)	Male = 9 (64.28%)Female = 5 (35.71%)	Male = 14 (58.3%)Female = 10 (41.66%)	Male = 78 (61.90%)Female = 48 (38.09%)	0.92
Medium (n/%)	Urban = 8 (57.14%)Rural = 6 (42.85%)	Urban = 10 (41.66%)Rural = 14 (58.33%)	Urban = 51 (40.47%)Rural = 75 (59.52%)	0.48
BMI, kg/m2 (mean ± SD)	25.76 ± 4.31	25.28 ± 3.69	24.82 ± 4.12	0.96
Smoking (n/%)	Yes = 3 (21.42%)	Yes = 4 (16.66%)	Yes = 20 (15.87%)	0.86
Positive mCRM (n/%)	Yes = 9 (64.29%)	Yes = 19 (79.17%)	Yes = 21 (16.66%)	<0.001
Positive mrEMVI (n/%)	Yes = 12 (85.71%)	Yes = 20 (83.33%)	Yes = 12 (9.52%)	<0.001
Anterior location (n/%)	Yes = 8 (57.14%)	Yes = 17 (70.83%)	Yes = 11 (8.73%)	<0.001
Distance from anal verge (n/%)	<4 cm = 7 (50%)>4 cm = 7 (50%)	<4 cm = 19 (79.17%)>4 cm = 5 (20.83%)	<4 cm = 35 (27.77%)>4 cm = 91 (72.22%)	<0.001
Type of surgery (n/%)	AP resection = 8 (57.14%)Hartmann procedure = 3 (21.42%)Anterior resection = 1 (7.14%)Pelvic exenteration = 2 (14.28%)	AP resection = 15 (62.5%)Hartmann procedure = 6 (25%)Anterior resection = 1 (4.16%)Pelvic exenteration = 2 (7.69%)	AP resection = 98 (77.77%)Hartmann procedure = 19 (15.07%)Anterior resection = 8 (6.34%)Pelvic exenteration = 1 (0.79%)	0.04
Incomplete TME (n/%)	Yes = 6 (42.85%)	Yes = 9 (37.5%)	Yes = 15 (11.90%)	0.0005
Need for blood transfusions (n/%)	Yes = 5 (35.71%)	Yes = 13 (54.16%)	Yes = 16 (12.69%)	<0.001
More than 5 positive lymph nodes (n/%)	Yes = 9 (64.29%)	Yes = 18 (75%)	Yes = 13 (10.31%)	<0.001
Positive pCRM (n/%)	Yes = 11 (78.57%)	Yes = 14 (58.33%)	Yes = 6 (4.76%)	<0.001

Table legend: pCRM—pathological (positive) circumferential resection margin; mCRM—positive circumferential resection margin on MRI; mrEMVI—extramural venous invasion on MRI; SD—standard deviation; BMI—body mass index; AP—abdominoperineal resection; TME—total mesorectal excision.

Table 2

Results from the Cox regression model for evaluating the impact of imagistic, surgical, and histopathological predictors on the CRC local recurrence.

Predictors	HR	Standard Error	95% CI (Lower Bound and Upper Bound)	p Value
mCRM	7.26	4.55	2.12–24.85	0.002
mrEMVI	28.56	22.37	6.15–132.62	<0.001
Anterior location	5.55	3.36	1.69–18.23	0.005
Less than 4 cm from anal verge	2.06	1.25	0.63–6.77	0.230
Incomplete TME	4.21	2.64	1.23–14.42	0.022
Need for blood transfusions	3.29	2.23	0.87–12.43	0.078
More than 5 positive lymph nodes	13.68	8.61	3.98–46.97	<0.001
Positive pCRM	53.33	42.03	11.37–249.97	<0.001

Table legend: HR—hazard ratio; CI—confidence interval; mCRM—positive circumferential resection margin on MRI; mrEMVI—extramural venous invasion on MRI; pCRM—pathological circumferential resection margin; TME—total mesorectal excision.

Table 3

Results from the Cox regression model for evaluating the impact of imagistic, surgical, and histopathological predictors on CRC distal metastasis.

Predictors	HR	Standard Error	95% CI (Lower Bound and Upper Bound)	p Value
mCRM	21.22	11.91	7.06–63.77	<0.001
mrEMVI	23.76	11.79	8.98–62.88	<0.001
Anterior location	13.40	6.57	5.12–35.07	<0.001
Less than 4 cm from anal verge	7.43	4.15	2.48–22.24	<0.001
Incomplete TME	4.32	2.03	1.71–10.86	0.002
Need for blood transfusions	3.16	1.63	1.14–8.72	0.026
More than 5 positive lymph nodes	48.02	25.81	16.74–137.70	<0.001
Positive pCRM	15.41	7.08	6.26–37.93	<0.001

Table legend: HR—hazard ratio; CI—confidence interval; mCRM—positive circumferential resection margin on MRI; mrEMVI—extramural venous invasion on MRI; pCRM—pathological circumferential resection margin; TME—total mesorectal excision.

Table 4

Predictive performance of ML-based algorithms for the prediction of CRC local recurrence and distal metastasis.

ML Model	Groups	Se (%)	SP (%)	FPR (%)	Matthews Coefficient	Accuracy (%)	AUC Value	F1 Score
DT	Recurrence (14 patients)	42.86	79.33	20.66	0.14	76.22	0.611	0.23
	Metastasis (24 patients)	54.17	81.43	18.57	0.29	77.4	0.678	0.41
NB	Recurrence (14 patients)	57.14	86	14	0.31	83.54	0.715	0.37
	Metastasis (24 patients)	66.67	85.71	14.2	0.44	82.93	0.761	0.53
SVM	Recurrence (14 patients)	71.43	89.33	10.6	0.46	87.8	0.803	0.5
	Metastasis (24 patients)	79.17	88.57	11.42	0.58	87.2	0.838	0.64
RF	Recurrence (14 patients)	85.71	91.33	8	0.59	90.85	0.885	0.61
	Metastasis (24 patients)	87.5	90	1	0.66	89.63	0.887	0.71

Table legend: DT—decision trees, NB—naïve Bayes, SVM—support vector machine, RF—random forest, Se—sensibility; SP—specificity, FPR—False positive rate; AUC—area under the curve.

References

1. Sung, H.; Ferlay, J.; Siegel, R.L.; Laversanne, M.; Soerjomataram, I.; Jemal, A.; Bray, F. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA Cancer J. Clin.; 2021; 71, pp. 209-249. [DOI: https://dx.doi.org/10.3322/caac.21660] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33538338]

2. Islami, F.; Goding Sauer, A.; Miller, K.D.; Siegel, R.L.; Fedewa, S.A.; Jacobs, E.J.; McCullough, M.L.; Patel, A.V.; Ma, J.; Soerjomataram, I. Proportion and number of cancer cases and deaths attributable to potentially modifiable risk factors in the United States. CA Cancer J. Clin.; 2018; 68, pp. 31-54. [DOI: https://dx.doi.org/10.3322/caac.21440]

3. Chu, K.C.; Tarone, R.E.; Chow, W.H.; Hankey, B.F.; Ries, L.A. Temporal patterns in colorectal cancer incidence, survival, and mortality from 1950 through 1990. J. Natl. Cancer Inst.; 1994; 86, pp. 997-1006. [DOI: https://dx.doi.org/10.1093/jnci/86.13.997] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/7980765]

4. Siegel, R.L.; Miller, K.D.; Wagle, N.S.; Jemal, A. Cancer statistics, 2023. CA Cancer J. Clin.; 2023; 73, pp. 17-48. [DOI: https://dx.doi.org/10.3322/caac.21763]

5. Gross, C.P.; Andersen, M.S.; Krumholz, H.M.; McAvay, G.J.; Proctor, D.; Tinetti, M.E. Relation between Medicare screening reimbursement and stage at diagnosis for older patients with colon cancer. JAMA; 2006; 296, pp. 2815-2822. [DOI: https://dx.doi.org/10.1001/jama.296.23.2815] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/17179458]

6. Lee, D.H.; Lee, J.M. Whole-body PET/MRI for colorectal cancer staging: Is it the way forward?. J. Magn. Reson. Imaging; 2017; 45, pp. 21-35. [DOI: https://dx.doi.org/10.1002/jmri.25337]

7. Murphy, C.C.; Harlan, L.C.; Lund, J.L.; Lynch, C.F.; Geiger, A.M. Patterns of colorectal cancer care in the United States: 1990–2010. J. Natl. Cancer Inst.; 2015; 107, djv198. [DOI: https://dx.doi.org/10.1093/jnci/djv198]

8. Globocan. 2020; Available online: https://gco.iarc.fr/today/data/factsheets/populations/642-romania-fact-sheets.pdf (accessed on 19 December 2023).

9. Xu, W.; He, Y.; Wang, Y.; Li, X.; Young, J.; Ioannidis, J.P.A.; Dunlop, M.G.; Theodoratou, E. Risk factors and risk prediction models for colorectal cancer metastasis and recurrence: An umbrella review of systematic reviews and meta-analyses of observational studies. BMC Med.; 2020; 18, 172. [DOI: https://dx.doi.org/10.1186/s12916-020-01618-6]

10. Amato, A.; Pescatori, M. Perioperative blood transfusions for the recurrence of colorectal cancer. Cochrane Database Syst. Rev.; 2006; 2006, Cd005033. [DOI: https://dx.doi.org/10.1002/14651858.CD005033.pub2]

11. Gloor, S.; Pozza, G.; Troller, R.; Wehrli, M.; Adamina, M. Surgical Outcomes, Long-Term Recurrence Rate, and Resource Utilization in a Prospective Cohort of 165 Patients Treated by Transanal Total Mesorectal Excision for Distal Rectal Cancer. Cancers; 2023; 15, 1190. [DOI: https://dx.doi.org/10.3390/cancers15041190]

12. Farhat, W.; Azzaza, M.; Mizouni, A.; Ammar, H.; Ben Ltaifa, M.; Lagha, S.; Kahloul, M.; Gupta, R.; Mabrouk, M.B.; Ali, A.B. Factors predicting recurrence after curative resection for rectal cancer: A 16-year study. World J. Surg. Oncol.; 2019; 17, 173. [DOI: https://dx.doi.org/10.1186/s12957-019-1718-1]

13. Harabor, V.; Mogos, R.; Nechita, A.; Adam, A.-M.; Adam, G.; Melinte-Popescu, A.-S.; Melinte-Popescu, M.; Stuparu-Cretu, M.; Vasilache, I.-A.; Mihalceanu, E. Machine Learning Approaches for the Prediction of Hepatitis B and C Seropositivity. Int. J. Environ. Res. Public Health; 2023; 20, 2380. [DOI: https://dx.doi.org/10.3390/ijerph20032380]

14. Melinte-Popescu, M.; Vasilache, I.A.; Socolov, D.; Melinte-Popescu, A.S. Prediction of HELLP Syndrome Severity Using Machine Learning Algorithms—Results from a Retrospective Study. Diagnostics; 2023; 13, 287. [DOI: https://dx.doi.org/10.3390/diagnostics13020287]

15. Melinte-Popescu, A.-S.; Vasilache, I.-A.; Socolov, D.; Melinte-Popescu, M. Predictive Performance of Machine Learning-Based Methods for the Prediction of Preeclampsia—A Prospective Study. J. Clin. Med.; 2023; 12, 418. [DOI: https://dx.doi.org/10.3390/jcm12020418]

16. Jiang, X.; Zhao, H.; Saldanha, O.L.; Nebelung, S.; Kuhl, C.; Amygdalos, I.; Lang, S.A.; Wu, X.; Meng, X.; Truhn, D. et al. An MRI Deep Learning Model Predicts Outcome in Rectal Cancer. Radiology; 2023; 307, e222223. [DOI: https://dx.doi.org/10.1148/radiol.222223]

17. Liu, Z.; Wang, Y.; Shen, F.; Zhang, Z.; Gong, J.; Fu, C.; Shen, C.; Li, R.; Jing, G.; Cai, S. et al. Radiomics based on readout-segmented echo-planar imaging (RS-EPI) diffusion-weighted imaging (DWI) for prognostic risk stratification of patients with rectal cancer: A two-centre, machine learning study using the framework of predictive, preventive, and personalized medicine. Epma J.; 2022; 13, pp. 633-647. [DOI: https://dx.doi.org/10.1007/s13167-022-00303-3]

18. Skrede, O.J.; De Raedt, S.; Kleppe, A.; Hveem, T.S.; Liestøl, K.; Maddison, J.; Askautrud, H.A.; Pradhan, M.; Nesheim, J.A.; Albregtsen, F. et al. Deep learning for prediction of colorectal cancer outcome: A discovery and validation study. Lancet; 2020; 395, pp. 350-360. [DOI: https://dx.doi.org/10.1016/S0140-6736(19)32998-8] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32007170]

19. Tsai, M.-J.; Tao, Y.-H. Deep Learning Techniques for the Classification of Colorectal Cancer Tissue. Electronics; 2021; 10, 1662. [DOI: https://dx.doi.org/10.3390/electronics10141662]

20. Diagnostic accuracy of preoperative magnetic resonance imaging in predicting curative resection of rectal cancer: Prospective observational study. BMJ; 2006; 333, 779. [DOI: https://dx.doi.org/10.1136/bmj.38937.646400.55] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/16984925]

21. Volovat, C.C.; Scripcariu, D.V.; Boboc, D.; Volovat, S.R.; Vasilache, I.A.; Lupascu-Ursulescu, C.; Gheorghe, L.; Baean, L.M.; Volovat, C.; Scripcariu, V. Predicting the Feasibility of Curative Resection in Low Rectal Cancer: Insights from a Prospective Observational Study on Preoperative Magnetic Resonance Imaging Accuracy. Medicina; 2024; 60, 330. [DOI: https://dx.doi.org/10.3390/medicina60020330]

22. Vasilache, I.A.; Scripcariu, I.S.; Doroftei, B.; Bernad, R.L.; Cărăuleanu, A.; Socolov, D.; Melinte-Popescu, A.S.; Vicoveanu, P.; Harabor, V.; Mihalceanu, E. et al. Prediction of Intrauterine Growth Restriction and Preeclampsia Using Machine Learning-Based Algorithms: A Prospective Study. Diagnostics; 2024; 14, 453. [DOI: https://dx.doi.org/10.3390/diagnostics14040453]

23. Altman, D.G.; Bland, J.M. Diagnostic tests. 1: Sensitivity and specificity. BMJ; 1994; 308, 1552. [DOI: https://dx.doi.org/10.1136/bmj.308.6943.1552]

24. Burke, D.S.; Brundage, J.F.; Redfield, R.R.; Damato, J.J.; Schable, C.A.; Putman, P.; Visintine, R.; Kim, H.I. Measurement of the False Positive Rate in a Screening Program for Human Immunodeficiency Virus Infections. N. Engl. J. Med.; 1988; 319, pp. 961-964. [DOI: https://dx.doi.org/10.1056/NEJM198810133191501]

25. Metz, C.E. Basic principles of ROC analysis. Semin. Nucl. Med.; 1978; 8, pp. 283-298. [DOI: https://dx.doi.org/10.1016/S0001-2998(78)80014-2] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/112681]

26. Chicco, D.; Tötsch, N.; Jurman, G. The Matthews correlation coefficient (MCC) is more reliable than balanced accuracy, bookmaker informedness, and markedness in two-class confusion matrix evaluation. BioData Min.; 2021; 14, 13. [DOI: https://dx.doi.org/10.1186/s13040-021-00244-z] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33541410]

27. Taha, A.A.; Hanbury, A. Metrics for evaluating 3D medical image segmentation: Analysis, selection, and tool. BMC Med. Imaging; 2015; 15, 29. [DOI: https://dx.doi.org/10.1186/s12880-015-0068-x] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/26263899]

28. Siegel, R.L.; Wagle, N.S.; Cercek, A.; Smith, R.A.; Jemal, A. Colorectal cancer statistics, 2023. CA Cancer J. Clin.; 2023; 73, pp. 233-254. [DOI: https://dx.doi.org/10.3322/caac.21772] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/36856579]

29. Stewart, S.L.; Wike, J.M.; Kato, I.; Lewis, D.R.; Michaud, F. A population-based study of colorectal cancer histology in the United States, 1998–2001. Cancer; 2006; 107, pp. 1128-1141. [DOI: https://dx.doi.org/10.1002/cncr.22010] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/16802325]

30. Ma, X.; Li, X.; Xu, L.; Shi, D.; Tong, T.; Huang, D.; Ding, Y.; Cai, S.; Peng, J. Characteristics and Prognostic Significance of Preoperative Magnetic Resonance Imaging-Assessed Circumferential Margin in Rectal Cancer. Gastroenterol. Res. Pract.; 2015; 2015, 410150. [DOI: https://dx.doi.org/10.1155/2015/410150] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/26089866]

31. Kim, J.H. Controversial issues in radiotherapy for rectal cancer: A systematic review. Radiat. Oncol. J.; 2017; 35, 295. [DOI: https://dx.doi.org/10.3857/roj.2017.00395]

32. Bernstein, T.E.; Endreseth, B.H.; Romundstad, P.; Wibe, A. Circumferential resection margin as a prognostic factor in rectal cancer. Br. J. Surg.; 2009; 96, pp. 1348-1357. [DOI: https://dx.doi.org/10.1002/bjs.6739]

33. Rouleau Fournier, F.; Motamedi, M.A.K.; Brown, C.J.; Phang, T.; Raval, M.J.; Hague, C.J.; Karimuddin, A.A. Oncologic Outcomes Associated With MRI-detected Extramural Venous Invasion (mrEMVI) in Rectal Cancer: A Systematic Review and Meta-analysis. Ann. Surg.; 2022; 275, pp. 303-314. [DOI: https://dx.doi.org/10.1097/SLA.0000000000004636]

34. Patel, U.B.; Brown, G.; Machado, I.; Santos-Cores, J.; Pericay, C.; Ballesteros, E.; Salud, A.; Isabel-Gil, M.; Montagut, C.; Maurel, J. et al. MRI assessment and outcomes in patients receiving neoadjuvant chemotherapy only for primary rectal cancer: Long-term results from the GEMCAD 0801 trial. Ann. Oncol.; 2017; 28, pp. 344-353. [DOI: https://dx.doi.org/10.1093/annonc/mdw616]

35. Peng, J.; Xu, Y.; Guan, Z.; Zhu, J.; Wang, M.; Cai, G.; Sheng, W.; Cai, S. Prognostic Significance of the Metastatic Lymph Node Ratio in Node-Positive Rectal Cancer. Ann. Surg. Oncol.; 2008; 15, pp. 3118-3123. [DOI: https://dx.doi.org/10.1245/s10434-008-0123-8]

36. Ueno, M.; Oya, M.; Azekura, K.; Yamaguchi, T.; Muto, T. Incidence and prognostic significance of lateral lymph node metastasis in patients with advanced low rectal cancer. Br. J. Surg.; 2005; 92, pp. 756-763. [DOI: https://dx.doi.org/10.1002/bjs.4975]

37. Chan, C.; Bokey, E.; Chapuis, P.; Renwick, A.; Dent, O. Local recurrence after curative resection for rectal cancer is associated with anterior position of the tumour. J. Br. Surg.; 2006; 93, pp. 105-112. [DOI: https://dx.doi.org/10.1002/bjs.5212]

38. Du, P.; Burke, J.P.; Khoury, W.; Lavery, I.C.; Kiran, R.P.; Remzi, F.H.; Dietz, D.W. Factors associated with the location of local rectal cancer recurrence and predictors of survival. Int. J. Color. Dis.; 2016; 31, pp. 825-832. [DOI: https://dx.doi.org/10.1007/s00384-016-2526-5]

39. Herzog, T.; Belyaev, O.; Chromik, A.M.; Weyhe, D.; Mueller, C.A.; Munding, J.; Tannapfel, A.; Uhl, W.; Seelig, M.H. TME quality in rectal cancer surgery. Eur. J. Med. Res.; 2010; 15, pp. 292-296. [DOI: https://dx.doi.org/10.1186/2047-783X-15-7-292]

40. van Oostendorp, S.E.; Smits, L.J.H.; Vroom, Y.; Detering, R.; Heymans, M.W.; Moons, L.M.G.; Tanis, P.J.; de Graaf, E.J.R.; Cunningham, C.; Denost, Q. et al. Local recurrence after local excision of early rectal cancer: A meta-analysis of completion TME, adjuvant (chemo)radiation, or no additional treatment. Br. J. Surg.; 2020; 107, pp. 1719-1730. [DOI: https://dx.doi.org/10.1002/bjs.12040]

41. Khan, M.A.S.; Ang, C.W.; Hakeem, A.R.; Scott, N.; Saunders, R.N.; Botterill, I. The impact of tumour distance from the anal verge on clinical management and outcomes in patients having a curative resection for rectal cancer. J. Gastrointest. Surg.; 2017; 21, pp. 2056-2065. [DOI: https://dx.doi.org/10.1007/s11605-017-3581-0]

42. Patel, S.V.; Roxburgh, C.S.; Vakiani, E.; Shia, J.; Smith, J.J.; Temple, L.K.; Paty, P.; Garcia-Aguilar, J.; Nash, G.; Guillem, J. et al. Distance to the anal verge is associated with pathologic complete response to neoadjuvant therapy in locally advanced rectal cancer. J. Surg. Oncol.; 2016; 114, pp. 637-641. [DOI: https://dx.doi.org/10.1002/jso.24358]

43. Gunka, I.; Dostalik, J.; Martinek, L.; Gunkova, P.; Mazur, M. Impact of blood transfusions on survival and recurrence in colorectal cancer surgery. Indian J. Surg.; 2013; 75, pp. 94-101. [DOI: https://dx.doi.org/10.1007/s12262-012-0427-6]

44. Wu, H.-L.; Tai, Y.-H.; Lin, S.-P.; Chan, M.-Y.; Chen, H.-H.; Chang, K.-Y. The Impact of Blood Transfusion on Recurrence and Mortality Following Colorectal Cancer Resection: A Propensity Score Analysis of 4030 Patients. Sci. Rep.; 2018; 8, 13345. [DOI: https://dx.doi.org/10.1038/s41598-018-31662-5]

45. Jeon, Y.; Kim, Y.J.; Jeon, J.; Nam, K.H.; Hwang, T.S.; Kim, K.G.; Baek, J.H. Machine learning based prediction of recurrence after curative resection for rectal cancer. PLoS ONE; 2023; 18, e0290141. [DOI: https://dx.doi.org/10.1371/journal.pone.0290141]

46. Xu, Y.; Ju, L.; Tong, J.; Zhou, C.-M.; Yang, J.-J. Machine Learning Algorithms for Predicting the Recurrence of Stage IV Colorectal Cancer After Tumor Resection. Sci. Rep.; 2020; 10, 2519. [DOI: https://dx.doi.org/10.1038/s41598-020-59115-y]

47. Liang, M.; Cai, Z.; Zhang, H.; Huang, C.; Meng, Y.; Zhao, L.; Li, D.; Ma, X.; Zhao, X. Machine Learning-based Analysis of Rectal Cancer MRI Radiomics for Prediction of Metachronous Liver Metastasis. Acad. Radiol.; 2019; 26, pp. 1495-1504. [DOI: https://dx.doi.org/10.1016/j.acra.2018.12.019] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/30711405]

48. Hamabe, A.; Ishii, M.; Kamoda, R.; Sasuga, S.; Okuya, K.; Okita, K.; Akizuki, E.; Sato, Y.; Miura, R.; Onodera, K. et al. Artificial intelligence-based technology for semi-automated segmentation of rectal cancer using high-resolution MRI. PLoS ONE; 2022; 17, e0269931. [DOI: https://dx.doi.org/10.1371/journal.pone.0269931]

49. Wang, P.P.; Deng, C.L.; Wu, B. Magnetic resonance imaging-based artificial intelligence model in rectal cancer. World J. Gastroenterol.; 2021; 27, pp. 2122-2130. [DOI: https://dx.doi.org/10.3748/wjg.v27.i18.2122] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34025068]

50. Qureshi, S.A.; Hussain, L.; Ibrar, U.; Alabdulkreem, E.; Nour, M.K.; Alqahtani, M.S.; Nafie, F.M.; Mohamed, A.; Mohammed, G.P.; Duong, T.Q. Radiogenomic classification for MGMT promoter methylation status using multi-omics fused feature space for least invasive diagnosis through mpMRI scans. Sci. Rep.; 2023; 13, 3291. [DOI: https://dx.doi.org/10.1038/s41598-023-30309-4] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/36841898]

51. Arrieta, A.B.; Díaz-Rodríguez, N.; Del Ser, J.; Bennetot, A.; Tabik, S.; Barbado, A.; García, S.; Gil-López, S.; Molina, D.; Benjamins, R. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Inf. Fusion; 2020; 58, pp. 82-115. [DOI: https://dx.doi.org/10.1016/j.inffus.2019.12.012]

52. Sluckin, T.C.; Hekhuis, M.; Kol, S.Q.; Nederend, J.; Horsthuis, K.; Beets-Tan, R.G.H.; Beets, G.L.; Burger, J.W.A.; Tuynman, J.B.; Rutten, H.J.T. et al. A Deep Learning Framework with Explainability for the Prediction of Lateral Locoregional Recurrences in Rectal Cancer Patients with Suspicious Lateral Lymph Nodes. Diagnostics; 2023; 13, 3099. [DOI: https://dx.doi.org/10.3390/diagnostics13193099] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/37835842]

53. Liu, X.; Zhang, D.; Liu, Z.; Li, Z.; Xie, P.; Sun, K.; Wei, W.; Dai, W.; Tang, Z.; Ding, Y. Deep learning radiomics-based prediction of distant metastasis in patients with locally advanced rectal cancer after neoadjuvant chemoradiotherapy: A multicentre study. EBioMedicine; 2021; 69, 103442. [DOI: https://dx.doi.org/10.1016/j.ebiom.2021.103442] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34157487]