Machine learning for the prediction of minor amputation in University of Texas grade 3 diabetic foot ulcers

Abstract

Minor amputations are performed in a large proportion of patients with diabetic foot ulcers (DFU) and early identification of the outcome of minor amputations facilitates medical decision-making and ultimately reduces major amputations and deaths. However, there are currently no clinical predictive tools for minor amputations in patients with DFU. We aim to establish a predictive model based on machine learning to quickly identify patients requiring minor amputation among newly admitted patients with DFU. Overall, 362 cases with University of Texas grade (UT) 3 DFU were screened from tertiary care hospitals in East China. We utilized the synthetic minority oversampling strategy to compensate for the disparity in the initial dataset. A univariable analysis revealed nine variables to be included in the model: random blood glucose, years with diabetes, cardiovascular diseases, peripheral arterial diseases, DFU history, smoking history, albumin, creatinine, and C-reactive protein. Then, risk prediction models based on five machine learning algorithms: decision tree, random forest, logistic regression, support vector machine, and extreme gradient boosting (XGBoost) were independently developed with these variables. After evaluation, XGBoost earned the highest score (accuracy 0.814, precision 0.846, recall 0.767, F1-score 0.805, and AUC 0.881). For convenience, a web-based calculator based on our data and the XGBoost algorithm was established (https://dfuprediction.azurewebsites.net/). These findings imply that XGBoost can be used to develop a reliable prediction model for minor amputations in patients with UT3 DFU, and that our online calculator will make it easier for clinicians to assess the risk of minor amputations and make proactive decisions.

Introduction

It is estimated that the number of diabetes patients worldwide will reach 300 million by 2025 [1]. Diabetic ulcers, particularly severe diabetic foot ulcers (DFU), are the most prevalent and severe diabetes consequence. When debridement and repair do not improve a patient’s diabetic foot condition [2], amputation becomes an unpleasant option. However, amputation can have a significant impact on the prognosis of individuals with DFU, decreasing their quality of life and increasing their chance of mortality [3–5]. According to statistics, the 5-year death rate for DFU patients will increase from 40% to 63% after amputation [1]. In addition, amputation multiplies the expense of treatment by a factor of five, hence increasing the load on the national health care system [6, 7].

DFU amputations can be divided into minor amputations below the ankle and major amputations above the ankle. Compared to major amputations as a life-saving alternative, minor amputations represent a larger clinical proportion (about 90%) of patients undergoing amputation [8]. Therefore, early diagnosis of the risk of small DFU amputation can benefit the majority of DFU patients, while early intervention for the risk of minor amputation is critical to prevent large amputations and preserve limbs [9, 10].

Machine learning is a subfield of artificial intelligence that has brought revolutionary changes to the field of health care through fast, efficient, accurate, and cost-effective computing decisions [11]. Machine learning plays an important role in the prediction of many common diseases, such as the diagnostic prediction of patients with type 2 diabetes [12] and the classification of cardiovascular diseases in patients with diabetes [13]. Several studies on DFU have shown the superior predictive performance of machine learning algorithms [14–19]. For the DFU diagnosis, Amith Khandakar et al. used thermogram images to establish a machine learning model based on CNN for early detection of DFU [15]. Rachita Nanda et al. constructed the prediction models for differentiating T2DM with DFU with four distinct machine learning methods [17]. For the DFU prognosis prediction, the Bayesian-based decision model [20] and the light gradient boosting machine model [19] were used to predict the amputation rate of DFU patients in different retrospective studies. Recently, a Chinese team focused on DFU mortality and amputation during the COVID-19 post-lockdown compared different machine-learning models [18]. However, these models’ performance was hampered due to a paucity of data. In a particular scenario, the choice of a machine learning algorithm depends on the sample dataset and the decision objective. The majority of studies utilized a single machine learning algorithm, they were unable to find the optimal algorithm.

In this study, we collected data on University of Texas grade 3 (UT3) DFU patients from two tertiary hospitals in eastern China. The University of Texas classification is a commonly used grading system, which is detailed in S1 Table. Patients with UT3 (bone or joint penetration) were selected because they constitute the majority of patients at tertiary hospitals and are the most susceptible to minor amputations. We used the following five supervised learning algorithms to filter out the best prediction model for minor amputation: logistic regression (LR), a long-standing statistical method to predict patients’ results based on the predictive variables of each patient [21]; random forest (RF), a mature learning algorithm widely used based on recursive methods [22]; decision tree (DT), a tree model in which each node has a question and each branch represents a result [23]; support vector machine (SVM), an excellent technology with independent integrity theory based on the optimal solution [24]; and extreme gradient boosting (XGBoost), an enhanced learning algorithm, which aims to transform weak learners into strong learners with high prediction accuracy [25].

Notably, the data collected by our team exhibits the class imbalance problem commonly associated with clinical data. To prevent its potential to lead to biased decision boundaries, we balanced the outcome variables using the synthetic minority oversampling technique (SMOTE) method, which has performed well in numerous studies [26–29]. To aid physicians, a final web-based calculator based on the best algorithm was also developed.

Methods

The study design flow chart and data screening flow sheet are shown in Fig 1. Below we present the study population, outcome and predictors, data processing, model building and model evaluation methods respectively.

[Figure omitted. See PDF.]

(A) Study design flowchart; (B) Data screening flowchart. Abbreviations: DFU, diabetic foot ulcer; DT, decision tree; LR, logistic regression; RF, random forest; SMOTE, synthetic minority oversampling technique; SVM, support vector machine; XGBoost, extreme gradient boosting.

Study population

All clinical data were selected from *** Hospital and the *** Hospital from January 2018 to December 2019. Both hospitals treated DFU under multidisciplinary cooperation. Only patients with UT3 DFU were included in the study while patients who received major amputation, gave up treatment, or had incomplete information were excluded. The demographic data, wound characteristics and laboratory indicators of patients were collected from the medical records. According to the Helsinki Declaration, our study was approved by the Ethics Committee of Nanjing Drum Tower Hospital (No. 2020–10901). Informed consent of the participants was waived because of the retrospective study design and the use of anonymized clinical data.

Outcome and predictors

Outcome.

A minor amputation is defined as any amputation distal to the ankle joint.

Predictors.

A total of 21 variables that have been shown to have a prognostic impact on diabetic ulcers in the previous literature were collected [30, 31]. All the enrolled predictors can be seen in the S2 Table. To avoid confusion, we have given specific explanations of the clinical indicators that may be controversial as follows:

1. Ulcer location: The location of the ulcer was judged by the endocrinologist and the burn plastic surgeon at the first visit to the hospital.

2. Wound duration: This period ranged from the first discovery to the first visit to the hospital.

3. Diabetic peripheral neuropathy (DPN): Clinical symptoms, such as finger/toe symmetrical sensory disturbance or abnormal nerve conduction velocity, are noted on electrophysiological nerve examination. Two of the following diagnostic criteria of DPN must be met [32]: (1) neuropathic pain, anesthesia, or other sensory abnormalities; (2) abnormal acupuncture sensation of the lower extremities or changes in the 10 g Sims-Weinstein monofilament test; or (3) decreased ankle reflex.

4. Smoking history: The definition refers to the standard recommended by the WHO in 1984, that is, smoking more than one cigarette a day for a continuous period of one month; otherwise, it is judged to be nonsmoking.

5. Drinking history: The criteria for being classified as a drinker includes a long history of drinking lasting for more than 5 years and daily alcohol consumption ≥80 g.

6. Hypertension: The criteria for determining hypertension are as follows: systolic blood pressure ≥140/90 mmHg or use of antihypertensive drugs with normal blood pressure.

7. Peripheral artery disease (PAD): If one or more lower extremity artery occlusions are found by Doppler ultrasound, it is diagnosed as peripheral artery disease [33].

8. Hyperlipidemia: Hyperlipidemia is known as lipid metabolism disorder and refers to total cholesterol, triglyceride, high-density lipoprotein cholesterol and low-density lipoprotein cholesterol exceeding the standard value.

To explore whether the variables were dose-dependent on the outcome of amputation of diabetic foot ulcers, we defined the abnormal values of variables according to clinical experience and guidelines [34–38]. Patients were divided based on 11.1–16.7 mmol/L and >16.7 mmol/L blood glucose. Patients were divided based on 25.0–34.9 g/L and <25.0 g serum albumin levels. Patients were divided based on 8–100 mg/L and >100 mg/L CRP levels. The duration of diabetes was divided into 0–10 years, 10–20 years and >20 years. Patients were divided based on creatine levels as follows: 134–186 μmol/L, 187–451 μmol/L, 452–771 μmol/L and >771 μmol/L. Details are provided in S2 Table.

SMOTE for the imbalanced dataset

Our dataset consisted of 75 (20.7%) patients with minor amputation and 287 (79.3%) patients without any amputation. A class imbalance exists in the original dataset, which is a common phenomenon in the field of data science.

The class imbalance will cause “undo” effect because they cannot weaken the deviations of the majority class. Therefore, we used the SMOTE technique to alleviate the data imbalance problem, which synthesizes new minority instances from the nearest neighbors of a straight line connecting a small number of samples. These new instances are created based on the characteristics of the original dataset, so they become similar to the original minority class instances [39]. The algorithm flow is as follows [40].

1. For each minority sample x, calculate its distance from the other samples in the minority class and get k-nearest neighbors.

2. Randomly select a number of samples from the k-nearest neighbors.

3. For each randomly chosen nearest neighbor x’, a new sample is synthesized according to the formula: xnew = x + rand(0,1)*(x’-x).

Determine the sampling multiplier based on the sample imbalance ratio and repeat the above process. Finally, a balanced dataset can be obtained.

Model construction

Statistical analyses were performed using SPSS 26.0. The enumeration data were expressed as count (percentage) and processed with a Chi-square test, whereas the measurement data were presented as the means ± standard deviation and analyzed by t-test. Factors with a p-value <0.05 in the univariate binary logistic regression analysis were used as candidate factors to construct predictive models. In this study, machine learning algorithms were selected as the modeling methods for DFU minor amputation prediction. The whole dataset was randomly divided into the training set and the verification set according to the proportion of approximately 7:3, in which the training set was used to build the prediction models, and the verification set was used to verify and evaluate the performance of the models. Five machine learning algorithms were adopted to build risk prediction models, including DT, RF, LR, SVM and XGBoost. Among them, RF and XGBoost are integrated machine learning classifiers, and the remainder are single classifiers. In the process of training, the optimal parameters were determined by 10 cross-validations.

Model evaluation

The evaluation indicators, including accuracy, precision, recall rate, F1-score, and AUC, were calculated to assess the constructed models. The closer the values are to 1, the better the performance of the prediction model. In addition, our experiment also used the receiver operating characteristic curve (ROC) to graphically represent the discernibility. As the most persuasive measurement for predictive analysis in machine learning, we also employed the confusion matrix (CM) for further model evaluation, which is a summarized table of the number of actual values and the predicted values yielded by the prediction model. Details of all these evaluation metrics are shown in Table 1.

[Figure omitted. See PDF.]

Results

Patient characteristics

From January 2018 to December 2019, a total of 362 patients with Texas grade 3 DFU were collected, including 257 males and 105 females aged 26–88 years. All the patient data, including demographics and disease and treatment characteristics, grouped by amputation are listed in S2 Table.

Univariable analysis results

Univariate regression analysis was conducted based on 362 patients with minor amputation as dependent variables, and demographics, wound characteristics, and laboratory indicators served as independent variables. Significant results are shown in Table 2, and 9 characteristic variables were selected (P<0.05): random blood glucose, years with diabetes, cardiovascular disease, peripheral arterial disease, smoking history, albumin, serum creatinine, C-reactive protein, and DFU history.

[Figure omitted. See PDF.]

SMOTE algorithm

To solve the class imbalance of the binary variables, we adopted the SMOTE technique. We sampled both the training dataset and the verification dataset, and the sample distribution before and after oversampling is shown in Table 3.

[Figure omitted. See PDF.]

Model establishment and evaluation

Undergoing a minor amputation operation or not was set as the label, and 9 statistically significant factors in univariate analysis were set as features. The risk prediction models of minor amputation in DFU patients were established by DT, RF, LR, SVM and XGBoost.

Fig 2 shows the results of the confusion matrix of various models. The confusion matrix of the classification results consists of four quadrants. In our study, 0 represents patients who did not receive minor amputation during this treatment, and 1 represents patients who received minor amputation during this treatment.

[Figure omitted. See PDF.]

(A) DT: decision tree. (B) RF: random forest. (C) LR: logistic regression. (D) SVM: support vector machine. (E) XGBoost: extreme gradient boosting.

Firstly, we compared the prediction performance of five machine learning algorithms before and after oversampling as shown in Table 4, which demonstrates the effectiveness of the SMOTE algorithm. Before SMOTE technique, all five machine learning algorithms suffered from data imbalance, resulting in biased classification boundaries. Although the overall classification accuracy was satisfactory, other metrics, such as precision, recall, and F1 score, were significantly lower, proving that the obtained models were clinically meaningless in practice. To further clearly compare the performance of the five machine learning algorithms with SMOTE technique, we visualized the evaluation metrics shown in Figs 3 and 4. It is not difficult to find that the integrated algorithms XGBoost and RF were higher than other single classification algorithms from an overall perspective, and XGBoost had the highest value in all indicators (accuracy 0.814, precision 0.846, recall 0.767, F1-score 0.805, and AUC 0.881), which indicates that XGBoost obtained optimal performance in the prediction of minor amputation in UT3 diabetic foot ulcer.

[Figure omitted. See PDF.]

Abbreviations: DT, Decision Tree; RF, Random Forest; LR, Logistic Regression; SVM, Support Vector Machine; XGBoost, extreme gradient boosting; AUC, area under the curve.

[Figure omitted. See PDF.]

Abbreviations: ROC, receiver operating characteristic curve; DFU, diabetic foot ulcers; DT, decision tree; RF, random forest; LR, logistic regression; SVM, support vector machine; XGBoost, extreme gradient boosting; FPR, false positive rate; TPR, true positive rate.

[Figure omitted. See PDF.]

Web-based calculator

Based on our dataset and the XGBoost algorithm, we constructed a web calculator using Microsoft Azure Web Sites. The address of the website is (https://dfuprediction.azurewebsites.net/). This web application enables the user to select nine variables to calculate the likelihood of minor amputation for DFU patients. S1 Fig contains an example.

The importance of characteristic variables

Based on the algorithm evaluation, XGBoost had the strongest prediction ability. Thus, we explored the relative importance of each feature variable in the XGBoost risk model. The feature importance ranking results are shown in Fig 5. Random blood glucose occupied the first, second and fifth places of importance, and history of DFU and serum albumin values less than 25 g/L ranked third and fourth in importance, respectively. Other important features included creatinine >771 μmol/L, peripheral artery disease, smoking history, diabetes mellitus, CRP and cardiovascular disease.

[Figure omitted. See PDF.]

Abbreviations: XGBoost, extreme gradient boosting.

Discussion

In recent years, a variety of linear regression models have been established to obtain the prognosis of clinical diseases. Linear regression models mainly refer to logistic regression and Cox regression. These models have strong predictability for the outcome of classified variables, but the predictive ability may be reduced because these models ignore the nonlinear relationship. With the continuous development of artificial intelligence, machine learning has been applied in the field of medical diagnosis and demonstrated better performance in the prediction of clinical disease prognosis [41]. In recent studies, many studies have shown the superior predictive power of machine learning algorithms for the diagnosis and prognosis of DFU [14–19]. However, there is a lack of studies with sufficient case data to determine comparatively which machine learning algorithm is the most suitable for use, especially in patients with UT3 DFU, which is most commonly seen in tertiary care hospitals. Our study focuses on predicting the rate of minor amputations in patients with UT3 DFU to more efficiently achieve limb preservation and reduce complications.

To identify the most suitable model for predicting minor amputation in UT3 DFU patients, our team collected 21 factors based on demographic features, wound features, and laboratory indicators in 362 cases. Five algorithms, namely DT, RF, LR, SVM and XGBoost, were used to predict the minor amputation probability. The XGBoost algorithm performed better than the commonly used linear regression model (LR), the single classifier (DT and SVM), and another ensemble learning machine learning algorithm (RF). Thus, the XGBoost algorithm was best to predict minor amputation of DFU patients (Table 4, Figs 3, 4). XGBoost is an ensemble learning algorithm that combines the predictions of multiple trees and adds the predicted scores of each tree to obtain the final score. With the advantages of high speed, high efficiency and high fault tolerance, XGBoost can effectively avoid overfitting and exhibit a high generalization ability. Many researchers have demonstrated that XGBoost has achieved good performance in the prediction of a variety of clinical diseases [42, 43]. Our study also demonstrates the good operation of SMOTE oversampling in solving the balance class problem in small samples of medical data.

In the results of our feature importance analysis, the top five important factors in our ranking were random blood glucose, DFU history and albumin. Random blood glucose accounted for three of the top five factors, emphasizing the importance of blood glucose control. Some studies have shown that intensive blood glucose control may lead to hypoglycemia [44–47]. Nevertheless, the relaxation of blood glucose control has contributed to the return of LEA based on an analysis of adult participants in the National Health and Nutrition Survey (NHANES) from 2010 to 2015 [48]. The Action to Control Cardiovascular Risk in Diabetes (ACCORD) trial collected 10251 randomly selected type 2 diabetic patients at 77 locations in the United States and Canada. This trial found that intensive blood glucose therapy was associated with a 31% reduced risk of major lower limb amputation [49]. Based on a systematic review of the literature on diabetic foot disease, the guidelines of the Society of Vascular Surgery (SVS) recommend enhanced blood glucose control (HbA1c = 6–7.5%) to reduce the risk of amputation [50]. Paying early attention to strict blood glucose control may be a key factor in preventing the progression of severe limb ischemia and subsequent major limb amputation. Our research particularly emphasizes this point.

The remaining top five important factors in our ranking were DFU history and albumin. Byung-Joon Jeon et al. demonstrated that a history of previous DFU increased the risk of LEA (P = 0.008, OR = 3.38) [51]. In our study, 32.8% of patients with a history of diabetic ulcers underwent a minor amputation. Our results are similar to those of Helm et al., who reported that 20–58% of patients relapsed within one year of wound healing [52]. However, Jiang et al. observed in a cohort of 669 Chinese people that the amputation rate of diabetic DFU patients within two years was 19.03% [8]. Our data differed because as a cross-sectional study, the interval between the last occurrence of ulcers in our patients may be greater than 2 years, and all patients in our cohort were UT3 patients whose ulcers were severe enough to reach the tendon. Patients with a history of ulcers are likely to have all the risk factors for ulcers. This finding reminds us to follow up on DFU after treatment, including disease care education and regular reexamination.

Low serum albumin always indicates malnutrition, which is more common in patients with DFU given elevated flux and poorly controlled blood glucose. Adequate nutrition is essential for tissue remodeling. The GNRI score for assessing nutrition-related risks consists of serum albumin, body weight, and ideal weight. In a study on nutrition and DFU healing, foot prognosis deteriorated with the increase in the risk of malnutrition (GNRI score) (linear trend, P<0.001) and the incidence of major LEA (11.3% vs. 1.8%) in the malnutrition (GNRI<92) group was approximately 6-fold increased [53, 54]. Meanwhile, a serum albumin level of <25 g/L, indicating severe malnutrition, has a greater impact on the amputation rate of DFU. In this case, even if DFU is treated, the prognosis is still poor [38]. The results of a nutritional supplement study showed improvement in patients with DFU who received arginine, glutamine and β-hydroxy-β-methyl butyrate supplements with albumin <40 g/L rather than below the normal value of 55 g/L. More research is needed in this area because it is still not clear whether the infection worsened due to the patient’s long-term hypoproteinemia or whether the serum albumin decreased sharply when dealing with the infection [55].

Our research is characterized by the following points: 1. Our study provides a real-world dataset from two tertiary hospitals in East China, focusing on outcomes in patients with UT3 diabetic foot ulcers and minor amputations. 2. We compared five popular machine learning algorithms (LR, RF, DT, SVM and XGBoost) to filter out the best-performing clinical models and ultimately provided a network calculator for use (https://dfuprediction.azurewebsites.net/). 3. Our study showed good results in small sample medical data using an oversampling approach (SMOTE) to balance the data. 4. Based on the results of the ranking of factors affecting the risk of minor amputation, we suggest that strict glycaemic control, active DFU follow-up including disease care education and regular review, and improved malnutrition are three key points to avoid minor amputation.

This study has some limitations. First, this study collected case data from the *** Hospital and the *** Hospital. Because these hospitals serve as the main referral centers for DFU, our conclusions are sufficiently representative. However, the amount of data is limited, limiting the performance of machine learning. We aim to verify the results in a larger sample size of patients from multiple centers in the future. Finally, due to incomplete examination records, we did not include some laboratory indicators that may be related to minor amputation, such as glycosylated hemoglobin and serum insulin levels, or the treatment of other diseases of the patient, such as hypolipidemic agents for ischemic heart disease and anticoagulants for peripheral vascular disease. The field of artificial intelligence is developing rapidly. Data from multisensory data and imaging CT and MRI data are already being used extensively in medical decision-making [56–58]. In the field of diabetic foot ulcers, we look forward to incorporating such data in our follow-up work to make more accurate monitoring and timely treatment of patients.

Conclusion

In this study, we used the data of patients with UT3 DFU collected from two tertiary care hospitals to train, test, and evaluate the predictive ability of five machine learning algorithms for minor amputation. Ultimately, XGBoost shows the best performance. The web calculator we have created would help clinical workers assess the risk of small amputation of diabetic foot ulcers at the time of admission, implement individualized treatment and optimize patient outcomes.

Supporting information

S1 Fig. An example of using the web calculator.

https://doi.org/10.1371/journal.pone.0278445.s001

(TIF)

S1 Table. University of Texas classification system.

https://doi.org/10.1371/journal.pone.0278445.s002

(DOCX)

S2 Table. Differences between demographic and clinical characteristics of non-amputation and minor amputation groups.

https://doi.org/10.1371/journal.pone.0278445.s003

(DOCX)

S1 Checklist. STROBE statement—Checklist of items that should be included in reports of observational studies.

https://doi.org/10.1371/journal.pone.0278445.s004

(DOCX)

Citation: Wang S, Wang J, Zhu MX, Tan Q (2022) Machine learning for the prediction of minor amputation in University of Texas grade 3 diabetic foot ulcers. PLoS ONE 17(12): e0278445. https://doi.org/10.1371/journal.pone.0278445

About the Authors:

Shiqi Wang

Contributed equally to this work with: Shiqi Wang, Jinwan Wang

Roles: Conceptualization, Data curation, Methodology, Writing – original draft

Affiliation: Department of Burns and Plastic Surgery, Affiliated Drum Tower Hospital, Medical School of Nanjing University, Nanjing, China

https://orcid.org/0000-0003-1991-8893

Jinwan Wang

Contributed equally to this work with: Shiqi Wang, Jinwan Wang

Roles: Conceptualization, Data curation, Methodology, Writing – original draft

Affiliation: School of Information Management, Nanjing University, Nanjing, China

Mark Xuefang Zhu

Roles: Conceptualization, Supervision, Writing – review & editing

E-mail: [email protected] (MXZ); [email protected] (QT)

Affiliation: School of Information Management, Nanjing University, Nanjing, China

https://orcid.org/0000-0002-8244-5999

Qian Tan

Roles: Funding acquisition, Supervision, Writing – review & editing

E-mail: [email protected] (MXZ); [email protected] (QT)

Affiliation: Department of Burns and Plastic Surgery, Affiliated Drum Tower Hospital, Medical School of Nanjing University, Nanjing, China

References

1. Jupiter DC, Thorud JC, Buckley CJ, Shibuya N. The impact of foot ulceration and amputation on mortality in diabetic patients. I: From ulceration to death, a systematic review. International wound journal. 2016;13(5):892–903. Epub 2015/01/21. pmid:25601358.

2. Barrientos S, Stojadinovic O, Golinko MS, Brem H, Tomic-Canic M. Growth factors and cytokines in wound healing. Wound repair and regeneration: official publication of the Wound Healing Society [and] the European Tissue Repair Society. 2008;16(5):585–601. Epub 2009/01/09. pmid:19128254.

3. Armstrong DG, Boulton AJM, Bus SA. Diabetic Foot Ulcers and Their Recurrence. New England Journal of Medicine. 2017;376(24):2367–75. pmid:28614678

4. Jeffcoate WJ, Vileikyte L, Boyko EJ, Armstrong DG, Boulton AJM. Current Challenges and Opportunities in the Prevention and Management of Diabetic Foot Ulcers. Diabetes Care. 2018;41(4):645–52. pmid:29559450

5. Dutra LMA, Melo MC, Moura MC, Leme LAP, De Carvalho MR, Mascarenhas AN, et al. Prognosis of the outcome of severe diabetic foot ulcers with multidisciplinary care. J Multidiscip Healthc. 2019;12:349–59. pmid:31118658.

6. Cavanagh P, Attinger C, Abbas Z, Bal A, Rojas N, Xu Z-R. Cost of treating diabetic foot ulcers in five different countries. Diabetes/Metabolism Research and Reviews. 2012;28(S1):107–11. pmid:22271734

7. Hicks CW, Selvarajah S, Mathioudakis N, Sherman RE, Hines KF, Black JH, et al. Burden of Infected Diabetic Foot Ulcers on Hospital Admissions and Costs. Annals of vascular surgery. 2016;33:149–58. Epub 2016/02/26. pmid:26907372.

8. Jiang Y, Ran X, Jia L, Yang C, Wang P, Ma J, et al. Epidemiology of type 2 diabetic foot problems and predictive factors for amputation in China. Int J Low Extrem Wounds. 2015;14(1):19–27. Epub 2015/01/13. pmid:25573978.

9. Meloni M, Morosetti D, Giurato L, Stefanini M, Loreni G, Doddi M, et al. Foot Revascularization Avoids Major Amputation in Persons with Diabetes and Ischaemic Foot Ulcers. Journal of clinical medicine. 2021;10(17). Epub 2021/09/11. pmid:34501432.

10. Ugwu E, Adeleye O, Gezawa I, Okpe I, Enamino M, Ezeani I. Predictors of lower extremity amputation in patients with diabetic foot ulcer: findings from MEDFUN, a multi-center observational study. Journal of foot and ankle research. 2019;12:34. Epub 2019/06/22. pmid:31223342.

11. Jayatilake S, Ganegoda GU. Involvement of Machine Learning Tools in Healthcare Decision Making. Journal of healthcare engineering. 2021;2021:6679512. Epub 2021/02/13. pmid:33575021.

12. Priya S, Rajalaxmi RR, editors. An Improved Data Mining Model to Predict the Occurrence of Type-2 Diabetes using Neural Network 2012.

13. Parthiban G, Srivatsa S. Applying Machine Learning Methods in Diagnosing Heart Disease for Diabetic Patients. International Journal of Applied Information Systems. 2012;3:25–30.

14. Schäfer Z, Mathisen A, Svendsen K, Engberg S, Rolighed Thomsen T, Kirketerp-Møller K. Toward Machine-Learning-Based Decision Support in Diabetes Care: A Risk Stratification Study on Diabetic Foot Ulcer and Amputation. Frontiers in medicine. 2020;7:601602. Epub 2021/03/09. pmid:33681236.

15. Khandakar A, Chowdhury MEH, Ibne Reaz MB, Md Ali SH, Hasan MA, Kiranyaz S, et al. A machine learning model for early detection of diabetic foot using thermogram images. Computers in biology and medicine. 2021;137:104838. Epub 2021/09/18. pmid:34534794.

16. Stefanopoulos S, Ayoub S, Qiu Q, Ren G, Osman M, Nazzal M, et al. Machine learning prediction of diabetic foot ulcers in the inpatient population. Vascular. 2021:17085381211040984. Epub 2021/09/01. pmid:34461765.

17. Nanda R, Nath A, Patel S, Mohapatra E. Machine learning algorithm to evaluate risk factors of diabetic foot ulcers and its severity. Medical & biological engineering & computing. 2022. Epub 2022/06/26. pmid:35751828.

18. Du C, Li Y, Xie P, Zhang X, Deng B, Wang G, et al. The amputation and mortality of inpatients with diabetic foot ulceration in the COVID-19 pandemic and postpandemic era: A machine learning study. International wound journal. 2021. Epub 2021/11/25. pmid:34818691.

19. Xie P, Li Y, Deng B, Du C, Rui S, Deng W, et al. An explainable machine learning model for predicting in-hospital amputation rate of patients with diabetic foot ulcer. International wound journal. 2022;19(4):910–8. Epub 2021/09/15. pmid:34520110.

20. Hüsers J, Hafer G, Heggemann J, Wiemeyer S, John SM, Hübner U. Predicting the amputation risk for patients with diabetic foot ulceration—a Bayesian decision support tool. BMC medical informatics and decision making. 2020;20(1):200. Epub 2020/08/26. pmid:32838777.

21. Meurer WJ, Tolles J. Logistic Regression Diagnostics: Understanding How Well a Model Predicts Outcomes. Jama. 2017;317(10):1068–9. Epub 2017/03/16. pmid:28291878.

22. Marchese Robinson RL, Palczewska A, Palczewski J, Kidley N. Comparison of the Predictive Performance and Interpretability of Random Forest and Linear Models on Benchmark Data Sets. Journal of chemical information and modeling. 2017;57(8):1773–92. Epub 2017/07/18. pmid:28715209.

23. Ghiasi MM, Zendehboudi S. Application of decision tree-based ensemble learning in the classification of breast cancer. Computers in biology and medicine. 2021;128:104089. Epub 2020/12/19. pmid:33338982.

24. Huang S, Cai N, Pacheco PP, Narrandes S, Wang Y, Xu W. Applications of Support Vector Machine (SVM) Learning in Cancer Genomics. Cancer genomics & proteomics. 2018;15(1):41–51. Epub 2017/12/25. pmid:29275361.

25. Zhang PB, Yang ZX. A Novel AdaBoost Framework With Robust Threshold and Structural Optimization. IEEE transactions on cybernetics. 2018;48(1):64–76. Epub 2016/11/30. pmid:27898387.

26. Taneja S, Suri B, Kothari C, editors. Application of Balancing Techniques with Ensemble Approach for Credit Card Fraud Detection. 2019 International Conference on Computing, Power and Communication Technologies (GUCON); 2019 27–28 Sept. 2019.

27. Davagdorj K, Lee JS, Pham VH, Ryu KH. A Comparative Analysis of Machine Learning Methods for Class Imbalance in a Smoking Cessation Intervention. 2020;10(9):3307.

28. Amin A, Anwar S, Adnan A, Nawaz M, Howard N, Qadir J, et al. Comparing Oversampling Techniques to Handle the Class Imbalance Problem: A Customer Churn Prediction Case Study. IEEE Access. 2016;4:7940–57.

29. Barros TM, Souza Neto PA, Silva I, Guedes LA. Predictive Models for Imbalanced Data: A School Dropout Perspective. 2019;9(4):275.

30. K KK, N HKR, Y NP. Risk factor analysis on the healing time and infection rate of diabetic foot ulcers in a referral wound care clinic. Journal of Wound Care. 2019;28:S4–S13. pmid:30724120.

31. Mohammad Zadeh M, Lingsma H, van Neck JW, Vasilic D, van Dishoeck A-M. Outcome predictors for wound healing in patients with a diabetic foot ulcer. International wound journal. 2019;16(6):1339–46. Epub 2019/08/16. pmid:31418528.

32. P-B R, B AJ, F EL, B V, F R, M RA, et al. Diabetic Neuropathy: A Position Statement by the American Diabetes Association. Diabetes care. 2017;40(1):136–54. pmid:27999003.

33. M S, F H, G U, H H, K K, K GT, et al. Long-term prognosis of diabetic foot patients and their limbs: amputation and death over the course of a decade. 2012;35(10):2021–7. pmid:22815299.

34. 2. Classification and Diagnosis of Diabetes: Standards of Medical Care in Diabetes-2019. Diabetes Care. 2019;42(Suppl 1):S13–s28. Epub 2018/12/19. pmid:30559228.

35. Akha O, Kashi Z, Makhlough A. Correlation between amputation of diabetic foot and nephropathy. Iranian journal of kidney diseases. 2010;4(1):27–31. Epub 2010/01/19. pmid:20081301.

36. Sharma H, Sharma S, Krishnan A, Yuan D, Vangaveti VN, Malabu UH, et al. The efficacy of inflammatory markers in diagnosing infected diabetic foot ulcers and diabetic foot osteomyelitis: Systematic review and meta-analysis. PloS one. 2022;17(4):e0267412. Epub 2022/04/28. pmid:35476639.

37. Resnick HE, Carter EA, Sosenko JM, Henly SJ, Fabsitz RR, Ness FK, et al. Incidence of lower-extremity amputation in American Indians: the Strong Heart Study. Diabetes Care. 2004;27(8):1885–91. Epub 2004/07/28. pmid:15277412.

38. Zhang SS, Tang ZY, Fang P, Qian HJ, Xu L, Ning G. Nutritional status deteriorates as the severity of diabetic foot ulcers increases and independently associates with prognosis. Experimental and therapeutic medicine. 2013;5(1):215–22. Epub 2012/12/20. pmid:23251271.

39. Chawla NV. Data Mining for Imbalanced Datasets: An Overview. In: Maimon O, Rokach L, editors. Data Mining and Knowledge Discovery Handbook. Boston, MA: Springer US; 2005. p. 853–67.

40. Fernández A, Garcia S, Herrera F, Chawla N. SMOTE for Learning from Imbalanced Data: Progress and Challenges, Marking the 15-year Anniversary. Journal of Artificial Intelligence Research. 2018;61:863–905.

41. Christodoulou E, Ma J, Collins GS, Steyerberg EW, Verbakel JY, Van Calster B. A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. Journal of clinical epidemiology. 2019;110:12–22. Epub 2019/02/15. pmid:30763612.

42. Hou N, Li M, He L, Xie B, Wang L, Zhang R, et al. Predicting 30-days mortality for MIMIC-III patients with sepsis-3: a machine learning approach using XGboost. J Transl Med. 2020;18(1):462-. pmid:33287854.

43. Tseng PY, Chen YT, Wang CH, Chiu KM, Peng YS, Hsu SP, et al. Prediction of the development of acute kidney injury following cardiac surgery by machine learning. Critical care (London, England). 2020;24(1):478. Epub 2020/08/02. pmid:32736589.

44. Action to Control Cardiovascular Risk in Diabetes Study G, Gerstein HC, Miller ME, Byington RP, Goff DC, Bigger JT, et al. Effects of intensive glucose lowering in type 2 diabetes. The New England journal of medicine. 2008;358(24):2545–59. Epub 2008/06/06. pmid:18539917.

45. Patel A, MacMahon S, Chalmers J, Neal B, Billot L, Woodward M, et al. Intensive blood glucose control and vascular outcomes in patients with type 2 diabetes. The New England journal of medicine. 2008;358(24):2560–72. Epub 2008/06/10. pmid:18539916.

46. Duckworth W, Abraira C, Moritz T, Reda D, Emanuele N, Reaven PD, et al. Glucose Control and Vascular Complications in Veterans with Type 2 Diabetes. New England Journal of Medicine. 2009;360(2):129–39. pmid:19092145

47. Turnbull FM, Abraira C, Anderson RJ, Byington RP, Chalmers JP, Duckworth WC, et al. Intensive glucose control and macrovascular outcomes in type 2 diabetes. Diabetologia. 2009;52(11):2288–98. pmid:19655124

48. Caruso P, Scappaticcio L, Maiorino MI, Esposito K, Giugliano D. Up and down waves of glycemic control and lower-extremity amputation in diabetes. Cardiovascular diabetology. 2021;20(1):135. Epub 2021/07/08. pmid:34229673.

49. Goldman MP, Clark CJ, Craven TE, Davis RP, Williams TK, Velazquez-Ramirez G, et al. Effect of Intensive Glycemic Control on Risk of Lower Extremity Amputation. Journal of the American College of Surgeons. 2018;227(6):596–604. Epub 2018/10/20. pmid:30336205.

50. Hasan R, Firwana B, Elraiyah T, Domecq JP, Prutsky G, Nabhan M, et al. A systematic review and meta-analysis of glycemic control for the prevention of diabetic foot syndrome. J Vasc Surg. 2016;63(2 Suppl):22S–8S.e1-2. Epub 2016/01/26. pmid:26804364.

51. Jeon BJ, Choi HJ, Kang JS, Tak MS, Park ES. Comparison of five systems of classification of diabetic foot ulcers and predictive factors for amputation. International wound journal. 2017;14(3):537–45. Epub 2016/10/11. pmid:27723246.

52. Helm PA, Walker SC, Pullium GF. Recurrence of neuropathic ulceration following healing in a total contact cast. Archives of physical medicine and rehabilitation. 1991;72(12):967–70. Epub 1991/11/01. pmid:1953319.

53. Gau BR, Chen HY, Hung SY, Yang HM, Yeh JT, Huang CH, et al. The impact of nutritional status on treatment outcomes of patients with limb-threatening diabetic foot ulcers. J Diabetes Complications. 2016;30(1):138–42. Epub 2015/10/23. pmid:26490754.

54. Shaikh I, Masood N, Shaikh D, Sheikh M. Diabetic Foot Ulcers: Correlation of Nutritional Status of Type 2 Diabetic Patients of Hyderabad Sindh, Pakistan. THE PROFESSIONAL MEDICAL JOURNAL. 2017;24:707–12.

55. Brookes JDL, Jaya JS, Tran H, Vaska A, Werner-Gibbings K, D’Mello AC, et al. Broad-Ranging Nutritional Deficiencies Predict Amputation in Diabetic Foot Ulcers. Int J Low Extrem Wounds. 2020;19(1):27–33. Epub 2019/09/25. pmid:31549527.

56. Srinivasu PN, Ijaz MF, Shafi J, Wozniak M, Sujatha R. 6G Driven Fast Computational Networking Framework for Healthcare Applications. IEEE Access. 2022;10:94235–48.

57. Ali S, El-Sappagh S, Ali F, Imran M, Abuhmed T. Multitask Deep Learning for Cost-Effective Prediction of Patient’s Length of Stay and Readmission State Using Multimodal Physical Activity Sensory Data. IEEE Journal of Biomedical and Health Informatics. 2022:1–14. pmid:36037451

58. Ali F, Khan S, Waseem Abbas A, Shah B, Hussain T, Song D, et al. A Two-Tier Framework Based on GoogLeNet and YOLOv3 Models for Tumor Detection in MRI. Computers, Materials & Continua. 2022;72(1):73–92.