The value of artificial intelligence in the diagnosis of lung cancer: A systematic review and meta-analysis

Abstract

Lung cancer is a common malignant tumor disease with high clinical disability and death rates. Currently, lung cancer diagnosis mainly relies on manual pathology section analysis, but the low efficiency and subjective nature of manual film reading can lead to certain misdiagnoses and omissions. With the continuous development of science and technology, artificial intelligence (AI) has been gradually applied to imaging diagnosis. Although there are reports on AI-assisted lung cancer diagnosis, there are still problems such as small sample size and untimely data updates. Therefore, in this study, a large amount of recent data was included, and meta-analysis was used to evaluate the value of AI for lung cancer diagnosis. With the help of STATA16.0, the value of AI-assisted lung cancer diagnosis was assessed by specificity, sensitivity, negative likelihood ratio, positive likelihood ratio, diagnostic ratio, and plotting the working characteristic curves of subjects. Meta-regression and subgroup analysis were used to investigate the value of AI-assisted lung cancer diagnosis. The results of the meta-analysis showed that the combined sensitivity of the AI-aided diagnosis system for lung cancer diagnosis was 0.87 [95% CI (0.82, 0.90)], specificity was 0.87 [95% CI (0.82, 0.91)] (CI stands for confidence interval.), the missed diagnosis rate was 13%, the misdiagnosis rate was 13%, the positive likelihood ratio was 6.5 [95% CI (4.6, 9.3)], the negative likelihood ratio was 0.15 [95% CI (0.11, 0.21)], a diagnostic ratio of 43 [95% CI (24, 76)] and a sum of area under the combined subject operating characteristic (SROC) curve of 0.93 [95% CI (0.91, 0.95)]. Based on the results, the AI-assisted diagnostic system for CT (Computerized Tomography), imaging has considerable diagnostic accuracy for lung cancer diagnosis, which is of significant value for lung cancer diagnosis and has greater feasibility of realizing the extension application in the field of clinical diagnosis.

1 Introduction

Lung cancer is a malignant tumor originates from the epithelial tissue. Air pollution, radiation exposure, and fungal infections contribute to the persistent incidence and mortality of lung cancer, which ranks first in incidence and mortality among malignant tumors in China and worldwide. The overall five-year survival rate of lung cancer is only 15.6%, and the prognosis of patients with different clinical stages is significantly different. The cure rate of carcinoma in situ is close to 100%, however, according to statistics from relevant sources, the 5-year survival rate of lung cancer is 70% in stage I, and less than 5% in stage IV, respectively [1]. Therefore, early diagnosis of lung cancer is critical. In Table 1, we provide the specific meanings of the abbreviations used throughout the paper. In Fig 1, we present the overall structure of the paper.

[Figure omitted. See PDF.]

[Figure omitted. See PDF.]

1.1 Diagnosis of early lung cancer

In recent years, techniques and methods for early lung cancer diagnosis have been broadly divided into three main categories: diagnostic imaging, diagnostic pathology tests, and diagnostic marker tests. Diagnostic imaging refers to the use of chest radiographs, chest CT, chest MRI (Magnetic Resonance Imaging), and PET-CT (Positron Emission Tomography-Computed Tomography) in the chest to identify the presence of a mass or lesion in the lung and determine the possibility of cancer [2]. In the 1990s, Naildich [3] proposed LDSCT (Low-Dose Spiral Computed Tomography) as a new method of lung cancer screening. LDSCT is significantly more sensitive than chest radiography for early lung cancer. Related studies in the US, Japan, and Germany have indicated that LDSCT can reduce patients’ radiation dose and have a higher sensitivity for lung nodule detection [4]. However, the imaging diagnosis is easily affected by factors such as the patient’s lung vacuolation and lung tissue reshaping, and the diagnostic accuracy is relatively low, so its application in early-stage lung cancer is minimal [5].

Pathological diagnosis often refers to histological examination through bronchoscopy or percutaneous puncture biopsy, which is the gold standard for lung cancer diagnosis. The main methods of obtaining histological specimens are fibreoptic bronchoscopy, ultrasound, or CT-guided percutaneous lung biopsy, among which bronchoscopy is the most commonly used and growing fastest in recent years [6]. Fibreoptic bronchoscopy is the essential means to confirm the diagnosis of lung cancer. However, the traditional white light bronchoscopy (WLB) has a meager diagnostic rate (<29%) for peripheral lung cancer, especially for some early mucosal and submucosal lesions and precancerous lesions [7]. Therefore, the rise of new bronchoscopic techniques in recent years, such as fluorescence bronchoscopy (AFB), fluorescence confocal microscopy (FCFM), endobronchial ultrasound (EBUS), virtual navigational bronchoscopy (VNB), and electromagnetic navigational bronchoscopy (ENB), have expanded the diagnostic field and improved the diagnostic rate, especially playing a vital role in the diagnosis of early-stage lung cancer. However, most of these techniques have limitations. For example, they are susceptible to external factors, have a high false-positive rate, or still need further experiments to verify whether they can be used in clinical practice.

Lung cancer marker testing refers to tumor markers such as glycoprotein substances, enzymes, and hormonal substances expressed and secreted by tumor tissue that can be obtained through blood and body fluid tests [8]. At present, dozens of tumor markers associated with lung cancer have been identified, such as gastrin-releasing peptide precursor (pro-GRP), neuron-specific enolase (NSE), and carcinoembryonic antigen (CEA), squamous cell carcinoma antigen (SCC-Ag), etc. [9]. However, a tumor marker with high sensitivity and specificity has not yet been applied to the clinical independently. Therefore, more research has been conducted to detect multiple tumor markers from different tissue sources such as pleural fluid, serum, and bronchial lavage fluid simultaneously, which is expected to improve the diagnostic rate of tumor markers for early lung cancer.

In addition, many novel markers can be used for the early detection and diagnosis of lung cancer, such as molecular biomarkers of susceptibility-related genes oncogenes LOH, DNA, methylated telomerase, and exhaled volatile organic compounds (VOCs) in high-risk groups. However, there are many different types of VOCs, complex sources, and many elements of detection technology involved, and there is still a lack of uniform standards [10]. Once standardized detection techniques are identified, the method will revolutionize the early screening of lung cancer.

1.2 Advances in AI for lung cancer diagnosis research

The current clinical practice of early screening using CT scans of the chest is a time-consuming and relatively subjective process that is prone to inter-observer variability. With the development of AI and digital pathology in recent years, the medical community has increasingly recognized the significant clinical and scientific value of AI in aiding pathological diagnosis. The application of AI recognition technology enables multi-parametric clustering analysis to help physicians screen for early-stage lung cancer [11], reducing errors and increasing problem-solving efficiency. AI has made breakthroughs in detecting, diagnosing, and treating lung cancer [12].

1.2.1 Cytopathological diagnosis of AI.

Lung cancer is divided into small cell lung cancer, non-small cell lung cancer (NSCLC), and NSCLC accounts for most lung cancers and is the primary pathological type of lung cancer death [13]. AI cytopathology diagnostic systems have been applied to lung cancer classification and diagnosis. Teramoto [14] developed a method to automatically generate cytological images using generative adversarial networks (GANs) to improve deep convolutional neural networks (DCNNs) by using actual and synthetic cytological images and GANs. Neural networks (DCNNs) use authentic and synthetic cytological images and generative adversarial networks to improve the classification accuracy of DCNNs. The results show a substantial increase in accuracy compared to previous studies that did not use GAN-generated images for pre-training. These results confirm the effectiveness of their proposed method for the classification of cytological images when only limited data is available.

1.2.2 Histopathological diagnosis of AI.

In lung cancer histopathological diagnosis, AI has been able to accurately classify lung cancer subtypes through analysis of digital pathological tissue sections and can predict the survival prognosis of NSCLC patients. Yu [15] used images of tissue sections from lung adenocarcinoma and squamous cell carcinoma patients for validation, extracted morphological image features and developed classifiers that effectively distinguish malignant tumors from adjacent healthy tissues (AUC = 0.81). In addition, it was able to accurately predict long-term survival in patients with stage I adenocarcinoma (log-rank test P = 0.002) and squamous cell carcinoma (log-rank test P = 0.023). Coudray [16] trained a deep convolutional neural network on the full range of section images obtained from The Cancer Genome Atlas, which accurately and automatically classified lung histopathological images into adenocarcinoma, squamous cell carcinoma, and normal lung tissue with results consistent with the pathologist’s analysis, with an average AUC (Area Under The Curve) of 0.97.

1.2.3 Diagnosis of lung cancer markers in AI.

In addition to detecting malignant lung lesions using imaging histology, tumor markers have a crucial role in cancer detection. Specific long non-coding RNAs (lncRNAs) have promoted or inhibited cancer progression in lung cancer patients and are expected to be used as diagnostic markers. Wang [17] used machine learning and weighted gene co-expression networks, the Lasso algorithm, and other techniques to screen the lung genome atlas database of 1364 lncRNAs for the adenocarcinoma best biological markers. LANCL1-AS1, MIR3945HG, and LINC01270 were identified as LUAD markers, with MIR3945HG also the biomarker with the highest diagnostic value for LUSC and strongly correlated with survival [18].

Detection of mutated genes is also now a routine and essential part of the treatment and prognosis of lung cancer. It has been demonstrated that AI can help detect mutated genes in lung cancer. Coudray [19] hypothesized that specific gene mutations would alter the arrangement of lung cancer tumor cells on whole-section images. Thus they predicted the ten most common mutated genes in adenocarcinoma by training a neural network and found that six (STK11, EGFR, FAT1, SETBP1, KRAS, and TP53) could be predicted by pathological images with an accuracy of 73.3% to 85.6%. The study illustrates that the AI histopathology diagnostic system has the potential to help pathologists rapidly detect mutated genes in lung cancer, facilitating the early initiation of targeted drug therapy to improve treatment outcomes and patient prognosis.

1.2.4 AI in the prognosis of lung cancer.

Wang [20] developed a ConvPath automated cell classification model, which can output microenvironmental characteristics of tumors based on the spatial distribution characteristics of different cell types, and the microenvironmental characteristic was shown to be an independent prognostic factor for lung cancer adenocarcinoma. Their studies also demonstrated that patients’ survival could be predicted by analyzing the spatial organization of 48 HE-stained images of lung adenocarcinoma. Late survival rates were significantly lower in the high-risk group than in the low-risk group. These studies illustrate that AI diagnostic systems can yield valuable quantitative information for the prognosis of lung cancer patients.

1.3 Research progress

In recent years, with satisfactory results, AI lung cancer screening has been used in the lung cancer population. Huang [21] investigated a novel diagnostic method based on deep transfer convolutional neural networks and extreme learning machines to deal with benign and malignant nodule classification, and the diagnostic method had an accuracy of 94.57% and an AUC of 0.95. In this study, the AI-assisted diagnostic system diagnosed lung cancer with a combined sensitivity of 0.86, the specificity was 0.88, the positive likelihood ratio was 7.2, the negative likelihood ratio was 0.16, the diagnostic ratio was 46, and the AUC value was 0.93. Irvin [22] used a deep learning algorithm called CheXNeXt to improve diagnostic accuracy for lung cancer on chest radiographs. The results showed that the sensitivity of the AI detection method for diagnosis in lung cancer populations was 0.899, the specificity was 0.901, and the AUC value was 0.935. It indicates that AI lung cancer screening for lung cancer populations can obtain high and that using AI algorithms to assist in the diagnosis of lung cancer can help to reduce the time required for radiologists to review images. It can provide an imaging basis and reference for clinical diagnosis and treatment, which is consistent with the results of this study. Dong [23] conducted a value analysis of the AI-assisted diagnostic system based on CT images for 4771 lung cancer diagnoses. The results showed that the combined sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, diagnostic ratio, and AUC values were 0.87, 0.89, 7.70, 0.14, 53.54, and 0.94. The findings were similar to the present study, indicating that the AI-assisted diagnostic system has a high diagnostic value. The results showed that the AI-assisted diagnostic system has a high diagnostic value for lung cancer and can be used to diagnose lung cancer in the clinical setting. Xie [24] et al. designed Semi-Supervised Adversarial Classification (SSAC) model that can be trained with limited labeled and unlabeled data. The model used Multi-View Knowledge-Based Collaborative (MV-KBC), and achieved 92.53% accuracy and 96.28% specificity in LIDC-IDRI database. This suggests that the results of AI lung cancer screening are not limited by the physician’s expertise, which is consistent with the findings of this study, and that AI-assisted lung cancer screening improves the sensitivity and accuracy of early lung cancer identification, aids clinicians in diagnosis, and reduces physician workload.

1.4 Study content

So far, there have been many studies on the effectiveness of AI-assisted diagnostic systems based on CT images in the diagnosis of lung cancer, but these studies have a small sample size, different study quality, and different AI algorithms. Therefore, this paper adopts the meta-analysis method to systematically evaluate and meta-analyze the diagnostic value of AI-assisted diagnostic systems in lung cancer, in order to provide evidence for clinical application. This paper expands the sample size and uses meta-analysis to evaluate the value of the AI diagnostic system for lung cancer diagnosis. With the help of STATA16.0, the importance of AI-aided diagnosis for lung cancer diagnosis was assessed by combining effect measures, including specificity, sensitivity, negative likelihood ratio, positive likelihood ratio, diagnostic ratio, and plotting the operational characteristics curve of subjects, and meta-regression. The study also explored the reasons for the heterogeneity between studies employing meta-regression and subgroup analysis to provide evidence for clinical application.

The results of the meta-analysis showed that the combined sensitivity of the AI-aided diagnosis system for lung cancer diagnosis was 0.87 [95% CI (0.82, 0.90)], specificity was 0.87 [95% CI (0.82, 0.91)] (CI stands for confidence interval.), the missed diagnosis rate was 13%, the misdiagnosis rate was 12%, the positive likelihood ratio was 6.5 [95% CI (4.6, 9.3)], the negative likelihood ratio was 0.15 [95% CI (0.11, 0.21)], a diagnostic ratio of 43 [95% CI (24, 76)] and a sum of area under the combined subject operating characteristic (SROC) curve of 0.93 [95% CI (0.91, 0.95)]. Therefore, this study shows that in clinical practice, AI recognition technology can effectively improve the diagnostic sensitivity of early lung cancer, assist physicians to screen early lung cancer more effectively and quickly, and become an auxiliary tool for clinical diagnosis of lung cancer, worthy of clinical promotion.

2 Materials and methods

This meta-analysis was based on the Cochrane Handbook for Systematic Reviews (v5.1.0) and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) NMA Checklist [25].

2.1 Inclusion and exclusion criteria

Inclusion criteria: (1) Screening articles were following the STARD (The Standards for Reporting of Diagnostic Accuracy) statement; (2) Published literature containing AI reading of chest CT images were used to diagnose lung cancer; (3) AI was used to read chest CT images with precise diagnostic results; (4) The specificity and sensitivity of AI reading chest CT images for the diagnosis of lung cancer is based on pathology testing as the gold standard.

Exclusion criteria: (1) literature for which complete data and duplicates are not available; (2) literature for which pathology testing is not the gold standard; (3) literature for which case studies, reviews, animal studies, reviews, abstracts, etc. are available.

2.2 Literature search methods

A comprehensive search of Chinese and English databases was conducted to retrieve literature on AI reading of chest CT images for the diagnosis of lung cancer from January 1, 2010 to August 2021. Chinese databases include: China Biomedical Literature Database (CBM), Wanfang Database, and China Knowledge Network (CNKI); English databases include: PubMed, EMbase, the Cochrane Library, This study combined a search for subject terms, keywords, or free words.

Our search strategies were as follows: “Pulmonary Neoplasms* OR Neoplasms, Lung* OR Lung Neoplasm* OR Neoplasm, Lung* OR Neoplasms, Pulmonary* OR Neoplasm, Pulmonary* OR Pulmonary Neoplasm* OR Lung Cancer* OR Cancer, Lung* OR Cancers, Lung* OR Lung Cancers* OR Pulmonary Cancer* OR Cancer, Pulmonary;Cancers, Pulmonary* OR Pulmonary Cancers* OR Cancer of the Lung* OR Cancer of Lung” AND “Intelligence, Artificial OR Computational Intelligence OR Intelligence, Computational OR Machine Intelligence OR Intelligence, Machine OR Computer Reasoning OR Reasoning, Computer OR AI (Artificial Intelligence) OR Computer Vision Systems OR Computer Vision System OR System, Computer Vision OR Systems, Computer Vision OR Vision System, Computer OR Vision Systems, Computer OR Knowledge Acquisition (Computer) OR Acquisition, Knowledge (Computer) OR Knowledge Representation (Computer) OR Knowledge Representations (Computer) OR Representation, Knowledge (Computer) OR mechanical ventilation” AND “randomized controlled trial* OR RCT*”. The search strategy are provided in Appendix 1–6.

2.3 Literature screening and data extraction

Storing and removing duplicate literature was done by EndNote X7 software. Two researchers were selected to screen the literature independently, and where there was disagreement, the agreement was reached after a discussion between the two researchers. Data collection included: authors, date of publication, number of cases, general patient information, AI algorithms, diagnostic criteria, classification models, labeling methods, processed images, features, database sources, etc., number of false positives (FP), sensitivity (Sen), number of true positives (TP), specificity (Spe), number of false negatives (FN), accuracy (Acc), number of true negatives (TN), etc.

2.4 Quality assessment of the included studies

The quality of the studies included in this study was assessed by reference to the QUADAS-2 tool, with the software Revman Manager(V5.3). The risk of bias was set in order according to the four components described on the website when analysing with reference to the QUADAS-2 tool, and this phase consisted of two main tasks, firstly comparing the information in the included literature against the QUADAS-2 device and answering each question with the available answers of "yes" After this has been completed, the second part of the work provides a ranking of the risk of bias according to the QUADAS-2 tool, together with information from the included literature as "high risk", "U", "U" and "U". High risk", "Unclear risk" or "Low risk", judged according to the following criteria: if the answer to all questions in a section is " If the answers to the questions in a section are all "yes", then "Low risk" can be selected, if there is a "no" answer in a section, then the risk of bias is considered high and "High risk "If "unclear" appears, the first two principles are followed, and if no result is found, the classification is based on "unclear risk". Finally, feedback on the risk of bias and quality score is provided in a quality assessment chart.

2.5 Statistical methods

Statistical analysis was carried out using Stata 16.0 software. The I^2 value test was used to determine the high or low heterogeneity to select an appropriate effect model. If I^2<50%, the heterogeneity among the included studies was considered inadequate, and a fixed-effects model was used for merging; if I^2≥50%, the heterogeneity among the included studies was deemed high a random-effects model should be used for connecting. A 2*2 four-compartment table of the AI-assisted diagnostic system for lung cancer diagnosis was presented for each included literature in turn, according to the gold standard of each included literature. The data from all the included literature were integrated to obtain the four-grid tables. The combined effect sizes were calculated, including combined sensitivity (Sen combined), combined specificity (Spe combined), positive likelihood ratio (+LR combined), negative likelihood ratio (-LR combined), diagnostic ratio (DOR combined), and 95% confidence interval (95% CI) of the above data, and the AUC was calculated by plotting the SROC curve. The diagnostic value of the AI-assisted diagnostic system for lung cancer was quantitatively evaluated using the above data. It was considered to have no diagnostic value when the AUC value was in the [0,0.5] range, a low diagnostic value when it was in the [0.5,0.7] field, a high diagnostic value when it was in the [0.7,0.9] degree, and a very high diagnostic value when it was above 0.9. Meta-regression analysis and subgroup analysis were then used to explore the sources of heterogeneity between the included studies. In the literature of the included studies, most of the data sets were divided into training and test sets to build models of AI-assisted diagnostic systems for lung cancer diagnosis. The training set is generally used to train the AI model’s ability to diagnose lung cancer, and the test set is used to test the AI model’s ability to diagnose lung cancer. In this study, the power of various AI models to diagnose lung cancer was studied, and therefore only the data from the test set was referred to for analysis. If the samples of part of the study did not distinguish between the training set and the test set, the entire selection was used for research by default.

3 Results

3.1 Literature search, screening process, and results

In total 3156 articles were obtained according to the search strategy from databases China Biomedical Literature Database (CBM), Wanfang Database, China Knowledge Network (CNKI), PubMed, EMbase, and the Cochrane Library, and 0 articles were obtained through other databases. After de-duplication, 2811 reports were obtained. From the initial screening of 469 articles according to the inclusion and exclusion criteria, 14 papers were accepted for inclusion in the qualitative analysis according to the inclusion and exclusion criteria. The literature included 5251 patients, including 2 papers in Chinese and 12 papers in English (Fig 2).

[Figure omitted. See PDF.]

3.2 Basic characteristics of the included studies and results of the quality assessment

The information of 28 included studies were concluded in Tables 2 and 3. To evaluate the quality of the 14 included studies, the risk of bias and the degree of relevance was assessed with the help of the QUADAS-2 tool combined with Revman software. In terms of index testing, about 40% of studies had a high risk of bias, and about 25% of the literature had an unclear risk of bias; in terms of the reference standard, about 40% of studies had an unclear risk of bias; and in terms of time, about 30% of studies had an unclear risk of bias (Fig 3).

[Figure omitted. See PDF.]

Red represents high degree of bias, yellow represents unclear and green represents a low degree of preference.

[Figure omitted. See PDF.]

[Figure omitted. See PDF.]

3.3 Meta-analysis results

To select an appropriate effect model for meta-analysis, the level of heterogeneity was measured by calculating the I^2 value. If I^2>50%, a high level of heterogeneity was considered to exist among the included literature and a random-effects model should be used for meta-analysis, if I^2<50%, a low level of heterogeneity was thought to live among the included literature meta-analysis could be performed using a fixed-effects model. The results showed that I^2 = 86.9% and I^2>50%, indicating a high heterogeneity among the included studies, and a random-effects model should be used for meta-analysis.

To investigate the diagnostic value of AI-assisted diagnosis for lung cancer diagnosis, the AI-assisted diagnosis system was evaluated by its merger sensitivity, merger specificity, positive likelihood ratio, negative likelihood ratio, diagnostic ratio, and AUC. The sensitivity and specificity of AI-assisted diagnosis were 0.87 [95% CI (0.82, 0.90)] and 0.87 [95% CI (0.82, 0.91)] respectively (Table 4, Figs 4 and 5), the underdiagnosis and misdiagnosis rates were 13% and 13% respectively. The AI-assisted diagnostic system had a high identification rate for lung cancer patients as well as non-lung cancer patients, and the diagnostic value of diagnosing lung cancer was high. According to the combined effect size of the AI-assisted diagnostic system for diagnosing lung cancer, the positive likelihood ratio and negative likelihood ratio were 6.5 [95% CI (4.6, 9.3)] and 0.15 [95% CI (0.11, 0.21)], respectively (Table 4), indicating that the AI-assisted diagnostic system had a high diagnostic value. For diagnosing lung cancer, patients were 6.5 times more likely to have a correct result than an incorrect result, and diagnosing non-lung cancer patients was 0.15 times more likely to have an incorrect result than a correct result, with a higher likelihood of AI being able to correctly diagnose lung cancer patients as well as non-lung cancer patients. Based on the combined effect size of the AI-assisted diagnostic system for lung cancer diagnosis, the diagnostic ratio of the AI-assisted diagnostic system for lung cancer diagnosis in this study was 43 [95% CI (24, 76)] (Table 4), the combined diagnostic ratio can indicate to some extent the strength of the relationship between diagnostic experimental results and disease, the higher the diagnostic ratio, the stronger the association, the combined DOR of this study was 43, indicating that AI-assisted diagnosis has a high diagnostic value for lung cancer. According to the SROC curve of AI-assisted diagnosis of lung cancer, the AUC was 0.93[95% CI (0.91, 0.95)] (Fig 6), indicating the high diagnostic value of AI for lung cancer.

[Figure omitted. See PDF.]

[Figure omitted. See PDF.]

[Figure omitted. See PDF.]

[Figure omitted. See PDF.]

To explore the sources of heterogeneity among the included literature, meta regressions were conducted the sample size of the test group was greater than 50 as covariates. The results of the regression analysis showed that the p-value of the group with a sample size greater than 50 was >0.5 (Table 5), indicating that the sample size greater than 50 was not associated with heterogeneity among the included literature.

[Figure omitted. See PDF.]

To investigate the relationship between the types of AI algorithms and the heterogeneity among the included literature, subgroup analysis was performed according to the kinds of AI algorithms (support vector machine, artificial neural network, random forest, and other algorithms), and the combined diagnostic ratio was used as the effect size to explore the source of heterogeneity. The results showed that the combined diagnostic ratio was significantly lower in the support vector machine group than in other groups (Fig 7), i.e., the use of support vector machine algorithms to assist in the diagnosis of benign malignancy in lung tumors was less effective, and studies using support vector machine algorithms were a possible source of heterogeneity.

[Figure omitted. See PDF.]

To test whether there was a publication bias in this study, a DEEK funnel plot was created by Stata 16.0 software. The results showed P = 0.33 (P>0.05) (Fig 8), indicating that there was no publication bias. The above results suggest that the AI-aided diagnosis system has a high diagnostic value for lung cancer and can be used to diagnose lung cancer in clinical practice.

[Figure omitted. See PDF.]

4 Discussion

Lung cancer is a malignant tumor originating from the mucosa or glands of the bronchi and has a high prevalence and mortality rate worldwide. Currently, CT of the chest is the most commonly used tool for lung cancer screening. Its high resolution can show the relationship of adjacent organs and blood vessels more clearly and plays a unique advantage in the early screening of lung cancer [40]. However, specific benign lesions, such as inflammation, tuberculosis, and necrosis, and some textures in the lung images, as well as some objective factors such as the experience of the film reviewer, may affect the accuracy of this method and may easily lead to misdiagnosis and omission. Since applying AI-assisted diagnostic systems to clinical work, a new page has been turned in the study of a lung cancer diagnosis. Some studies have shown [41–43] that AI technologies are being used more and more extensively in the practice of clinical diagnosis and treatment. These technologies are mainly applied to diagnose and analyze various medical images such as skin lesions, pathological microscopic images, and radiological data, and AI technologies are remarkable in their ability to improve diagnostic accuracy, stability, and work efficiency. The use of AI-assisted diagnosis of lung cancer has now become commonplace in clinical research and work.

The results of the meta-analysis showed that the combined sensitivity of the AI-aided diagnosis system for lung cancer diagnosis was 0.87 [95% CI (0.82, 0.90)], specificity was 0.87 [95% CI (0.82, 0.91)] (CI stands for confidence interval.), the missed diagnosis rate was 13%, the misdiagnosis rate was 13%, the positive likelihood ratio was 6.5 [95% CI (4.6, 9.3)], the negative likelihood ratio was 0.15 [95% CI (0.11, 0.21)], a diagnostic ratio of 43 [95% CI (24, 76)] and a sum of area under the combined subject operating characteristic (SROC) curve of 0.93 [95% CI (0.91, 0.95)]. Based on the results, the AI-assisted diagnostic system for CT (Computerized Tomography), imaging has considerable diagnostic accuracy for lung cancer diagnosis, which is of significant value for lung cancer diagnosis and has greater feasibility of realizing the extension application in the field of clinical diagnosis. In addition, the results of this study are also consistent with previous literature [44–46] reports, indicating that the results of this study have reference value, AI with the help of a deep learning model, can compensate for the missed diagnosis caused by physicians’ inexperience or incompetence, reduce the false-positive rate, and can improve the work efficiency to a certain extent. Therefore, this study suggests increasing the promotion of AI application in the clinical diagnosis of lung cancer.

AI-assisted diagnosis using different algorithms has different diagnostic outcomes. It has been shown [47] that traditional shallow learning algorithms are more advantageous for minor sample diseases than deep learning algorithms. However, for lung cancer lung nodules, a condition with a large amount of sample data, deep learning algorithms improve the diagnostic accuracy of lung cancer more than shallow learning algorithms improve the diagnostic accuracy of lung cancer. Different algorithms have different diagnostic capabilities, especially radionics and deep learning, which can help identify not only the benignity and malignancy of lung nodules but even predict the aggressiveness and the prognosis of small cell lung cancer of the lung. The results of the meta-regression analysis in this study showed that whether the sample size was greater than 50 were not possible sources of heterogeneity. The subgroup analysis concluded that the diagnostic value of support vector machine for lung cancer was significantly lower than that of other algorithms since the literature on AI-assisted diagnostic studies of lung cancer using support vector machine is small, so the results of this subgroup analysis may be related to study on support vector machine algorithm.

This study also has certain limitations: (1) The high heterogeneity among the original studies included in this study may also be related to the different sources of study subjects, the large sample size gap between studies, and the varying number of features extracted by AI, and the results are subject to further study. (2) This study excluded literature for which diagnostic data were not available in full, which may have biased the results. (3) A comprehensive search of relevant databases was conducted in this study, but only Chinese and English literature were included, which may be subject to language bias. (4) The original studies included in this study were mainly retrospective, and the quality of the original studies will affect the quality of the systematic evaluation.

Although the effectiveness of AI in lung cancer diagnosis has been initially verified, most of the advances in AI pathology diagnosis at this stage are still at the laboratory research stage and have not entered the clinic. Its limitations are manifested in image data quality, data integration, complex pathology diagnosis, legal liability definition, and cost of use. With the advancement of AI and digital pathology technology, more and more experienced pathologists are involved in AI for lung cancer pathology image annotation. It is believed that AI diagnostic systems will play a more significant role in the accurate pathology diagnosis of lung cancer.

In summary, the results of this study show that the AI-aided diagnosis system based on CT images has a high value in the diagnosis of lung cancer and can be promoted as a method to diagnose lung cancer in clinical applications. Integrating various data such as CT images, pathology, patient’s history, clinical features, physician’s diagnosis, and patient follow-up into the AI-assisted diagnosis system for all-round evaluation of patients is the future direction of AI development, which will not only improve the diagnostic accuracy of lung cancer and reduce physician’s workload but may also change the current medical model and promote the balanced development of medical resources in China.

Supporting information

S1 Checklist. PRISMA 2009 checklist.

https://doi.org/10.1371/journal.pone.0273445.s001

(DOC)

S1 Data. Minimum data set.

https://doi.org/10.1371/journal.pone.0273445.s002

(DOCX)

S1 Appendix.

https://doi.org/10.1371/journal.pone.0273445.s003

(ZIP)

Citation: Liu M, Wu J, Wang N, Zhang X, Bai Y, Guo J, et al. (2023) The value of artificial intelligence in the diagnosis of lung cancer: A systematic review and meta-analysis. PLoS ONE 18(3): e0273445. https://doi.org/10.1371/journal.pone.0273445

About the Authors:

Mingsi Liu

Contributed equally to this work with: Mingsi Liu, Jinghui Wu

Roles: Data curation, Software, Writing – original draft

Affiliation: Department of Computer and Artificial Intelligence, Zhengzhou University, Zhengzhou, Henan, China

Jinghui Wu

Contributed equally to this work with: Mingsi Liu, Jinghui Wu

Roles: Investigation, Resources, Writing – original draft, Writing – review & editing

Affiliation: College of Life Science, Sichuan University, Chengdu, Sichuan, China

Nian Wang

Roles: Writing – review & editing

Affiliation: School of Basic Medical Sciences, Chengdu Medical College, Chengdu, Sichuan, China

Xianqin Zhang

Roles: Writing – review & editing

Affiliation: School of Basic Medical Sciences, Chengdu Medical College, Chengdu, Sichuan, China

Yujiao Bai

Roles: Software

Affiliations: School of Basic Medical Sciences, Chengdu Medical College, Chengdu, Sichuan, China, Non-Coding RNA and Drug Discovery Key Laboratory of Sichuan Province, Chengdu Medical College, Chengdu, Sichuan, China

Jinlin Guo

Roles: Data curation

Affiliation: Chongqing Key Laboratory of Sichuan-Chongqing Co-construction for Diagnosis and Treatment of Infectious Diseases Integrated Traditional Chinese and Western Medicine, Chengdu University of Traditional Chinese Medicine, Chengdu, China

Lin Zhang

Roles: Methodology

E-mail: [email protected] (KT); [email protected] (LZ); [email protected] (SL)

Affiliation: Department of Pharmacy, Shaoxing people’s Hospital, Shaoxing, Zhejiang, China

ORICD: https://orcid.org/0000-0002-3767-0249

Shulin Liu

Roles: Resources

E-mail: [email protected] (KT); [email protected] (LZ); [email protected] (SL)

Affiliation: Department of the First Affiliated Hospital of Chengdu Medical College, Sichuan, China

Ke Tao

Roles: Supervision

E-mail: [email protected] (KT); [email protected] (LZ); [email protected] (SL)

Affiliation: College of Life Science, Sichuan University, Chengdu, Sichuan, China

ORICD: https://orcid.org/0000-0002-6440-2397

References

1. Amir GJ, Lehmann HP. After detection: The improved accuracy of lung cancer assessment using radiologic computer-aided diagnosis. Acad Radiol, 2021, 23(2): 186–191.

2. Schabath MB, Aberle DR. MILD trial, strong confirmation of lung cancer screening efficacy[J]. Nature Reviews Clinical Oncology. 2019,16(9): 529–530.

3. Naidich DP, Marshall CH, Gribbin C, et al. Lowdose CT of the lungs:preliminary observations[J]. Radiology, 2010, 175 (3): 729–731.

4. Nawa T, Nakagawa T, Kusano S, et al. Lung cancer screening u-sing low-dose spiral CT: results of baseline and 1-year follow-upstudies[J]. Chest, 2012, 122: 15–20.

5. Hubers A J,Heideman D A M.Duin S, et al. DNA hypermethylation analysis in sputum of asymptomatic subjects at risk for lung cancer participating in the NELSON trial: argument for maximum screening interval of 2 years[J]. Journal of Clinical Pathology, 2017, 70(3): 250.

6. Zhang L, Pu D, Liu D, et al. Identification and validation of novel circulating biomarkers for early diagnosis of lung cancer—ScienceDirect[J]. Lung Cancer, 2019, 135:130–137.

7. Hirsch FR, Franklin WA, Gazdar AG, et al. Early detection of lung cancer. clinical perspectives of recent advances in biologyc. Clin Can Res, 2011, 7: 5.

8. Dyabd E, Yang LC, CBA E, et al. Epidemiology of lung cancer and lung cancer screening programs in Ch ina and the United States[J]. Cancer Letters, 2021, 468:82–87.

9. Martin C, TammemOgi , Hormuzd A, et al. Selection Criteria forLung-Cancer Screening[J]. N Engl J Med, 2013, 368: 728–736.

10. Svensson MC, Svensson M, Nodin B, Borg D, Hedner C, Hjalmarsson C, et al. High Infiltration of CD68+/CD163- Macrophages Is an Adverse Prognostic Factor after Neoadjuvant Chemotherapy in Esophageal and Gastric Adenocarcinoma. J Innate Immun. 2022;14(6):615–628. Epub 2022 May 3. pmid:35504250; PMCID: PMC9801256.

11. Setio A, Traverso A, de Bel T, et al. Validation, comparison, and combination of algorithms for automatic detection of pulmonary no dules in computed tomography images:The LUNA16 challenge. Med Image Anal, 2017, 42:1–13.

12. Hosny A, Aerts HJWL. Artificial intelligence for global health. Science, 2019, 366(6468): 955–956. pmid:31753987

13. Kichenadasse G, Miners JO, Mangoni AA, et al. Association between body mass index and overall survival with immune checkpoint inhibitor therapy for JAMA Oncol, 2020, 6(4): 512–518.

14. Teramoto A, Tsukamoto T, Yamada A, et al. Deep learning approach to classification of lung cytological images: Two-step training using actual and synthesized images by progressive growing of generative adversarial networks [J]. PLoS One, 2020, 15(3):229–251.

15. Yu KH, Zhang C, Berry GJ, et al. Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features [J]. Nat Commun, 2016, 7: 12474.

16. Coudray N, Ocampo PS, Sakellaropoulos T, et al. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat Med, 2021, 24(10): 1559–1567.

17. Wang Y, Fu J, Wang Z, et al. Screening key lnc RNAs for human lung adenocarcinoma based on machine learning and weighted gene co-expression network. Cancer Biomark, 2019, 25(4): 313–324.

18. Chen WJ, Tang RX, He RQ, et al. Clinical roles of the aberrantly expressed IncRNAs in lung squamous cell carcinoma: a study based on RNA- sequencing and Oncotarget, 2017, 8(37): 61282–61304.

19. Coudray N, Ocampo PS, Sakellaropoulos T, et al. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning [J]. Nat Med, 2018, 24(10): 1559–1567.

20. Wang S, Wang T, Yang L, et al. ConvPath: a software tool for lung adenocarcinoma digital pathological image analysis aided by a convolutional neural network. EBio Medicine, 2019, 50: 103–110. pmid:31767541

21. Huang XF, Lei Q, Xie T,. Deep transfer convolutional neuralnetwork and extreme learning machine for lung nodule diagnosison CT images. Knowledge-Based Sys, 2020, 204(27): 106–230.

22. Rajpurkar P, Irvin J, Ball R L, et al. Deep learning for chest radiograph diagnosis: A retrospective comparison of the CheXNeXt algorithm to practicing radiologists[J]. PLoS medicine, 2018, 15(11): e1002686.

23. Dong LD, Huang G. Diagnostic value of artificial intelligence-assisted diagnostic system for pulmonary cancer based on CT images: A systematic review and meta-analysis of 4771 patients. Chinese Journal of Thoracic and Cardiovascular Surgery, 2021, 28(10): 1183–1192.

24. Xie Y, Zhang J, Xia Y. Semi-supervised adversarial model for benign–malignant lung nodule classification on chest CT[J]. Medical image analysis, 2019, 57: 237–248.

25. Hutton B, Salanti G, Caldwell D M, et al. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations[J]. Annals of internal medicine, 2015, 162(11): 777–784.

26. Chamberlin J, Kocher M R, Waltz J, et al. Automated detection of lung nodules and coronary artery calcium using artificial intelligence on low-dose CT scans for lung cancer screening: accuracy and prognostic value[J]. BMC medicine, 2021, 19(1):55.

27. Sun T, Zhang R, Wang J, et al. Computer-aided diagnosis for early-stage lung cancer based on longitudinal and balanced data[J]. PloS one, 2013, 8(5): e63559.

28. Teramoto A, Tsujimoto M, Inoue T, et al. Automated Classification of Pulmonary Nodules through a Retrospective Analysis of Conventional CT and Two-phase PET Images in Patients Undergoing Biopsy[J]. 2019.

29. Wang J, Liu X, Dong D, et al. Prediction of malignant and benign of lung tumor using a quantitative radiomic method[C]//2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 2016: 1272–1275.

30. Hsu YC, Tsai YH, Weng HH, et al. Artificial neural networks improve LDCT lung cancer screening: A comparative validation study[J]. BMC Cancer, 2020.

31. Li T, Li XD, Liu JY. Clinical value of artificial intelligence in the diagnosis of pulmonary nodules. Chinese General Practice, 2020, 023(007):828–831,836.

32. Randolph ShangDan, xiao-yu Chen Xu LP, SHANGDan CHENXiaoyu. Application of fuzzy neural network in CT diagnosis of lung cancer [J]. Journal of Zhengzhou University (Medical Edition), 2014(2): 191–194.

33. Dilger SK, Uthoff J, Judisch A, Hammond E, Mott SL, Smith BJ, et al. Improved pulmonary nodule classification utilizing quantitative lung parenchyma features. J Med Imaging (Bellingham). 2015 Oct;2(4):041004. Epub 2015 Sep 1. pmid:26870744; PMCID: PMC4748146.

34. Ren Y, Tsai MY, Chen L, Wang J, Li S, Liu Y, et al. A manifold learning regularization approach to enhance 3D CT image-based lung nodule classification. Int J Comput Assist Radiol Surg. 2020 Feb;15(2):287–295. Epub 2019 Nov 25. pmid:31768885.

35. Duan S, Cao H, Liu H, Miao L, Wang J, Zhou X, et al. Development of a machine learning-based multimode diagnosis system for lung cancer. Aging (Albany NY). 2020 May 23;12(10):9840–9854. Epub 2020 May 23. pmid:32445550; PMCID: PMC7288961.

36. Li X X, Li B, Tian L F, et al. Automatic benign and malignant classification of pulmonary nodules in thoracic computed tomography based on RF algorithm[J]. IET Image Processing, 2018, 12(7): 1253–1264.

37. Manikandan T, Bharathi N. Lung Cancer Detection Using Fuzzy Auto-Seed Cluster Means Morphological Segmentation and SVM Classifier. J Med Syst. 2016 Jul;40(7):181. Epub 2016 Jun 14. pmid:27299354.

38. da Silva G L F, da Silva Neto O P, Silva A C, et al. Lung nodules diagnosis based on evolutionary convolutional neural network[J]. Multimedia Tools and Applications, 2017, 76(18): 19039–19055.

39. Dilger S K N. The use of surrounding lung parenchyma for the automated classification of pulmonary nodules[D]. University of Iowa, 2013.

40. Hosny A, Parmar C, Quackenbush J, et al. Artificial intelligence in radiology[J]. Nature Reviews Cancer, 2021, 18(8): 500–510.

41. Li Z, Koban KC, Schenck TL, Giunta RE, Li Q, Sun Y. Artificial Intelligence in Dermatology Image Analysis: Current Developments and Future Trends. J Clin Med. 2022 Nov 18;11(22):6826. pmid:36431301; PMCID: PMC9693628.

42. Liu Kai, Li Qiong, Ma Jiechao, Evaluating a Fully Automated Pulmonary Nodule Detection Approach and Its Impact on Radiologist Performance [J]. Radiology:Artificial Intelligence. 2019, 1(3): 1–8.

43. Yan Y, Zhang JW, Zang GY, Pu J. The primary use of artificial intelligence in cardiovascular diseases: what kind of potential role does artificial intelligence play in future medicine? J Geriatr Cardiol. 2019 Aug;16(8):585–591. pmid:31555325; PMCID: PMC6748906.

44. Jacobs C, Van Rikxoort EM, Murphy K, et al. Computer aided detection of pulmonary nodules: a comparative study usingthe public LIDC/IDRI database[J]. Eur Radiol, 2016, 26(7): 2139–2147.

45. Gu Y, Lu X, Yang L, et al. Automatic lung nodule detection using a 3D deep convolutional neural network combined with amulti scale prediction strategy in chest CTs[J]. Comput Biol Med, 2018, 103 22; 0–231.

46. Setio AAA, Ciompi F, Litjens G, et al. Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks[J]. IEEE Trans Med Imaging, 2016, 35(5): 1160–1169.

47. Sun W, Zheng B, Qian W. Automatic feature learning using multichannel ROI based on deep structured algorithms or computerized lung cancer diagnosis. Comput Biol Med, 2017, 89: 530–539.