Incidentally found resectable lung cancer with the usage of artificial intelligence on chest radiographs

Abstract

Purpose

Detection of early lung cancer using chest radiograph remains challenging. We aimed to highlight the benefit of using artificial intelligence (AI) in chest radiograph with regard to its role in the unexpected detection of resectable early lung cancer.

Materials and methods

Patients with pathologically proven resectable lung cancer from March 2020 to February 2022 were retrospectively analyzed. Among them, we included patients with incidentally detected resectable lung cancer. Because commercially available AI-based lesion detection software was integrated for all chest radiographs in our hospital, we reviewed the clinical process of detecting lung cancer using AI in chest radiographs.

Results

Among the 75 patients with pathologically proven resectable lung cancer, 13 (17.3%) had incidentally discovered lung cancer with a median size of 2.6 cm. Eight patients underwent chest radiograph for the evaluation of extrapulmonary diseases, while five underwent radiograph in preparation of an operation or procedure concerning other body parts. All lesions were detected as nodules by the AI-based software, and the median abnormality score for the nodules was 78%. Eight patients (61.5%) consulted a pulmonologist promptly on the same day when the chest radiograph was taken and before they received the radiologist’s official report. Total and invasive sizes of the part-solid nodules were 2.3–3.3 cm and 0.75–2.2 cm, respectively.

Conclusion

This study demonstrates actual cases of unexpectedly detected resectable early lung cancer using AI-based lesion detection software. Our results suggest that AI is beneficial for incidental detection of early lung cancer in chest radiographs.

Introduction

Lung cancer is the second most commonly diagnosed cancer, with an estimated 2.2 million new cancer cases and 1.8 million deaths worldwide in 2020 [1]. It is also the leading cause of morbidity and mortality from cancer in both men and women [2]. Although the 5-year relative survival rate for lung cancer in South Korea increased from 16.6% in 2001–2005 to 34.7% in 2013–2019 [3], it still remains significantly lower than that of other cancers. This is probably because most lung cancer patients are diagnosed at an advanced stage as there are no obvious specific symptoms in the early stage of the disease [4]. The clinical stage at presentation has the greatest impact on the prognosis of lung cancer, which indicates that early detection and screening methods are of paramount importance [5].

According to the National Lung Screening Trial, which was published in 2011, lung cancer screening with annual low-dose computed tomography (CT) reduced the mortality of lung cancer by approximately 20% among those at high risk [6]. However, as screening is indicated for high-risk individuals with a smoking history of more than 30 pack-years, the current screening guidelines may miss many lung cancer patients, especially non-smokers [7, 8].

Chest radiograph is the most commonly utilized imaging modality in clinical practice; however, its use for lung cancer screening is still controversial [9]. Although there are concerns that chest radiograph may overlook malignant pulmonary nodules owing to its lower sensitivity than CT [10], some studies have also demonstrated its potential role in screening for lung cancer [11, 12]. A recent study by Koo et al. [13] showed that early lung cancer was detected in 38% of patients receiving chest radiographs versus 26% of those without surveillance. Furthermore, they found that surveillance with chest radiographs at intervals of less than 3 years was an independent predictor of improved survival in the women in the screened population.

In recent years, artificial intelligence (AI) was emerged as a new solution for lesion detection in medical imaging, especially for chest radiographs [14]. Many recent studies have developed and validated their own AI algorithms for chest radiographs [15, 16], which have been tested to increase the detection rate of abnormal findings, such as pulmonary nodules [17–20]. Recently, several softwares have been approved for commercial use to assist imaging interpretation in clinical practice [20–22]. However, the actual clinical adoption of AI and its usefulness have still not been verified, and many researchers have stated that proving its contribution in real-world clinical practice is necessary for the next step [21].

In the current study, we have presented actual cases of incidentally detected lung cancer in a hospital where all chest radiographs are immediately analyzed by an AI-based lesion detection software. The objective of this study was to highlight the actual benefit of AI in chest radiograph with respect to its role in the unexpected detection of resectable early lung cancer.

Materials and methods

Participants

The Institutional Review Board of Yongin Severance Hospital approved this retrospective study, and the requirement for informed consent was waived (IRB No. 2021-0491-001). Patients with pathologically confirmed lung cancer after surgical resection from March 2020 to February 2022 were retrospectively reviewed. Among them, the patients with incidentally detected resectable lung cancer were included in this study. We defined the incidentally detected resectable lung cancer as follows; (1) patients who had visited outpatient clinic except for pulmonology or thoracic surgery department for the evaluation or treatment of extrapulmonary diseases, (2) discovered lung nodule on initial chest radiographs by AI and confirmed it as lung cancer after the resection. We excluded patients who had initially visited the pulmonology or thoracic surgery department, were referred from another hospital owing to the suspicion of a lung abnormality in chest imaging, visited another department owing to respiratory symptoms, or underwent a health check-up or screening for lung cancer.

We retrospectively reviewed the medical records of the patients to describe the entire process of detecting lung cancer by using AI in chest radiographs. In addition to the age and sex of the included patients, information regarding the final pathologic diagnosis, tumor location, size, characteristics, TNM stage, and stage, which was based on the AJCC 8^th edition, were evaluated using the pathologic reports after surgical resection.

Chest radiographs and the applied AI-based lesion detection software

All included patients had undergone chest radiograph, which was followed by CT scanning owing to suspected lung abnormalities that were observed in the radiographs. We collected the data regarding the reasons for undergoing the chest radiograph in the first place, the department that order the tests, and radiologists’ evaluation of the radiographs. Because our general hospital used a commercially available AI-based lesion detection software (Lunit INSIGHT CXR, version 2 and 3, Lunit Inc., Korea) for all chest radiographs from March 2020, we additionally assessed the AI-based results using a ResNet34-based software, which is approved for usage to analyze chest radiographs of adults [21, 23]. It was integrated into the picture archiving and communicating system (PACS) to analyze the chest radiographs to detect specific lung lesions immediately after the chest radiographs were taken.

Two versions of the software were integrated into the PACS during the study period. Version 2 was integrated for the detection of three lesions (pneumothorax, nodule, and consolidation) from March 2020 to February 2021. Lesion location was displayed in the chest radiographs by a color heatmap when the abnormality score (probability of an abnormal lesion existing) was above the preset operating point of 15%. The total abnormality score was displayed as a representative value for each radiograph by selecting the highest score for three lesions.

From March 2021 to February 2022, the upgraded version 3 was applied, and the number of detectable lesions was increased to eight (pneumothorax, nodule, consolidation, fibrosis, atelectasis, cardiomegaly, pleural effusion, and pneumoperitoneum). Additionally, this version displayed details of the detected lesion and its abnormality score separately with a grayscale contour map for localization. These two versions of software were used in different time periods and the upgraded version 3 had the function of detecting additional six lesions. Lung nodule was detected identically in these two versions and integration of different versions in two years did not affect the method of lung nodule detection.

The software was applied to chest radiographs of all patients over 18 years old. As soon as the patient underwent chest radiograph, the image was sent to the server for processing the AI-based analyzed results. Then, the AI-based results were attached to the original radiograph as a secondary series simultaneously. Clinicians could see the AI-based results as soon as the patients underwent chest radiograph just by scrolling down to the images on PACS (Fig 1).

[Figure omitted. See PDF.]

For the included patients, we assessed the abnormality score of AI for the detected lesion in each chest radiograph. In addition, the interval between radiograph and consultation with a pulmonologist was evaluated. Whether the consultation with pulmonologists occurred before or after receiving the radiologists’ official report was assessed based on the reading time and consultation time recorded in the electronic medical records.

Results

From March 2020 to February 2022, 75 patients were confirmed to have lung cancer on pathology following lung resection in our hospital. Among them, 13 patients (17.3%, male:female = 9:3, median = 65 years, range, 52–79 years) had incidentally detected resectable lung cancer. The remaining 62 patients were excluded because they initially visited the pulmonology or thoracic surgery department because of known lung abnormalities. Table 1 summarizes the demographic and clinical characteristics of these patients.

[Figure omitted. See PDF.]

At first, all patients were suspected to have lung abnormalities based on their initial chest radiographs, which were ordered by cardiologists (n = 4), orthopedic surgeons (n = 3), gastroenterologists (n = 2), neurologist (n = 1), neurosurgeon (n = 1), general surgeon (n = 1), and psychiatrist (n = 1). Five patients (38.5%) underwent chest radiograph before an operation or procedure involving other body parts, such as cholecystectomy, spine, or knee surgery. The remaining eight patients had undergone chest radiograph for the evaluation or treatment of extrapulmonary diseases, such as coronary artery disease, stroke, gastritis, and schizophrenia. All lesions were detected as nodules by the AI-based software, and the median abnormality score for the nodules was 78% (range, 26–95%). In eight of 13 patients (61.5%), pulmonologists were consulted for the abnormal chest radiograph findings on the same day when the chest radiographs were taken. In addition, these eight patients had received consultation before receiving the radiologist’s official report regarding the chest radiographs.

After resection of the nodules, pathologic results confirmed the diagnosis of adenocarcinoma (n = 10, 76.9%), squamous cell carcinoma (n = 2), and small cell lung cancer (n = 1). Seven patients (53.8%) were in stage I, five patients were in stage II, and one was in stage III. According to the pathologic reports, the median size of the lesions was 2.6 cm (range, 1.9–5.5 cm). Most of the nodules were solid, but three lesions were part-solid (IA1 stage in #2 patient and IA3 stage in #1 and #8 patients). Total and invasive sizes of the part-solid nodules were 2.3–3.3 cm and 0.75–2.2 cm, respectively, and they were all detected as nodules by AI. Figs 2–4 showed chest radiograph, immediate AI-based results, and chest CT images of representative three lung cancer patients (Figs 2–4).

[Figure omitted. See PDF.]

(A) Chest radiograph was taken before performing cholecystectomy for chronic cholecystitis. (B) Artificial intelligence-based lesion detection software detected a lung nodule in the left upper lobe with an abnormality score of 37%. (C) After cholecystectomy, chest CT demonstrated a part-solid nodule in the left upper lobe (arrow), and pathology confirmed adenocarcinoma after surgical resection (total tumor size of 2.3 cm; invasive size of 0.75 cm). The final stage was IA1 (pT1aN0M0). The patient is now undergoing routine follow-up in the outpatient clinic without additional treatment.

[Figure omitted. See PDF.]

(A) Chest radiograph was taken at the time of admission to the psychiatric ward for schizophrenia. (B) Artificial intelligence-based lesion detection software detected a lung nodule in the left lower lung field with an abnormality score of 81%. (C) Chest CT demonstrated an irregular solid nodule in the left lower lobe (arrow), and adenocarcinoma of approximately 1.9 cm was confirmed after surgical resection. The final stage was IA2 (pT1bN0M0), and she is now receiving routine follow-up for lung cancer.

[Figure omitted. See PDF.]

(A) Chest radiograph was taken in the cardiomegaly outpatient clinic for the evaluation of arrhythmia. (B) Artificial intelligence-based lesion detection software detected a lung nodule in the right lower lobe with an abnormality score of 26%. (C) Chest CT demonstrated an irregular solid mass in the posterior subpleural space of the right lower lobe superior segment (arrow). After surgical resection, adenocarcinoma with an approximate size of 2.8 cm and the final stage of IA3 (pT1cN0M0) was confirmed.

Discussion

Among all patients with resectable lung cancer, 13 patients (17.3%) had incidentally detected lung cancer, and all of them had first undergone chest radiographs. Regardless of the size and characteristics of the nodules, AI-based lesion detection software detected all of the lesions as nodules with a median abnormality score of 78%. Eight patients underwent chest radiographs for the evaluation of extrapulmonary diseases, while the remaining five underwent radiographs in preparation of an operation or procedure concerning other body parts and were not expected to have pulmonary problems when they first underwent the chest radiographs. Even though they were not expected to have lung lesions, 61.5% of patients had a prompt consultation with a pulmonologist on the same day as the chest radiograph examination and before receiving the official radiologists’ report.

Seven patients (53.8%) were in stage I, which meant no additional treatment was required after surgery. This is clinically very important in terms of detecting early lung cancer, which could have been missed if the physicians had not paid attention to the screening chest radiographs, and it has great implications in that AI analysis could help prompt decision-making during this process. Chest radiograph is a basic examination that is performed countless times a day in an emergency room, outpatient clinic, preoperative, and inpatient screening examination even if there are no respiratory symptoms. However, in actual clinical practice, many doctors are implementing the real-time medical practice of reading the radiographs immediately before receiving the official radiologists’ report, and even if an abnormal finding is reported by the radiologists, it may be often overlooked or not considered in a timely manner. According to a previous study, about 20% of lung cancer patients had evidence of lung cancer in previous chest radiographs, which was missed at that time [24]. In the present study, 61.5% of patients had a prompt consultation with pulmonologists on the same day as the chest radiograph, suggesting that AI-based analysis in real-time could be valuable as a supporting tool for decision-making in clinical practice.

In the developing field of AI in radiology, chest radiograph is the most commonly adopted imaging modality for AI [25]. This is because chest radiograph is the most frequently performed imaging study, which results in a huge workload for radiologists who spend most of their working time on the interpretation of special studies such as ultrasonography, CT, or magnetic resonance imaging. However, chest radiograph is still a fundamental investigation for screening for diseases or preparing for treatment, and it has the power to guide the management of patients. Early and accurate diagnosis through chest radiograph is important because critical diseases, such as lung cancer or pneumothorax, can still be initially detected in chest radiographs. Lack of enough radiologists for prompt interpretation of chest radiographs at the outpatient clinic or emergency department may be a problem for clinicians who need to make urgent decisions based on chest radiographs. Therefore, AI has emerged as a novel solution as a computer-aided diagnosis system for chest radiographs. In our study, 8 patients had consultation before receiving the official radiologists’ report. Even though we could not compare the effect of official reports on patient management or survival due to the small number of the patients, this could be a reason why AI integration for chest radiographs has received more attention by researchers and doctors.

Especially for lung nodules, chest radiograph has limited sensitivity in the detection of lung cancer by radiologists, reported as 36–84% according to the nodule’s size and radiologist’s experience [26]. However, recent studies have examined the potential of utilizing AI for screening of lung nodules on chest radiographs and reported a sensitivity of 52–87% [26, 27]. According to them, radiologists or clinicians could have a better diagnostic performance when aided by AI for detecting malignant lung nodules [27–29]. Even though several studies have focused on developing and validating their own algorithm or commercially approved software for detecting lung nodules, most of them were retrospective studies simulating clinical situations [27, 28, 30, 31]. Clinical demand for validating AI in real-world practice is increasing nowadays, and according to a recent consensus statement, validation of AI for real-world clinical practice should be performed to confirm its usefulness [21]. Therefore, our study is meaningful because it showed how AI could actually play a useful role in the decision-making process of clinicians, especially in lung nodule detection tasks.

In our study, the median size of the resected lung cancers was 2.6 cm, and most of them were solid nodules. However, three cases were part-solid nodules, which are known to be difficult to detect in chest radiographs. In addition, one patient (#2 patient) had invasive lung adenocarcinoma of 0.75 cm size, which was also detected by the AI-based software. Further research on the effectiveness of AI for the detection of lung nodules and early lung cancer in actual clinical practice is necessary.

There are several limitations of our study. First, we did not analyze all chest radiographs that were interpreted to have a nodule by AI and did not review whether these nodules were actually proven to be lung cancer. Since this study evaluated cases with surgically resected lung cancer, it was difficult to demonstrate the diagnostic performance of AI for lung nodule detection and its usefulness for screening early lung cancer. In addition, cases of unexpectedly detected unresectable advanced lung cancer were not included in this study. Since it was not possible to compare before and after the introduction of AI, it was difficult to determine whether the introduction of AI actually increased the diagnosis rate of lung cancer and improved the patient’s prognosis. However, our study tried to highlight the usefulness of AI-based software for the detection of unexpected early lung cancer in a timely manner. Second, a controversy could exist regarding whether the clinicians referred to the radiologists’ reports or AI-based results at the time of consultation. We assumed that the clinicians were aided by the AI-based results immediately after the images were taken, based on the time recorded in the medical records. Whether the number or time of consultations was changed after AI usage could not be analyzed in this study. Nevertheless, our results are meaningful because they show how AI could aid in the decision-making process of clinicians, especially in lung cancer detection. Further prospective studies with large populations are needed to prove the clinical efficacy of AI.

Conclusions

This study examined the actual cases of unexpectedly detected early lung cancer, which were all detected by AI-based lesion detection software as nodules. About 62% of the patients had a consultation with a pulmonologist for their abnormal chest radiograph findings on the same day when the radiograph was taken and before receiving the radiologists’ official report. With the increasing use of AI in medical imaging, the actual benefit of AI in chest radiographs for the detection of unexpected early lung cancer needs to be validated.

Supporting information

S1 File. Data file.

https://doi.org/10.1371/journal.pone.0281690.s001

(XLSX)

Acknowledgments

The authors thank Medical Illustration & Design, part of the Medical Research Support Services of Yonsei University College of Medicine, for all artistic support related to this work.

Citation: Kwak SH, Kim E-K, Kim MH, Lee EH, Shin HJ (2023) Incidentally found resectable lung cancer with the usage of artificial intelligence on chest radiographs. PLoS ONE 18(3): e0281690. https://doi.org/10.1371/journal.pone.0281690

About the Authors:

Se Hyun Kwak

Roles: Data curation, Investigation, Project administration, Resources, Visualization, Writing – original draft, Writing – review & editing

Affiliation: Division of Pulmonology, Department of Internal Medicine, Allergy and Critical Care Medicine, Yongin Severance Hospital, Yonsei University College of Medicine, Yongin-si, Gyeonggi-do, Republic of Korea

Eun-Kyung Kim

Roles: Data curation, Investigation, Project administration, Resources, Software, Supervision, Writing – review & editing

Affiliation: Department of Radiology, Research Institute of Radiological Science and Center for Clinical Imaging Data Science, Yongin Severance Hospital, Yonsei University College of Medicine, Yongin-si, Gyeonggi-do, Republic of Korea

Myung Hyun Kim

Roles: Data curation, Project administration, Resources, Software

Affiliation: Department of Radiology, Research Institute of Radiological Science and Center for Clinical Imaging Data Science, Yongin Severance Hospital, Yonsei University College of Medicine, Yongin-si, Gyeonggi-do, Republic of Korea

Eun Hye Lee

Contributed equally to this work with: Eun Hye Lee, Hyun Joo Shin

Roles: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

E-mail: [email protected] (EHL); [email protected] (HJS)

Affiliations: Division of Pulmonology, Department of Internal Medicine, Allergy and Critical Care Medicine, Yongin Severance Hospital, Yonsei University College of Medicine, Yongin-si, Gyeonggi-do, Republic of Korea, Center for Digital Health, Yongin Severance Hospital, Yonsei University College of Medicine, Yongin-si, Gyeonggi-do, Republic of Korea

Hyun Joo Shin

Contributed equally to this work with: Eun Hye Lee, Hyun Joo Shin

Roles: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

E-mail: [email protected] (EHL); [email protected] (HJS)

Affiliations: Department of Radiology, Research Institute of Radiological Science and Center for Clinical Imaging Data Science, Yongin Severance Hospital, Yonsei University College of Medicine, Yongin-si, Gyeonggi-do, Republic of Korea, Center for Digital Health, Yongin Severance Hospital, Yonsei University College of Medicine, Yongin-si, Gyeonggi-do, Republic of Korea

ORICD: https://orcid.org/0000-0002-7462-2609

References

1. Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA: a cancer journal for clinicians. 2021;71: 209–249 pmid:33538338

2. Korea S. Annual Report on the Causes of Death Statistics. http://kostat.go.kr/portal/korea/kor_nw/1/6/2/index.board.

3. Welfare MoHa. Annual report of cancer statistics in Korea in 2017. https://www.cancer.go.kr/.

4. Allemani C, Matsuda T, Di Carlo V, Harewood R, Matz M, Nikšić M, et al. Global surveillance of trends in cancer survival 2000–14 (CONCORD-3): analysis of individual records for 37 513 025 patients diagnosed with one of 18 cancers from 322 population-based registries in 71 countries. Lancet. 2018;391: 1023–1075. pmid:29395269

5. Goldstraw P, Chansky K, Crowley J, Rami-Porta R, Asamura H, Eberhardt WE, et al. The IASLC Lung Cancer Staging Project: Proposals for Revision of the TNM Stage Groupings in the Forthcoming (Eighth) Edition of the TNM Classification for Lung Cancer. J Thorac Oncol. 2016;11: 39–51. pmid:26762738

6. Aberle DR, Adams AM, Berg CD, Black WC, Clapp JD, Fagerstrom RM, et al. Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med. 2011;365: 395–409. pmid:21714641

7. Moyer VA, Force* UPST. Screening for lung cancer: US Preventive Services Task Force recommendation statement. Annals of internal medicine. 2014;160: 330–338

8. Yang P, Wang Y, Wampfler JA, Xie D, Stoddard SM, She J, et al. Trends in subpopulations at high risk for lung cancer. Journal of Thoracic Oncology. 2016;11: 194–202 pmid:26811226

9. van Beek EJ, Mirsadraee S, Murchison JT. Lung cancer screening: Computed tomography or chest radiographs? World journal of radiology. 2015;7: 189 pmid:26339461

10. Quadrelli S, Lyons G, Colt H, Chimondeguy D, Buero A. Clinical Characteristics and Prognosis of Incidentally Detected Lung Cancers. International Journal of Surgical Oncology. 2015;2015: 287604. pmid:25685550

11. van Beek EJ, Mullan B, Thompson B. Evaluation of a real-time interactive pulmonary nodule analysis system on chest digital radiographic images: a prospective study. Academic radiology. 2008;15: 571–575 pmid:18423313

12. White CS, Flukinger T, Jeudy J, Chen JJ. Use of a computer-aided detection system to detect missed lung cancer at chest radiography. Radiology. 2009;252: 273–281 pmid:19561261

13. Koo HJ, Choi CM, Park S, Lee HN, Oh DK, Ji WJ, et al. Chest radiography surveillance for lung cancer: Results from a National Health Insurance database in South Korea. Lung Cancer. 2019;128: 120–126. pmid:30642443

14. Kwon T, Lee SP, Kim D, Jang J, Lee M, Kang SU, et al. Diagnostic performance of artificial intelligence model for pneumonia from chest radiography. PLoS One. 2021;16: e0249399. pmid:33857181

15. Ausawalaithong W, Thirach A, Marukatat S, Wilaiprasitporn T. Automatic lung cancer prediction from chest X-ray images using the deep learning approach. 2018 11th Biomedical Engineering International Conference (BMEiCON): IEEE; 2018. pp. 1–5.

16. Tataru C, Yi D, Shenoyas A, Ma A. Deep Learning for abnormality detection in Chest X-Ray images. IEEE Conference on Deep Learning; 2017.

17. Nam JG, Park S, Hwang EJ, Lee JH, Jin K-N, Lim KY, et al. Development and validation of deep learning–based automatic detection algorithm for malignant pulmonary nodules on chest radiographs. Radiology. 2019;290: 218–228 pmid:30251934

18. Rajpurkar P, Irvin J, Ball RL, Zhu K, Yang B, Mehta H, et al. Deep learning for chest radiograph diagnosis: A retrospective comparison of the CheXNeXt algorithm to practicing radiologists. PLoS medicine. 2018;15: e1002686 pmid:30457988

19. Yoo H, Kim KH, Singh R, Digumarthy SR, Kalra MK. Validation of a Deep Learning Algorithm for the Detection of Malignant Pulmonary Nodules in Chest Radiographs. JAMA Network Open. 2020;3: e2017135–e2017135. pmid:32970157

20. Shin HJ, Son NH, Kim MJ, Kim EK. Diagnostic performance of artificial intelligence approved for adults for the interpretation of pediatric chest radiographs. Sci Rep. 2022;12: 10215. pmid:35715623

21. Hwang EJ, Goo JM, Yoon SH, Beck KS, Seo JB, Choi BW, et al. Use of Artificial Intelligence-Based Software as Medical Devices for Chest Radiography: A Position Paper from the Korean Society of Thoracic Radiology. Korean J Radiol. 2021;22: 1743–1748. pmid:34564966

22. Lee S, Shin HJ, Kim S, Kim EK. Successful Implementation of an Artificial Intelligence-Based Computer-Aided Detection System for Chest Radiography in Daily Clinical Practice. Korean J Radiol. 2022;23: 847–852. pmid:35762186

23. Kim SJ, Roh JW, Kim S, Park JY, Choi D. Current State and Strategy for Establishing a Digitally Innovative Hospital: Memorial Review Article for Opening of Yongin Severance Hospital. Yonsei Med J. 2020;61: 647–651. pmid:32734727

24. Quekel LG, Kessels AG, Goei R, van Engelshoven JM. Miss rate of lung cancer on the chest radiograph in clinical practice. Chest. 1999;115: 720–724. pmid:10084482

25. Chassagnon G, Vakalopoulou M, Paragios N, Revel MP. Artificial intelligence applications for thoracic imaging. Eur J Radiol. 2020;123: 108774. pmid:31841881

26. Tandon YK, Bartholmai BJ, Koo CW. Putting artificial intelligence (AI) on the spot: machine learning evaluation of pulmonary nodules. J Thorac Dis. 2020;12: 6954–6965. pmid:33282401

27. Nam JG, Park S, Hwang EJ, Lee JH, Jin KN, Lim KY, et al. Development and Validation of Deep Learning-based Automatic Detection Algorithm for Malignant Pulmonary Nodules on Chest Radiographs. Radiology. 2019;290: 218–228. pmid:30251934

28. Sim Y, Chung MJ, Kotter E, Yune S, Kim M, Do S, et al. Deep Convolutional Neural Network-based Software Improves Radiologist Detection of Malignant Lung Nodules on Chest Radiographs. Radiology. 2020;294: 199–209. pmid:31714194

29. Armato SG 3rd. Deep Learning Demonstrates Potential for Lung Cancer Detection in Chest Radiography. Radiology. 2020;297: 697–698. pmid:32965172

30. Homayounieh F, Digumarthy S, Ebrahimian S, Rueckel J, Hoppe BF, Sabel BO, et al. An Artificial Intelligence-Based Chest X-ray Model on Human Nodule Detection Accuracy From a Multicenter Study. JAMA Netw Open. 2021;4: e2141096. pmid:34964851

31. Yoo H, Lee SH, Arru CD, Doda Khera R, Singh R, Siebert S, et al. AI-based improvement in lung cancer detection on chest radiographs: results of a multi-reader study in NLST dataset. Eur Radiol. 2021;31: 9664–9674. pmid:34089072