Integrating artificial intelligence into lung cancer screening: a randomised controlled trial protocol

Abstract

Introduction

Lung cancer (LC) is the most common cause of cancer-related deaths worldwide. Its early detection can be achieved with a CT scan. Two large randomised trials proved the efficacy of low-dose CT (LDCT)-based lung cancer screening (LCS) in high-risk populations. The decrease in specific mortality is 20%–25%.

Nonetheless, implementing LCS on a large scale faces obstacles due to the low number of thoracic radiologists and CT scans available for the eligible population and the high frequency of false-positive screening results and the long period of indeterminacy of nodules that can reach up to 24 months, which is a source of prolonged anxiety and multiple costly examinations with possible side effects.

Deep learning, an artificial intelligence solution has shown promising results in retrospective trials detecting lung nodules and characterising them. However, until now no prospective studies have demonstrated their importance in a real-life setting.

Methods and analysis

This open-label randomised controlled study focuses on LCS for patients aged 50–80 years, who smoked more than 20 pack-years, whether active or quit smoking less than 15 years ago. Its objective is to determine whether assisting a multidisciplinary team (MDT) with a 3D convolutional network-based analysis of screening chest CT scans accelerates the definitive classification of nodules into malignant or benign. 2722 patients will be included with the aim to demonstrate a 3-month reduction in the delay between lung nodule detection and its definitive classification into benign or malignant.

Ethics and dissemination

The sponsor of this study is the University Hospital of Nice. The study was approved for France by the ethical committee CPP (Comités de Protection des Personnes) Sud-Ouest et outre-mer III (No. 2022-A01543-40) and the Agence Nationale du Medicament et des produits de Santé (Ministry of Health) in December 2023. The findings of the trial will be disseminated through peer-reviewed journals and national and international conference presentations.

Trial registration number

NCT05704920.

Correspondence to Dr Jacques Boutros; [email protected]

STRENGTHS AND LIMITATIONS OF THIS STUDY

• This is a prospective national multicentre randomised trial assessing the role of a machine-learning analysis of chest CT for lung cancer screening following a long series of promising retrospective studies.

• Insufficient power of this study to show clinical benefit (ie, reduction in lung cancer mortality).

• Absence of a definitive histological diagnosis in all patients with nodules.

Background and rationale

Lung cancer (LC) is the leading cause of cancer deaths worldwide,¹ with 33 000 occurring in France in 2018.² After two landmark randomised controlled trials, LC screening (LCS) using low-dose CT (LDCT) was demonstrated to be efficacious. The American National Lung Screening Trial (NLST)^{3 4} demonstrated a significant 20% reduction in LC mortality and the Belgo-Dutch Nelson trial^{5 6} showed that 130 patients needed to be screened to prevent one LC-related death over 10 years of follow-up. In addition, they found a significant all-cause decrease in mortality. In the USA, the United States Preventive Services Task Force (USPSTF) recommends screening for adults aged 50–80 years, who have a 20-pack-year smoking history and currently smoke or have quit within the past 15 years.⁶ American⁷ and European^{8 9} guidelines have paved the way to organised screening. Nonetheless, LCS in France is still opportunistic, with France’s national Health Authority (Haute Authorité de Santé) currently encouraging pilot LCS programmes using LDCT.¹⁰ A recent Cochrane review paper including 11 trials¹¹ confirmed a reduction in LC‐related mortality of LDCT screening for high‐risk populations. However, this later review concluded that there exists limited data on the harm related to overdiagnosis, false positives and the excess of screening-related diagnostic intervention. The Nelson trial performed growth-rate assessment for indeterminate lung nodules (ILNs), which reduced the high 24% false positive rate reported in the NLST to only 2%. Therefore, the current challenge is to reduce the false positive rate while maintaining a high sensitivity.

Artificial intelligence (AI) has recently been developed for medical imaging and deep learning (DL) is the fastest-growing approach in this regard, which has many radiological applications that aid the field, specifically thoracic CT.¹² The main purpose of DL for LCS is to help clinicians detect and classify lung nodules more accurately and quickly. In the last 15 years, computer-aided detection and computer-aided diagnosis systems have therefore been developed extensively.¹³ Various DL techniques were retrospectively tested including the Scale Invariant Feature Transform (SIFT)-based classifier, the Support Vector Machine (SVM), multiscale convolutional network (CNN), and the multicrop CNN, which are impressively accurate, exceeding 95% in some cases .¹² However, the effect on medical decision-making of the AI-based estimated malignancy risk has not been studied prospectively.¹⁴

Therefore, we have planned to prospectively test the impact of a DL-trained lung nodule malignancy assessment software ScanRads, on the decision-making of multidisciplinary teams (MDTs). Our working hypothesis is that AI provides physicians with operational decision support for LCS candidates who are found to have an ILN and thus shortens the indeterminacy period.

Methods

Objectives and study design

The primary objective of this study, namely ‘Da Capo’, is to compare the ‘time to definitive classification’ of lung nodules (ie, benign or malignant) detected during LCS, using two management strategies: ‘multidisciplinary team-only’ versus ‘MDT+AI’.

The secondary objectives will include evaluation and comparison of the diagnostic performance of the two management strategies for lung nodule detection during screening, versus the gold standard; comparison of the rate of invasive procedures between the two management strategies; evaluation of the medicoeconomic impact of the two management strategies; and evaluation of the predictive character four-marker protein panel for LC diagnosis at 2 years of follow-up.¹⁵

Da Capo is a prospective national multicentre open-label randomised study with two experimental arms: (1) a standard-of-care (SOC) and (2) an interventional arm. Volunteers will be enrolled in a screening programme including a consultation, where electronic medical records will be collected, and current smokers will receive smoking cessation advice and intervention. Volunteers will perform a first LDCT (T0). When an ILN is found, the patient will be randomised into the SOC or interventional arm. If no ILN is found, a second round of LDCT (T1) will be performed 1 year later and randomisation takes place again if an ILN is found during this second round.

In the SOC arm, the MDT will classify the nodule as benign, indeterminate or malignant, and management will be based on the French society of pulmonary diseases guidelines for lung nodule management.⁹

In the interventional arm, the MDT will classify the nodule as benign, indeterminate or malignant while knowing the ILN analysis made with ScanRads, expressed as a malignancy score on a scale of 1 to 10, in ascending order of the chance of malignancy. In this interventional arm, nodule management will be based on the same guidelines as in the SOC arm.⁹

In France, as required by the ‘haute authorité de santé’, for thoracic oncology MDTs, a minimum of a thoracic surgeon, a radiologist, a radiotherapist, an oncologist and a pulmonologist must attend.

As we did in our previous national study,^{16 17} the gold standard to decide on the definitive nature (benign or malignant) of an ILN will be the histological diagnosis. In the absence of histology, it is the radiological evolution of the nodule (stable or increasing in size) at the end of 24 months of follow-up that will allow a decision to be made between malignant and benign. Ground-glass nodules stable in size over 24 months of follow-up will be categorised as ‘indeterminate’. These nodules will not be included in the diagnostic performance analysis as the benignity of stable ground-glass nodules over 24 months is not guaranteed and therefore does not meet the defined gold standard. It should be noted that the classification of a nodule by the MDT may be different from the real nature of the nodule. In patients without histology at 24 months of follow-up, a chest CT will be systematically done.

A blood sample (two tubes of 10 mL Cell-Free DNA BCT, Streck, Omaha, USA) will be taken on a voluntary basis during the screening and will be sent within 72 hours to the Nice Hospital Biobank (BB-0033–00025) (http://www.biobank-cotedazur.fr/). The non-exclusive purpose of this sample will be to evaluate the predictive character of the appearance of LC at 2 years of follow-up of a four-marker protein panel¹⁵

Endpoints

Primary

This includes comparison of time to definitive classification of ILN (ie, benign or malignant) between the SOC arm (MDT-only) and the interventional arm (MDT+AI) with the log-rank test.

If the nodule remains indeterminate at the end of the study period (24 months), the patient’s data are censored for analysis. This may occur in particular with ground-glass nodules that are not diagnosed as malignant at the end of follow-up.

Secondary

Sensitivity, specificity, positive predictive value and negative predictive value of the two management strategies will be compared with the gold standard; it also includes comparison of invasiveness and the complication rate of diagnostic procedures between the two management strategies as well as comparative cost-consequence analysis of the two management strategies.

ScanRad

ScanRad is a software developed by the Université Côte d’Azur (Nice, France) that generates, for each nodule, a malignancy score based on a DL algorithm: 3D CNN applied to images. The algorithm was trained on CT scans of 1103 patients from the NLST study, and the results were presented at the European Congress of Radiology 2020.¹⁸

Its operating principle is based on machine learning. During a training phase, the algorithm relies on supervised learning techniques to estimate a risk of malignancy of pulmonary nodules. For this, a training dataset consisting of annotated scanners is used. The annotations consist in particular of the location of the nodule(s) on the scanner as well as their associated label (ie, ‘malignant’ or ‘benign’). From these data, the training consists in inferring the particularities, making it possible to detect a nodule and to characterise its state, in other words, to estimate whether it is malignant or benign. The medical device trained in this way can then be used to analyse new datasets corresponding, for example, to patients monitored for LCS. (figure 1)

Figure 1. ScanRad training principle. FP, false positive; TP, true positive.

Inclusion and procedures

Participants who meet all the inclusion criteria and none of the exclusion criteria will be enrolled by a designated investigator from each centre, after signing the written informed consent. Inclusion will be performed online using the dedicated Da Capo-secured web-based platform, which has been developed to centralise all patient data, images, LDCT reports, ScanRads analyses of the images and MDT reports. Only participants with a 6–30 mm lung nodule found on the first (T0) or second (T1) LDCT will be randomised. Inclusion and exclusion criteria are shown in table 1. The study flow diagram is displayed in figure 2.

Figure 2. Flow diagram of the trial. ILN, indeterminate lung nodule; Interv, interventional arm; LDCT, low-dose chest CT scan; SOC, standard of care.

Table 1

Inclusion and exclusion criteria

Inclusion criteria

Exclusion criteria

Age between 50 and 80 years Active smoker or ex-smoker who quit less than 15 years ago Smoking history of at least 20 pack-years Capacity to give informed consent Affiliated to the French social security

Clinical signs suggestive of cancer. Chest CT within 12 months before enrolment. Radiological abnormality requiring follow-up or additional investigations. Health problems considerably limiting life expectancy. Health problems limiting the possibility of obtaining a histological diagnosis. History of neoplasia treated less than 5 years ago, except basal cell carcinoma of the skin or localised prostate cancer with radical treatment. Vulnerable people: adults under guardianship or curatorship. Medical and/or psychiatric problems possibly limiting adherence to the study.

Recruitment

Information to the general public about the trial will be delivered via a public information campaign in the investigating centres participating in the study, via social networks and a dedicated web-based application named Depiscann (http://depiscann.dacapo-preprod.ascan.io/) that allows citizens to test their eligibility for LCS and via a local advertising campaign among family practitioners and the regional daily press.

Statistical analysis and sample size

We considered that with the SOC strategy, the estimated median time between the discovery of a solid ILN and the definitive classification by MDT into benign or malignant is 12 months and that the addition of ScanRads analysis could reduce this median time to 9 months. This 3-month improvement interval is not based on trials. First, it is based on clinical pertinence as evaluated by thoracic oncologists in France. Also, the 3 months delay is the usual minimal interval between follow-up chest CT scans in patients with a suspicious nodule.

Considering that a two-sided test has a power of 90% and a level of significance of 5%, the estimated number of subjects with a solid ILN is 324 (log-rank test, nQuery Advisor, V.9.1). The proportion of subjects with a solid ILN is estimated at 25% at baseline or on the first year of follow-up. Considering that there will be 5% of possible loss to follow-up, it will be necessary to include 2722 volunteers in the study.

In the absence of a decision by the end of 24 months of follow-up, patients will be censored on this date. Data from patients lost to follow-up or who died during the 24 months of follow-up after the discovery of a ILN will be censored at the last date of follow-up. The time to definitive classification will be described for each group using Kaplan-Meier survival curves and compared using the log-rank test.

Collection of data and monitoring

The usual demographic data will be recorded and an electronic case report form will be created. All LDCTs will be anonymised and stored in electronic format for further centralised analysis. The MDT decision will be stored along with all follow-up visits. Access to nominative data will be strictly reserved to the referring physician, radiologist and doctors constituting the MDT of the corresponding centre.

Quality control will be done by clinical research monitors appointed by the sponsor. The nature and frequency of monitoring will be based on the rate of inclusion.

They will check for the accuracy and completeness of the case report form entries, source documents and other trial-related records. They will verify that written informed consent was obtained and that the trial follows the currently approved protocol/amendment(s) in each centre, with good clinical practice and with the applicable regulatory requirement(s).

Adverse events

An adverse event (AE) is defined as any untoward medical occurrence happening during the participation of a subject in the trial, independently of the relationship with the study-related interventions and procedures. A serious adverse event (SAE) is defined as any AE that results in death, is life-threatening, requiring participants’ hospitalisation or prolongation of an existing hospitalisation and that results in persistent or significant disability/incapacity. Each AE must be judged by the investigator and the sponsor, for assessment of the severity of the causal link between AE and the procedure and the character expected or unexpected.

A suspected unexpected severe adverse reaction is an adverse reaction that is both unexpected (not consistent with the study-related interventions and procedures) and also meets the definition of an SAE. According to the law of 9 August 2004 of the Code of Public Health, any occurrence of an SAE will immediately be reported to the sponsor. The sponsor will declare SAE likely to be related to biomedical research to the Ethics Committees (EC) and to the French ‘Agence Nationale du Medicament et des produits de Santé’ (ANSM) without delay and no later than 7 calendar days in case of death or life-threatening situations and at the latest within 15 days for the other SAEs.

AEs and SAEs might occur in patients in whom further investigations will be programmed in case of abnormalities on chest CTs.

Ethics, regulatory clearances and dissemination

The study sponsor is the University Hospital of Nice. The study was approved for France on 14 December 2022 by the ethical committee CPP Sud-Ouest et outre-mer III (No. 2022-A01543-40) and on 21 December 2022 by the ANSM (Ministry of Health) in December 2023 (N° IDRCB 2022-A01543-40-A). ClinicalTrial.gov no: NCT05704920. The findings of the trial will be disseminated through peer-reviewed journals and national and international conference presentations.

Patient involvement

The patient and members of the public are not involved so far at any stage of the trial.

Data sharing statement

The individual participant data collected during the trial will be shared after deidentification. Data will be available since publication and until 5 years following publication, for researchers who provide a methodologically sound proposal.

Milestones

The first study participant will be enrolled on April 2024. The estimated study completion date is in 2029.

Discussion

AI is a game changer for various medical fields. Nonetheless, some of its impressive retrospective results need to be demonstrated prospectively, which is due to many methodological issues including trials comparing the diagnostic performance of DL models and that of healthcare professionals often lacking external validation.¹⁹ In addition, the performance and utility of a machine-learning algorithm is unpredictable because it depends on the quality and relevance of the data on which it is trained.²⁰ One main driver of AI system malfunction, known as ‘dataset shift’ occurs when a machine-learning system underperforms because of a mismatch between the training dataset and the data on which it is applied.²¹ Therefore, until the beginning of the trial, ScanRads will be enriched by real-life chest CTs with annotated nodules.

Another interesting observation is that this prospective trial will explore the clinicians’ behaviour in case of mismatch between the predictions of the model and their own clinical judgement, a facet that is seldom explored in the literature. Will the clinicians be influenced and dragged into inaccuracies when counting on AI? If that is the case, we might find a shorter time to definitive classification of nodules in the interventional arm but with a lower specificity.

As with all the studies into LCS, this trial will have one main limitation, that is the absence of a definitive histological diagnosis in patients.

We hope that by the end of our recruitment, we will be able to tell if incorporating DL models into MDT discussions can improve their performance and accelerate the time to nodule classification and therefore to appropriate decisions.

Ethics statements

Patient consent for publication

Not applicable.

Footnote

Contributors JoB, PH, NR, EF, SyL, BP, CHM and JaB designed the study, JaB and CHM wrote the paper with input from all authors, and StL developed the artificial intelligence software.

Funding Institut National du Cancer, Conseil Départemental 06, Maskini Foundation, Fondation du Souffle, AstraZeneca, Award/Grant number is not applicable.

Competing interests None declared.

Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Provenance and peer review Not commissioned; externally peer reviewed.

References

¹ Ferlay J, Colombet M, Soerjomataram I, et al. Estimating the global cancer incidence and mortality in 2018: GLOBOCAN sources and methods. Int J Cancer 2019; 144: 1941–53. doi:10.1002/ijc.31937

² Institut national du cancer. Les cancers en France en 2018 - L’Essentiel des Faits et Chiffres (Édition 2019). 2019. Available: https://www.e-cancer.fr/Expertises-et-publications/Catalogue-des-publications/Les-cancers-en-France-en-2018-L-essentiel-des-faits-et-chiffres-edition-2019 [Accessed 01 Jan 2023 ].

³ Aberle DR, Adams AM, Berg CD, et al. Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med 2011; 365: 395–409. doi:10.1056/NEJMoa1102873

⁴ National Lung Screening Trial Research Team. Lung cancer incidence and mortality with extended follow-up in the National lung screening trial. J Thorac Oncol 2019; 14: 1732–42. doi:10.1016/j.jtho.2019.05.044

⁵ de Koning HJ, van der Aalst CM, de Jong PA, et al. Reduced lung-cancer mortality with volume CT screening in a randomized trial. N Engl J Med 2020; 382: 503–13. doi:10.1056/NEJMoa1911793

⁶ Jonas DE, Reuland DS, Reddy SM, et al. Screening for lung cancer with low-dose computed tomography: updated evidence report and systematic review for the US preventive services task force. JAMA 2021; 325: 971–87. doi:10.1001/jama.2021.0377

⁷ Wood DE, Kazerooni EA, Aberle D, et al. NCCN guidelines® insights: lung cancer screening, version 1.2022: featured updates to the NCCN guidelines. J Natl Compr Canc Netw 2022; 20: 754–64. doi:10.6004/jnccn.2022.202glup

⁸ Kauczor H-U, Baird A-M, Blum TG, et al. ESR/ERS statement paper on lung cancer screening. Eur Respir J 2020; 55: 1900506. doi:10.1183/13993003.00506-2019

⁹ Couraud S, Ferretti G, Milleron B, et al. Recommandations de l’Intergroupe francophone de cancérologie thoracique, de la société de pneumologie de langue Française, et de la société d’Imagerie thoracique sur le dépistage du cancer bronchopulmonaire par tomodensitométrie À faible dose d’Irradiation. Revue Des Maladies Respiratoires 2021; 38: 310–25. doi:10.1016/j.rmr.2021.02.003

¹⁰ Haute Autorité de Santé. Dépistage du cancer bronchopulmonaire par scanner thoracique faible dose sans injection : actualisation de l’avis de 2016. 2021. Available: https://www.has-sante.fr/jcms/p_3310940/fr/depistage-du-cancer-bronchopulmonaire-par-scanner-thoracique-faible-dose-sans-injection-actualisation-de-l-avis-de-2016 [Accessed 01 Jan 2023 ].

¹¹ Bonney A, Malouf R, Marchal C, et al. Impact of low-dose computed tomography (LDCT) screening on lung cancer-related mortality. Cochrane Database Syst Rev 2022; 8: CD013829. doi:10.1002/14651858.CD013829.pub2

¹² Wang L. Deep learning techniques to diagnose lung cancer. Cancers 2022; 14: 5569. doi:10.3390/cancers14225569

¹³ Lee JH, Hwang EJ, Kim H, et al. A narrative review of deep learning applications in lung cancer research: from screening to prognostication. Transl Lung Cancer Res 2022; 11: 1217–29. doi:10.21037/tlcr-21-1012

¹⁴ Schreuder A, Scholten ET, van Ginneken B, et al. Artificial intelligence for detection and characterization of pulmonary nodules in lung cancer CT screening: ready for practice. Transl Lung Cancer Res 2021; 10: 2378–88. doi:10.21037/tlcr-2020-lcs-06

¹⁵ Fahrmann JF, Marsh T, Irajizad E, et al. Blood-based biomarker panel for personalized lung cancer risk assessment. JCO 2022; 40: 876–83. doi:10.1200/JCO.21.01460

¹⁶ Leroy S, Benzaquen J, Mazzetta A, et al. Circulating tumour cells as a potential screening tool for lung cancer (the AIR study): protocol of a prospective multicentre cohort study in France. BMJ Open 2017; 7: e018884. doi:10.1136/bmjopen-2017-018884

¹⁷ Marquette C-H, Boutros J, Benzaquen J, et al. Circulating tumour cells as a potential biomarker for lung cancer screening: a prospective cohort study. Lancet Respir Med 2020; 8: 709–16. doi:10.1016/S2213-2600(20)30081-3

¹⁸ Lopez S, Fillard P, Diascorn Y, et al. Lung screening assistance: how to cut down the false-positive rate and detect lung cancer earlier (a retrospective study). ECR 2020 - Conference Paper;

¹⁹ Liu X, Faes L, Kale AU, et al. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. Lancet Digit Health 2019; 1: e271–97. doi:10.1016/S2589-7500(19)30123-2

²⁰ Wilkinson J, Arnold KF, Murray EJ, et al. Time to reality check the promises of machine learning-powered precision medicine. Lancet Digit Health 2020; 2: e677–80. doi:10.1016/S2589-7500(20)30200-4

²¹ Finlayson SG, Subbaswamy A, Singh K, et al. The clinician and dataset shift in artificial intelligence. N Engl J Med 2021; 385: 283–6. doi:10.1056/NEJMc2104626