Role of machine learning in the management of epilepsy: a systematic review protocol

Abstract

Introduction

Machine learning is a rapidly expanding field and is already incorporated into many aspects of medicine including diagnostics, prognostication and clinical decision-support tools. Epilepsy is a common and disabling neurological disorder, however, management remains challenging in many cases, despite expanding therapeutic options. We present a systematic review protocol to explore the role of machine learning in the management of epilepsy.

Methods and analysis

This protocol has been drafted with reference to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) for Protocols. A literature search will be conducted in databases including MEDLINE, Embase, Scopus and Web of Science. A PRISMA flow chart will be constructed to summarise the study workflow. As the scope of this review is the clinical application of machine learning, the selection of papers will be focused on studies directly related to clinical decision-making in management of epilepsy, specifically the prediction of response to antiseizure medications, development of drug-resistant epilepsy, and epilepsy surgery and neuromodulation outcomes. Data will be extracted following the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies checklist. Prediction model Risk Of Bias ASsessment Tool will be used for the quality assessment of the included studies. Syntheses of quantitative data will be presented in narrative format.

Ethics and dissemination

As this study is a systematic review which does not involve patients or animals, ethics approval is not required. The results of the systematic review will be submitted to peer-review journals for publication and presented in academic conferences.

PROSPERO registration number

CRD42023442156.

Correspondence to Professor Patrick Kwan; [email protected]

STRENGTHS AND LIMITATIONS OF THIS STUDY

• This protocol proposes a systematic review on the role of machine learning in epilepsy management.

• The review will follow international recommendations with regard to reporting items, data extraction and quality assessment.

• Two independent reviewers will select studies, and an experienced adjudicator will resolve any disagreements.

• The review focuses mainly on the clinical application of machine learning. Selection criteria are aimed at recruiting studies directly related to clinical decision-making in epilepsy management. Papers on technical aspects of machine learning will not be included.

Introduction

The mainstay of epilepsy management is antiseizure medications (ASMs). The prescribing practice of ASMs via a trial-and-error approach has remained unchanged for decades. Although international guidelines recommend the ASM be individualised to each patient,^{1 2} this could be difficult clinically as there is no accurate biomarker which can reliably predict an individual’s responsiveness to a specific medication.³ Resultingly, the percentage of patients achieving 1-year seizure freedom on their first ASM has remained relatively unchanged over last 30 years, despite the surge of drug options.^{4 5} For a subset of patients with drug-resistant epilepsy (DRE), epilepsy surgery may offer a chance at seizure freedom. Epilepsy surgery is the most effective management for focal seizure control for surgically amenable patients, with up to two-thirds of patients with drug-resistant temporal lobe epilepsy attaining seizure freedom after surgery.⁶ However, only a small proportion of people with DRE are referred for a surgical workup and concerningly,⁷ there may be a long delay before surgical operation is considered.⁸

Machine learning algorithms have been developed to support clinicians in optimising the pharmacological and surgical management of epilepsy. The novel technique is well suited to supporting clinician decision-making as it can learn directly from data without human instruction and find hidden pattern in variables that humans and statistical methods cannot readily identify.³ This is evidenced by the broad applications of machine learning for epilepsy. Most machine learning is applied to detect and classify seizures, reflected in multiple narrative reviews and systematic reviews.^9–13 Similar amounts of reviews exist for machine learning and its utility for epilepsy neuroimaging diagnosis and prognostication.^14–17

In contrast, there are minimal reviews detailing the wide-ranging application of machine learning for epilepsy management. A systematic review addressing this gap is pertinent as the studies exploring this topic are heterogeneous and there is no universal framework used when building and assessing the validity of machine learning models.¹⁸ Thus, it would be beneficial to compare studies and examine the clinical utility of machine learning and epilepsy management decision-making. There will be a particular focus on the clinical aspect of machine learning and epilepsy because, despite the many machine learning algorithms developed, they rarely become a part of routine clinical care.^{19 20} However, machine learning shows potential with increased use in epilepsy decision-making as there are instances where it outperforms clinicians.²¹

Methods and analysis

The objective of the proposed systematic review is to assess the current role of machine learning in the epilepsy management. This protocol has been registered with the International Prospective Register of Systematic Reviews (PROSPERO). The structure of the protocol has been drafted according to the checklist of Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols 2015 statement.²²

Four well-established literature databases will be included in the literature search: MEDLINE, Embase, Scopus and Web of Science. They will be searched from inception to May 2023. There will be no limitations regarding publication date or language. Reference lists of each included paper will be manually searched for additional studies.

Search strategy will be formulated according to the population, intervention, control and outcomes framework (table 1). A concept map will be drafted to visually represent our search strategy. The search strategy will incorporate words or phrases selected from a controlled vocabulary, including MeSH of MEDLINE and ENTREE of Embase (please see online supplemental file 1). Also, keyword terms related to both concepts of machine learning and management of epilepsy will be included in the literature search. An academic librarian will be consulted before we finalised our search strategy. Covidence software will be used to manage the review process. Search results from the databases will be inputted into the software to exclude any duplication. Title-abstract screening will be performed independently by two reviewers (RS-kC and SN). Trial screening of 10 randomly selected papers will be attempted to ensure the concordance of the 2 reviewers before commencement of screening. After that, the two reviewers will proceed to full-text screening of the included papers. We will try to obtain the full paper by contacting the authors of papers which do not have full-text publication online. Any disagreement during title-abstract or full-text screens will be discussed by the two reviewers together. All unsettled conflicts will be settled by a senior adjudicator (ZC or EF).

Table 1

Elements of PICO framework and their contents

PICO elements	Element content
Population	People with epilepsy of all age groups
Intervention	Application of machine learning in epilepsy
Comparison	Conventional management without ML
Outcome	Performance metrics of ML (eg, AUC, sensitivity, specificity and accuracy) Clinical outcomes (eg, seizure- freedom rate) Patient-reported outcomes (eg, quality of life) Social and health economical outcomes

AUC, area under the curve; ML, machine learning.

Inclusion and exclusion criteria

Studies will be selected according to the following criteria:

Inclusion criteria:

• Original research including randomised controlled trials, cohort studies and case–control studies. Both prospective and retrospective study designs will be eligible for inclusion.

• Human-only studies.

• Studies conducted in populations with a clinician-confirmed diagnosis of epilepsy.

• Studies focused on clinical decision-making in the management of epilepsy, including:

  • Prediction of response to ASMs.

  • Prediction of DRE.

  • Prediction of epilepsy surgery/neuromodulation outcomes.

  • Selection of epilepsy surgery/neuromodulation candidates.

• Involve patients of any age and demographics.

Exclusion criteria:

• Studies on populations with isolated seizures, status epilepticus and psychogenic non-epileptic seizures.

• Studies with less than 10 participants.

• No clear direct relationship with clinical decision-making in the management of epilepsy.

• Studies on diagnosis, classification, comorbidities and prognostication (except surgical outcomes or drug responsiveness) of epilepsy.

• Studies on lateralisation or localisation of epileptic focus.

• Studies focused on the engineering and technical aspects of electroencephalogram (EEG) (including intracranial EEG and electrocorticogram) which have no direct relation to clinical decision-making in the management of epilepsy. This includes studies on developing machine learning algorithms for seizure and epileptiform discharges detection and epileptic focus localisation. However, studies that evaluated the use of these algorithms to support clinician’s decision-making for patient management will be included.

• Studies focused on the engineering and technical aspects of neuroimaging which have no direct relation to clinical decision-making in the management of epilepsy. This includes studies on developing machine learning algorithms for the analysis of structural or functional modalities. However, studies that evaluate the use of these algorithms to support clinician’s decision-making for patient management are eligible for inclusion.

• Studies on wearable devices or ECG.

• Studies that only employ traditional statistical models instead of machine learning techniques. This includes an explicit description of development, validation and performance metrics of machine learning model.

Data extraction

Reviewers will extract data of the included articles according to the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS).²³ Other specific information about machine learning such as algorithm and validation method, performance metrics such as area under curve, F1-score and accuracy will also be included in the data extraction. We will also extract information on the explainability of the machine learning models described in the included studies.

Risk of bias assessment

Risk of bias assessment will follow the Prediction model Risk Of Bias ASsessment Tool (PROBAST),²⁴ which is a published scheme for the quality assessment of diagnostic and prognostic prediction model studies. It contains 20 signalling questions under 4 domains, namely participants, predictors, outcome and analysis. Any discrepancies regarding the risk of bias will be settled by discussion between the reviewers until a consensus is reached. We will also summarise any methodological problems specific to machine learning such as lack of external validation.

Data synthesis and analysis

Narrative approach will be adopted to summarise the data. Tables or graphs will be constructed to illustrate the characteristics of included articles.

Ethics and dissemination

The current study is a protocol for a systematic review, ethical approval is not required. The study findings will be disseminated via conference presentation and submission to peer-reviewed journal for publication.

Patient and public involvement

No patient and public involvement is applicable to the design, delivery or dissemination of the findings of the proposed systematic review.

Discussion

Previous efforts have been dedicated to summarising the role of machine learning in management of epilepsy. Critical and narrative reviews have been published, but a systematic review is lacking.^{25 26} A systematic review on using machine learning in predicting outcomes of epilepsy surgery has been reported in abstract form.²⁷ To our best knowledge, our systematic review stands among the first to delve into machine learning’s roles in both pharmacological and non-pharmacological epilepsy management.

Machine learning has been applied to different fields of epilepsy encompassing diagnosis, management and prognostication. Given the expansive array of machine learning applications, our emphasis is directed towards the clinical management of epilepsy. However, even within this constrained scope, we anticipate encountering numerous articles pertaining to technical or engineering aspects of EEG or neuroimaging. Therefore, we have established a relatively stringent inclusion and exclusion criteria. We have performed a preliminary search and tried to triage the results according to our eligibility criteria. We consider our criteria effective in sorting out papers with clinical implications.

With reference to a critical review in this area by our team,²⁵ we focus on four areas including (1) selection of ASMs; (2) prediction of DRE; (3) prediction of epilepsy surgery/neuromodulation outcomes and (4) selection of epilepsy surgery/neuromodulation candidates. These four topics are comprehensive and cover most of the clinical uses of machine learning in epilepsy management. They also encompass the themes of the papers in our preliminary search and will make our review more organised. CHARMS guideline is a tool commonly employed for data extraction in systemic review involving predictive system including machine learning. Essential outcomes specific to the machine learning will be included in the data extraction also.

Machine learning models have been criticised for working as a ‘black box’. Stakeholders often do not understand how the prediction is made. It could become a hurdle in the clinical application of this technique. Efforts have been made to enhance the comprehensibility of the models. Noticeably, explainable artificial intelligence is a new trend.²⁸ We will summarise if the studies have addressed the transparency of their models.

Currently, there are no specific qualitative assessment guidelines for machine learning studies. PROBAST has been widely employed to assess risk of bias and applicability of prediction models such as machine learning.²⁹ There is an ongoing initiative to develop specific reporting guidelines in this field, an example is Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis specific to machine learning (TRIPOD-ML), which is an extension of popular TRIPOD guideline.³⁰

We hope our readers will gain insights into how machine learning could contribute to clinical management of epilepsy.

Ethics statements

Patient consent for publication

Not applicable.

Footnote

RS-kC and SN contributed equally.

Contributors The study concept and design were conceived by RS-kC. RS-kC and SN will conduct article screening and data extraction. RS-kC and SN will also perform data analysis. ZC, EF and PK gave advice during the process. RS-kC, SN, ZC, EF and PK drafted this manuscript, revised it for important content and have provided the final approval of this version. The corresponding author is the guarantor of the review.

Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Competing interests None declared.

Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

References

¹ National Institute of Health and Excellence. Epilepsies in children, young people and adults (NG217). 2022. Available: https://www.nice.org.uk/guidance/ng217

² Fisher RS, Acevedo C, Arzimanoglou A, et al. ILAE official report: a practical clinical definition of epilepsy. Epilepsia 2014; 55: 475–82. doi:10.1111/epi.12550

³ Chen Z, Rollo B, Antonic-Baker A, et al. New era of personalised epilepsy management. BMJ 2020; 371: m3658. doi:10.1136/bmj.m3658

⁴ Kwan P, Brodie MJ. Epilepsy after the first drug fails: substitution or add-on Seizure 2000; 9: 464–8. doi:10.1053/seiz.2000.0442

⁵ Chen Z, Brodie MJ, Liew D, et al. Treatment outcomes in patients with newly diagnosed epilepsy treated with established and new antiepileptic drugs: a 30-year longitudinal cohort study. JAMA Neurol 2018; 75: 279–86. doi:10.1001/jamaneurol.2017.3949

⁶ Englot DJ, Chang EF. Rates and predictors of seizure freedom in resective epilepsy surgery: an update. Neurosurg Rev 2014; 37: 389–404; doi:10.1007/s10143-014-0527-9

⁷ Engel J. The current place of epilepsy surgery. Curr Opin Neurol 2018; 31: 192–7. doi:10.1097/WCO.0000000000000528

⁸ Engel J, McDermott MP, Wiebe S, et al. Early surgical therapy for drug-resistant temporal lobe epilepsy: a randomized trial. JAMA 2012; 307: 922–30. doi:10.1001/jama.2012.220

⁹ Maimaiti B, Meng H, Lv Y, et al. An overview of EEG-based machine learning methods in seizure prediction and opportunities for neurologists in this field. Neuroscience 2022; 481: 197–218. doi:10.1016/j.neuroscience.2021.11.017

¹⁰ Rasheed K, Qayyum A, Qadir J, et al. Machine learning for predicting epileptic seizures using EEG signals: a review. IEEE Rev Biomed Eng 2021; 14: 139–55. doi:10.1109/RBME.2020.3008792

¹¹ Natu M, Bachute M, Gite S, et al. Review on epileptic seizure prediction: machine learning and deep learning approaches. Comput Math Methods Med 2022; 2022: 7751263. doi:10.1155/2022/7751263

¹² Siddiqui MK, Morales-Menendez R, Huang X, et al. A review of epileptic seizure detection using machine learning classifiers. Brain Inform 2020; 7: 5. doi:10.1186/s40708-020-00105-1

¹³ Ahmad I, Wang X, Zhu M, et al. EEG-based epileptic seizure detection via machine/deep learning approaches: a systematic review. Comput Intell Neurosci 2022; 2022: 6486570. doi:10.1155/2022/6486570

¹⁴ Sone D, Beheshti I. Clinical application of machine learning models for brain imaging in epilepsy: a review. Front Neurosci 2021; 15: 684825. doi:10.3389/fnins.2021.684825

¹⁵ Yuan J, Ran X, Liu K, et al. Machine learning applications on neuroimaging for diagnosis and prognosis of epilepsy: a review. J Neurosci Methods 2022; 368: 109441. doi:10.1016/j.jneumeth.2021.109441

¹⁶ Shoeibi A, Moridian P, Khodatars M, et al. An overview of deep learning techniques for epileptic seizures detection and prediction based on neuroimaging modalities: methods, challenges, and future works. Comput Biol Med 2022; 149: 106053. doi:10.1016/j.compbiomed.2022.106053

¹⁷ Sollee J, Tang L, Igiraneza AB, et al. Artificial intelligence for medical image analysis in epilepsy. Epilepsy Res 2022; 182: 106861. doi:10.1016/j.eplepsyres.2022.106861

¹⁸ Scott I, Carter S, Coiera E. Clinician checklist for assessing suitability of machine learning applications in healthcare. BMJ Health Care Inform 2021; 28: e100251. doi:10.1136/bmjhci-2020-100251

¹⁹ de Jong J, Cutcutache I, Page M, et al. Towards realizing the vision of precision medicine: AI based prediction of clinical drug response. Brain 2021; 144: 1738–50. doi:10.1093/brain/awab108

²⁰ Kelly CJ, Karthikesalingam A, Suleyman M, et al. Key challenges for delivering clinical impact with artificial intelligence. BMC Med 2019; 17: 195. doi:10.1186/s12916-019-1426-2

²¹ Devinsky O, Dilley C, Ozery-Flato M, et al. Changing the approach to treatment choice in epilepsy using big data. Epilepsy Behav 2016; 56: 32–7. doi:10.1016/j.yebeh.2015.12.039

²² PRISMA-P Group, Moher D, Shamseer L, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 2015; 4. doi:10.1186/2046-4053-4-1

²³ Moons KGM, de Groot JAH, Bouwmeester W, et al. Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist. PLoS Med 2014; 11: e1001744. doi:10.1371/journal.pmed.1001744

²⁴ Wolff RF, Moons KGM, Riley RD, et al. PROBAST: a tool to assess the risk of bias and applicability of prediction model studies. Ann Intern Med 2019; 170: 51–8. doi:10.7326/M18-1376

²⁵ Smolyansky ED, Hakeem H, Ge Z, et al. Machine learning models for decision support in epilepsy management: a critical review. Epilepsy Behav 2021; 123: 108273. doi:10.1016/j.yebeh.2021.108273

²⁶ Abbasi B, Goldenholz DM. Machine learning applications in epilepsy. Epilepsia 2019; 60: 2037–47. doi:10.1111/epi.16333

²⁷ Chia CWL, Bhatia S, Shastin D, et al. Machine learning for outcome prediction in epilepsy surgery: a systematic review. Br J Surg 2021; 108: 175. doi:10.1093/bjs/znab259.657

²⁸ Adadi A, Berrada M. Peeking inside the black-box: a survey on explainable artificial intelligence (XAI). IEEE Access 2018; 6: 52138–60. doi:10.1109/ACCESS.2018.2870052

²⁹ Andaur Navarro CL, Damen JAA, Takada T, et al. Risk of bias in studies on prediction models developed using supervised machine learning techniques: systematic review. BMJ 2021; 375: n2281. doi:10.1136/bmj.n2281

³⁰ Collins GS, Moons KGM. Reporting of artificial intelligence prediction models. Lancet 2019; 393: 1577–9. doi:10.1016/S0140-6736(19)30037-6