Abstract

Alzheimer’s disease (AD) is a progressive neurodegenerative condition marked by a decline in cognitive functions with no validated disease modifying treatment. It is critical for timely treatment to detect AD in its earlier stage before clinical manifestation. Mild cognitive impairment (MCI) is an intermediate stage between cognitively normal older adults and AD. To predict conversion from MCI to probable AD, we applied a deep learning approach, multimodal recurrent neural network. We developed an integrative framework that combines not only cross-sectional neuroimaging biomarkers at baseline but also longitudinal cerebrospinal fluid (CSF) and cognitive performance biomarkers obtained from the Alzheimer’s Disease Neuroimaging Initiative cohort (ADNI). The proposed framework integrated longitudinal multi-domain data. Our results showed that 1) our prediction model for MCI conversion to AD yielded up to 75% accuracy (area under the curve (AUC) = 0.83) when using only single modality of data separately; and 2) our prediction model achieved the best performance with 81% accuracy (AUC = 0.86) when incorporating longitudinal multi-domain data. A multi-modal deep learning approach has potential to identify persons at risk of developing AD who might benefit most from a clinical trial or as a stratification approach within clinical trials.

Details

Title
Predicting Alzheimer’s disease progression using multi-modal deep learning approach
Author
Lee, Garam 1 ; Nho Kwangsik 2 ; Kang Byungkon 3 ; Sohn Kyung-Ah 3 ; Kim Dokyoon 4 ; Weiner, Michael W 5 ; Aisen, Paul 6 ; Petersen, Ronald 7 ; Jack, Clifford R, Jr 7 ; Jagust, William 8 ; Trojanowki, John Q 9 ; Toga, Arthur W 10 ; Beckett, Laurel 11 ; Green, Robert C 12 ; Saykin, Andrew J 13 ; Morris, John 14 ; Shaw, Leslie M 9 ; Khachaturian Zaven 15 ; Sorensen, Greg 16 ; Carrillo, Maria 17 ; Kuller Lew 18 ; Raichle, Marc 14 ; Paul, Steven 19 ; Davies, Peter 20 ; Fillit Howard 21 ; Hefti Franz 22 ; Holtzman Davie 14 ; Marcel, Mesulam M 23 ; Potter, William 24 ; Snyder, Peter 25 ; Montine, Tom 26 ; Thomas, Ronald G 6 ; Donohue, Michael 6 ; Walter, Sarah 6 ; Sather Tamie 6 ; Jiminez Gus 6 ; Balasubramanian, Archana B 6 ; Mason, Jennifer 6 ; Sim, Iris 6 ; Harvey, Danielle 11 ; Bernstein, Matthew 7 ; Fox, Nick 27 ; Thompson, Paul 28 ; Schuff Norbert 5 ; DeCArli, Charles 11 ; Borowski Bret 7 ; Gunter, Jeff 7 ; Senjem Matt 7 ; Vemuri Prashanthi 7 ; Jones, David 7 ; Kantarci Kejal 7 ; Ward, Chad 7 ; Koeppe, Robert A 29 ; Foster, Norm 30 ; Reiman, Eric M 31 ; Chen, Kewei 31 ; Mathis, Chet 18 ; Landau, Susan 8 ; Cairns, Nigel J 14 ; Householder, Erin 14 ; Taylor-Reinwald, Lisa 14 ; Lee, Virginia 28 ; Korecka Magdalena 28 ; Figurski Michal 28 ; Crawford, Karen 10 ; Neu, Scott 10 ; Foroud, Tatiana M 13 ; Potkin, Steven 32 ; Shen, Li 13 ; Faber, Kelley 13 ; Kim, Sungeun 13 ; Lean, Tha 16 ; Frank, Richard 33 ; Hsiao, John 34 ; Kaye, Jeffrey 35 ; Quinn, Joseph 35 ; Silbert, Lisa 35 ; Lind, Betty 35 ; Carter, Raina 35 ; Dolen, Sara 35 ; Ances Beau 14 ; Carroll, Maria 14 ; Creech, Mary L 14 ; Franklin, Erin 14 ; Mintun Mark A 14 ; Schneider, Stacy 14 ; Oliver, Angela 14 ; Schneider, Lon S 10 ; Pawluczyk Sonia 10 ; Beccera Mauricio 10 ; Liberty, Teodoro 10 ; Spann, Bryan M 10 ; Brewer, James 6 ; Vanderswag Helen 6 ; Fleisher, Adam 6 ; Marson, Daniel 36 ; Griffith, Randall 36 ; Clark, David 36 ; Geldmacher, David 36 ; Brockington, John 36 ; Roberson, Erik 36 ; Love, Marissa Natelson 37 ; Heidebrink, Judith L 8 ; Lord, Joanne L 8 ; Mason, Sara S 7 ; Albers, Colleen S 7 ; Knopman, David 7 ; Johnson, Kris 7 ; Grossman, Hillel 38 ; Mitsis Effie 39 ; Shah, Raj C 40 ; deToledo-Morrell Leyla 40 ; Doody, Rachelle S 41 ; Villanueva-Meyer, Javier 41 ; Chowdhury Munir 41 ; Rountree, Susan 41 ; Dang, Mimi 41 ; Duara Ranjan 42 ; Varon, Daniel 42 ; Greig, Maria T 42 ; Roberts, Peggy 42 ; Stern, Yaakov 43 ; Honig, Lawrence S 43 ; Bell, Karen L 43 ; Albert, Marilyn 37 ; Onyike Chiadi 37 ; D’Agostino Daniel II 37 ; Kielb, Stephanie 37 ; Galvin, James E 44 ; Cerbone Brittany 44 ; Michel, Christina A 44 ; Pogorelec, Dana M 44 ; Rusinek, Henry 44 ; de Leon Mony J 44 ; Glodzik Lidia 44 ; De, Santi Susan 44 ; Womack, Kyle 45 ; Mathews, Dana 45 ; Quiceno, Mary 45 ; Murali, Doraiswamy P 46 ; Petrella, Jeffrey R 46 ; Borges-Neto, Salvador 46 ; Wong, Terence Z 46 ; Coleman, Edward 46 ; Levey, Allan I 47 ; Lah, James J 47 ; Cella, Janet S 47 ; Burns, Jeffrey M 48 ; Swerdlow, Russell H 48 ; Brooks, William M 48 ; Arnold, Steven E 9 ; Karlawish, Jason H 9 ; Wolk, David 9 ; Clark, Christopher M 9 ; Apostolova Liana 28 ; Tingus Kathleen 28 ; Woo, Ellen 28 ; Silverman, Daniel H, S 28 ; Lu, Po H 28 ; Bartzokis, George 28 ; Smith, Charles D 49 ; Jicha, Greg 49 ; Hardy, Peter 49 ; Sinha Partha 49 ; Oates, Elizabeth 49 ; Conrad, Gary 49 ; Graff-Radford, Neill R 50 ; Parfitt, Francine 50 ; Kendall, Tracy 50 ; Johnson, Heather 50 ; Lopez, Oscar L 18 ; Oakley, MaryAnn 18 ; Simpson, Donna M 18 ; Farlow, Martin R 13 ; Hake, Ann Marie 13 ; Matthews, Brandy R 13 ; Brosch, Jared R 13 ; Herring, Scott 13 ; Hunt, Cynthia 13 ; Porsteinsson, Anton P 51 ; Goldstein, Bonnie S 51 ; Kim, Martin 51 ; Makino, Kelly M 51 ; Saleem, Ismail M 51 ; Brand, Connie 51 ; Mulnard, Ruth A 51 ; Thai Gaby 51 ; Mc-Adams-Ortiz, Catherine 51 ; van Dyck Christopher H 52 ; Carson, Richard E 52 ; MacAvoy, Martha G 52 ; Varma Pradeep 52 ; Chertkow Howard 53 ; Bergman, Howard 53 ; Hosein, Chris 53 ; Black, Sandra 54 ; Stefanovic Bojana 54 ; Caldwell, Curtis 54 ; Hsiung, Ging-Yuek Robin 55 ; Feldman, Howard 55 ; Mudge Benita 55 ; Assaly Michele 55 ; Finger, Elizabeth 56 ; Pasternack, Stephen 56 ; Rachisky Irina 56 ; Trost, Dick 56 ; Kertesz, Andrew 57 ; Bernick, Charles 58 ; Munic Donna 58 ; Lipowski Kristine 23 ; Weintraub Masandra 23 ; Bonakdarpour Borna 23 ; Kerwin, Diana 23 ; Chuang-Kuo, Wu 23 ; Johnson, Nancy 23 ; Sadowsky, Carl 59 ; Villena, Teresa 59 ; Turner, Raymond Scott 60 ; Johnson, Kathleen 60 ; Reynolds, Brigid 60 ; Sperling, Reisa A 12 ; Johnson, Keith A 12 ; Marshall, Gad 12 ; Yesavage Jerome 61 ; Taylor, Joy L 61 ; Lane, Barton 61 ; Rosen, Allyson 61 ; Tinklenberg Jared 61 ; Sabbagh, Marwan N 31 ; Belden, Christine M 31 ; Jacobson, Sandra A 31 ; Sirrel, Sherye A 31 ; Kowall, Neil 62 ; Killiany, Ronald 62 ; Budson, Andrew E 62 ; Norbash, Alexander 62 ; Johnson, Patricia Lynn 62 ; Obisesan, Thomas O 63 ; Wolday Saba 63 ; Allard, Joanne 63 ; Lerner, Alan 64 ; Ogrocki Paula 64 ; Tatsuoka Curtis 64 ; Fatica Parianne 64 ; Fletcher, Evan 11 ; Maillard, Pauline 11 ; Olichney, John 11 ; Carmichael, Owen 11 ; Kittur Smita 65 ; Borrie, Michael 66 ; T-Y, Lee 66 ; Bartha, Rob 66 ; Johnson, Sterling 66 ; Asthana Sanjay 66 ; Carlsson, Cynthia M 66 ; Preda, Adrian 67 ; Nguyen, Dana 67 ; Tariot Pierre 31 ; Burke, Anna 31 ; Trncic Nadira 31 ; Reeder, Stephanie 31 ; Bates Vernice 68 ; Capote Horacio 68 ; Rainka Michelle 68 ; Scharre, Douglas W 69 ; Kataki, Maria 69 ; Adeli Anahita 69 ; Zimmerman, Earl A 70 ; Celmins Dzintra 70 ; Brown, Alice D 70 ; Pearlson, Godfrey D 71 ; Blank, Karen 71 ; Anderson, Karen 71 ; Flashman, Laura A 72 ; Seltzer, Marc 72 ; Hynes, Mary L 72 ; Santulli, Robert B 72 ; Sink, Kaycee M 73 ; Gordineer Leslie 73 ; Williamson, Jeff D 73 ; Garg Pradeep 73 ; Watkins, Franklin 73 ; Ott, Brian R 25 ; Querfurth, Henry 25 ; Tremont, Geoffrey 25 ; Salloway, Stephen 25 ; Malloy, Paul 25 ; Correia, Stephen 25 ; Rosen, Howard J 5 ; Miller, Bruce L 5 ; Perry, David 5 ; Mintzer Jacobo 74 ; Spicer, Kenneth 74 ; Bachman, David 74 ; Finger Elizabether 66 ; Pasternak, Stephen 66 ; Rachinsky Irina 66 ; Rogers, John 66 ; Drost, Dick 66 ; Pomara Nunzio 75 ; Hernando Raymundo 75 ; Sarrael Antero 75 ; Schultz, Susan K 76 ; Ponto Laura L Boles 76 ; Shim Hyungsub 76 ; Smith, Karen Ekstam 76 ; Relkin, Norman 19 ; Chaing Gloria 19 ; Lin, Michael 19 ; Ravdin, Lisa 19 ; Smith, Amanda 77 ; Ashok, Raj Balebail 77 ; Fargher, Kristin 77 

1  Ajou University, Department of Software and Computer Engineering, Suwon, South Korea (GRID:grid.251916.8) (ISNI:0000 0004 0532 3933); Biomedical & Translational Informatics Institute, Geisinger, Danville, USA (GRID:grid.251916.8) 
2  Center for Computational Biology and Bioinformatics, Indiana University School of Medicine, Indianapolis, USA (GRID:grid.257413.6) (ISNI:0000 0001 2287 3919); Indiana University School of Medicine, Center for Neuroimaging, Department of Radiology and Imaging Sciences, Indianapolis, USA (GRID:grid.257413.6) (ISNI:0000 0001 2287 3919) 
3  Ajou University, Department of Software and Computer Engineering, Suwon, South Korea (GRID:grid.251916.8) (ISNI:0000 0004 0532 3933) 
4  Biomedical & Translational Informatics Institute, Geisinger, Danville, USA (GRID:grid.251916.8); Pennsylvania State University, The Huck Institute of the Life Sciences, University Park, USA (GRID:grid.29857.31) (ISNI:0000 0001 2097 4281) 
5  UC San Francisco, San Francisco, USA (GRID:grid.266102.1) (ISNI:0000 0001 2297 6811) 
6  UC San Diego, La Jolla, USA (GRID:grid.266100.3) (ISNI:0000 0001 2107 4242) 
7  Mayo Clinic, Rochester, USA (GRID:grid.66875.3a) (ISNI:0000 0004 0459 167X) 
8  UC Berkeley, Berkeley, San Francisco, USA (GRID:grid.47840.3f) (ISNI:0000 0001 2181 7878) 
9  University of Pennsylvania, Philadelphia, USA (GRID:grid.25879.31) (ISNI:0000 0004 1936 8972) 
10  USC, Los Angeles, USA (GRID:grid.42505.36) (ISNI:0000 0001 2156 6853) 
11  UC Davis, Sacramento, USA (GRID:grid.27860.3b) (ISNI:0000 0004 1936 9684) 
12  Brigham and Women’s Hospital/Harvard Medical School, Boston, USA (GRID:grid.62560.37) (ISNI:0000 0004 0378 8294) 
13  Indiana University, Bloomington, USA (GRID:grid.411377.7) (ISNI:0000 0001 0790 959X) 
14  Washington University St. Louis, St. Loui, USA (GRID:grid.4367.6) (ISNI:0000 0001 2355 7002) 
15  Prevent Alzheimer’s Disease 2020, Rockville, USA (GRID:grid.468171.d) 
16  Siemens, Erlangen, Germany (GRID:grid.5406.7) (ISNI:000000012178835X) 
17  Alzheimer’s Association, Chicago, USA (GRID:grid.422384.b) (ISNI:0000 0004 0614 7003) 
18  University of Pittsburg, Pittsburgh, USA (GRID:grid.21925.3d) (ISNI:0000 0004 1936 9000) 
19  Cornell University, Ithaca, USA (GRID:grid.5386.8) (ISNI:000000041936877X) 
20  Albert Einstein College of Medicine of Yeshiva University, Bronx, USA (GRID:grid.251993.5) (ISNI:0000000121791997) 
21  AD Drug Discovery Foundation, New York, USA (GRID:grid.251993.5) 
22  Acumen Pharmaceuticals, Livermore, USA (GRID:grid.427650.2) 
23  Northwestern University, Chicago, USA (GRID:grid.16753.36) (ISNI:0000 0001 2299 3507) 
24  National Institute of Mental Health, Bethesda, USA (GRID:grid.416868.5) (ISNI:0000 0004 0464 0574) 
25  Brown University, Providence, USA (GRID:grid.40263.33) (ISNI:0000 0004 1936 9094) 
26  University of Washington, Seattle, USA (GRID:grid.34477.33) (ISNI:0000000122986657) 
27  University of London, London, UK (GRID:grid.4464.2) (ISNI:0000 0001 2161 2573) 
28  UCLA, Torrance, USA (GRID:grid.239844.0) (ISNI:0000 0001 0157 6501) 
29  University of Michigan, Ann Arbor, USA (GRID:grid.214458.e) (ISNI:0000000086837370) 
30  University of Utah, Salt Lake City, USA (GRID:grid.223827.e) (ISNI:0000 0001 2193 0096) 
31  Banner Alzheimer’s Institute, Phoenix, USA (GRID:grid.418204.b) (ISNI:0000 0004 0406 4925) 
32  UUC Irvine, Orange, USA (GRID:grid.411377.7) 
33  Richard Frank Consulting, Consulting, USA (GRID:grid.5406.7) 
34  National Institute on Aging, Baltimore, USA (GRID:grid.419475.a) (ISNI:0000 0000 9372 4913) 
35  Oregon Health and Science University, Portland, USA (GRID:grid.5288.7) (ISNI:0000 0000 9758 5690) 
36  University of Alabama, Birmingham, USA (GRID:grid.265892.2) (ISNI:0000000106344187) 
37  Johns Hopkins University, Baltimore, USA (GRID:grid.21107.35) (ISNI:0000 0001 2171 9311) 
38  Richard Frank Consulting, Consulting, USA (GRID:grid.66875.3a) 
39  Mount Sinai School of Medicine, New York, USA (GRID:grid.59734.3c) (ISNI:0000 0001 0670 2351) 
40  Rush University Medical Center, Chicago, USA (GRID:grid.240684.c) (ISNI:0000 0001 0705 3621) 
41  Baylor College of Medicine, Houston, USA (GRID:grid.39382.33) (ISNI:0000 0001 2160 926X) 
42  Wien Center, Miami Beach, USA (GRID:grid.39382.33) 
43  Columbia University Medical Center, New York, USA (GRID:grid.239585.0) (ISNI:0000 0001 2285 2675) 
44  New York University, New York, USA (GRID:grid.137628.9) (ISNI:0000 0004 1936 8753) 
45  University of Texas Southwestern Medical School, Galveston, USA (GRID:grid.267313.2) (ISNI:0000 0000 9482 7121) 
46  Duke University Medical Center, Durham, USA (GRID:grid.189509.c) (ISNI:0000000100241216) 
47  Emory University, Atlanta, USA (GRID:grid.189967.8) (ISNI:0000 0001 0941 6502) 
48  University of Kansas Medical Center, Kansas City, USA (GRID:grid.412016.0) (ISNI:0000 0001 2177 6375) 
49  University of Kentucky, Lexington, USA (GRID:grid.266539.d) (ISNI:0000 0004 1936 8438) 
50  Mayo Clinic, Jacksonville, USA (GRID:grid.417467.7) (ISNI:0000 0004 0443 9942) 
51  University of Rochester Medical Center, Rochester, USA (GRID:grid.412750.5) (ISNI:0000 0004 1936 9166) 
52  Yale University School of Medicine, New Haven, USA (GRID:grid.47100.32) (ISNI:0000000419368710) 
53  McGill Univ. Montreal-Jewish General Hospital, Montreal, Canada (GRID:grid.14709.3b) (ISNI:0000 0004 1936 8649) 
54  Sunnybrook Health Sciences, Toronto, Canada (GRID:grid.413104.3) (ISNI:0000 0000 9743 1587) 
55  U.B.C. Clinic for AD & Related Disorders, Vancouver, Canada (GRID:grid.413104.3) 
56  Cognitive Neurology - St. Joseph’s, London, Canada (GRID:grid.413104.3) 
57  Cognitive Neurology - St. Joseph’s, London, Canada (GRID:grid.413104.3); St. Joseph’s Health Care, London, Canada (GRID:grid.416448.b) (ISNI:0000 0000 9674 4717) 
58  Cleveland Clinic Lou Ruvo Center for Brain Health, Las Vegas, USA (GRID:grid.239578.2) (ISNI:0000 0001 0675 4725) 
59  Premiere Research Inst (Palm Beach Neurology), W Palm Beach, USA (GRID:grid.477769.c) 
60  Georgetown University Medical Center, Washington, USA (GRID:grid.411667.3) (ISNI:0000 0001 2186 0438) 
61  Stanford University, Stanford, USA (GRID:grid.168010.e) (ISNI:0000000419368956) 
62  Boston University, Boston, Massachusetts, USA (GRID:grid.189504.1) (ISNI:0000 0004 1936 7558) 
63  Howard University, Washington, USA (GRID:grid.257127.4) (ISNI:0000 0001 0547 4545) 
64  Case Western Reserve University, Cleveland, USA (GRID:grid.67105.35) (ISNI:0000 0001 2164 3847) 
65  Neurological Care of CNY, Liverpool, USA (GRID:grid.27860.3b) 
66  St. Joseph’s Health Care, London, Canada (GRID:grid.416448.b) (ISNI:0000 0000 9674 4717) 
67  UUC Irvine, Orange, USA (GRID:grid.416448.b) 
68  Dent Neurologic Institute, Amherst, USA (GRID:grid.417854.b) 
69  Ohio State University, Columbus, USA (GRID:grid.261331.4) (ISNI:0000 0001 2285 7943) 
70  Albany Medical College, Albany, USA (GRID:grid.413558.e) (ISNI:0000 0001 0427 8745) 
71  Hartford Hospital Olin Neuropsychiatry Research Center, Hartford, USA (GRID:grid.277313.3) (ISNI:0000 0001 0626 2712) 
72  Dartmouth-Hitchcock Medical Center, Lebanon, USA (GRID:grid.413480.a) (ISNI:0000 0004 0440 749X) 
73  Wake Forest University Health Sciences, Winston-Salem, USA (GRID:grid.412860.9) (ISNI:0000 0004 0459 1231) 
74  Medical University South Carolina, Charleston, USA (GRID:grid.259828.c) (ISNI:0000 0001 2189 3475) 
75  Nathan Kline Institute, Orangeburg, USA (GRID:grid.250263.0) (ISNI:0000 0001 2189 4777) 
76  University of Iowa College of Medicine, Iowa City, USA (GRID:grid.214572.7) (ISNI:0000 0004 1936 8294) 
77  University of South Florida: USF Health Byrd Alzheimer’s Institute, Tampa, USA (GRID:grid.170693.a) (ISNI:0000 0001 2353 285X) 
Publication year
2019
Publication date
Dec 2019
Publisher
Nature Publishing Group
e-ISSN
20452322
Source type
Scholarly Journal
Language of publication
English
DOI
https://doi.org/10.1038/s41598-018-37769-z
ProQuest document ID
2179726028
Copyright
This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.