No support for white matter connectivity differences in the combined and inattentive ADHD presentations

Abstract

About the Authors: Jacqueline F. Saad Roles Conceptualization, Formal analysis, Investigation, Methodology, Writing – original draft, Writing – review & editing * E-mail: [email protected] Affiliations The Brain Dynamics Centre, Westmead Institute for Medical Research, The University of Sydney, Sydney, Australia, Discipline of Psychiatry, Western Clinical School, The University of Sydney, Sydney, Australia ORCID logo https://orcid.org/0000-0002-3555-2361 Kristi R. Griffiths Roles Data curation, Methodology, Writing – review & editing Affiliation: The Brain Dynamics Centre, Westmead Institute for Medical Research, The University of Sydney, Sydney, Australia Michael R. Kohn Roles Supervision Affiliations The Brain Dynamics Centre, Westmead Institute for Medical Research, The University of Sydney, Sydney, Australia, Department of Adolescent and Young Adult Medicine, Centre for Research into Adolescents’ Health, Westmead Hospital, Sydney, New South Wales, Australia Taylor A. Braund Roles Formal analysis Affiliations The Brain Dynamics Centre, Westmead Institute for Medical Research, The University of Sydney, Sydney, Australia, Discipline of Psychiatry, Western Clinical School, The University of Sydney, Sydney, Australia ORCID logo https://orcid.org/0000-0001-8408-9070 Simon Clarke Roles Conceptualization, Data curation Affiliations The Brain Dynamics Centre, Westmead Institute for Medical Research, The University of Sydney, Sydney, Australia, Department of Adolescent and Young Adult Medicine, Centre for Research into Adolescents’ Health, Westmead Hospital, Sydney, New South Wales, Australia Leanne M. Williams Roles Methodology, Supervision, Writing – review & editing Affiliations Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, California, United States of America, Sierra-Pacific Mental Illness Research, Education, and Clinical Center (MIRECC), VA Palo Alto Health Care System, Palo Alto, California, United States of America Mayuresh S. Korgaonkar Roles Conceptualization, Methodology, Resources, Supervision, Writing – review & editing Affiliations The Brain Dynamics Centre, Westmead Institute for Medical Research, The University of Sydney, Sydney, Australia, Discipline of Psychiatry, Western Clinical School, The University of Sydney, Sydney, Australia Introduction Characterized by clinical deficits in attention, hyperactivity, and impulsivity, attention deficit hyperactivity disorder (ADHD) is a highly prevalent neurodevelopmental condition with an estimated global prevalence of 3.4% of children worldwide [1]. [...]neuroimaging studies have investigated structural, functional, and more recently connectome features in ADHD which may inform the underlying pathophysiology of ADHD presentations. [...]growing evidence from these studies may help better reconceptualize ADHD by linking brain-based features with improved clinical care models and treatment outcomes [4]. [...]we applied tract-based spatial statistics (TBSS), using DTI derived scalar measures [23] of fractional anisotropy (FA), mean (MD), radial (RD), and axial (AD) diffusivity as indices of water diffusion properties in WM tracts. [...]we applied tractography analyses linking 84 brain regions distributed throughout the brain and used network-based Statistics (NBS) to explore connectomic differences and brain network organization, and graph theoretical analysis to capture topographic properties, which may underpin these subtypes.

About the Authors:

Jacqueline F. Saad

Roles Conceptualization, Formal analysis, Investigation, Methodology, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliations The Brain Dynamics Centre, Westmead Institute for Medical Research, The University of Sydney, Sydney, Australia, Discipline of Psychiatry, Western Clinical School, The University of Sydney, Sydney, Australia

ORCID logo https://orcid.org/0000-0002-3555-2361

Kristi R. Griffiths

Roles Data curation, Methodology, Writing – review & editing

Affiliation: The Brain Dynamics Centre, Westmead Institute for Medical Research, The University of Sydney, Sydney, Australia

Michael R. Kohn

Roles Supervision

Affiliations The Brain Dynamics Centre, Westmead Institute for Medical Research, The University of Sydney, Sydney, Australia, Department of Adolescent and Young Adult Medicine, Centre for Research into Adolescents’ Health, Westmead Hospital, Sydney, New South Wales, Australia

Taylor A. Braund

Roles Formal analysis

Affiliations The Brain Dynamics Centre, Westmead Institute for Medical Research, The University of Sydney, Sydney, Australia, Discipline of Psychiatry, Western Clinical School, The University of Sydney, Sydney, Australia

ORCID logo https://orcid.org/0000-0001-8408-9070

Simon Clarke

Roles Conceptualization, Data curation

Affiliations The Brain Dynamics Centre, Westmead Institute for Medical Research, The University of Sydney, Sydney, Australia, Department of Adolescent and Young Adult Medicine, Centre for Research into Adolescents’ Health, Westmead Hospital, Sydney, New South Wales, Australia

Leanne M. Williams

Roles Methodology, Supervision, Writing – review & editing

Affiliations Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, California, United States of America, Sierra-Pacific Mental Illness Research, Education, and Clinical Center (MIRECC), VA Palo Alto Health Care System, Palo Alto, California, United States of America

Mayuresh S. Korgaonkar

Roles Conceptualization, Methodology, Resources, Supervision, Writing – review & editing

Affiliations The Brain Dynamics Centre, Westmead Institute for Medical Research, The University of Sydney, Sydney, Australia, Discipline of Psychiatry, Western Clinical School, The University of Sydney, Sydney, Australia

Introduction

Characterized by clinical deficits in attention, hyperactivity, and impulsivity, attention deficit hyperactivity disorder (ADHD) is a highly prevalent neurodevelopmental condition with an estimated global prevalence of 3.4% of children worldwide [1]. Presentation types (i.e. subtypes) of ADHD are categorized as either predominantly inattentive (ADHD-I), predominantly hyperactive-impulsive (ADHD-HI) or combined (ADHD-C) [2]. Cognitive and behavioral differences between clinical presentations have been well documented across the neuropsychological literature [3]. Furthermore, the knowledge gap remains in advancing the neurobiological framework in ADHD. Consequently, neuroimaging studies have investigated structural, functional, and more recently connectome features in ADHD which may inform the underlying pathophysiology of ADHD presentations. Moreover, growing evidence from these studies may help better reconceptualize ADHD by linking brain-based features with improved clinical care models and treatment outcomes [4].

Research from task and resting-state functional connectivity studies reveal neural differences between the ADHD-C and ADHD-I types with atypical patterns of increased and reduced activation in the cingulo frontal parietal attentional (CFP) and default mode (DMN) networks [5]. The regions involved in these networks are of significance to the underlying pathophysiology in ADHD as they are concordant with the clinical deficits associated with the ADHD-I and ADHD-C types, respectively. However, evidence of neuroanatomical differences between the ADHD presentation types remains conflicted, with multiple studies reporting no grey matter volumetric differences [6–10], yet others reporting decreased volumes of the caudate and anterior cingulate cortex (ACC) in ADHD-C relative to ADHD-I type [11], regions associated with nodes of the DMN in ADHD-I than ADHD-C [12] and also in the hippocampus in ADHD-C relative to ADHD-I type [13]. A small number of studies investigating microstructural white matter (WM) properties between the ADHD-C and ADHD-I subtypes, using diffusion tensor imaging (DTI), also yield equivocal findings. DTI is an MRI technique which can provide information on WM microstructure by observing the directionality and coherence of water diffusion [14]. This is typically quantified using scalar metrics such as fractional anisotropy (FA), a marker of microstructural architecture; radial diffusivity (RD) to assess axonal myelination; axial diffusivity (AD) as a variable of axonal maturation; and mean diffusivity (MD) as an average of molecular diffusion, independent of directionality [15]. Using these scalar measures to evaluate the ADHD subtypes, Svatkova, Nestrasil [16] reported increased RD in the forceps minor in ADHD-I relative to ADHD-C [16]. Other studies have also reported increased RD bilaterally and AD in mostly left-lateralized fronto-striato-cerebellar regions [17], increased FA and RD in the right thalamus, increased AD in the left postcentral gyrus and right caudate, and increased RD in the left postcentral gyrus and supplementary motor area [18] for ADHD-C relative to ADHD-I. Additionally, studies comparing the ADHD-C type only to controls have observed lower FA values in ADHD-C in regions surrounding the basal ganglia [19,20]. Considering the dearth of studies available in this field, the goal of this study, using both voxelwise analyses of scalar measures and connectomic analyses of tractography measures, was to investigate whether differences in white matter microstructural properties and connectivity, distinguish the two most common ADHD types, ADHD-C and ADHD-I. In this same cohort, we have previously demonstrated differences in graph properties of structural volume covariance networks between the two ADHD types [6]. Based on this our goal was to test whether these ADHD types also differed in organization of their brain white matter.

The evolving framework of ADHD pathophysiology is shaped by the application of neuroimaging measures which include the more recent connectomic approach in assessing possible disruptions in brain network connectivity. An example of this paradigm shift can be seen in the proposal of a neurocircuitry based model in ADHD, which incorporates knowledge on the role of inter-regional network organization involving frontal, temporal and parietal regions, from the historical view of dopaminergic regulation and frontal-striatal circuitry deficit [5]. Network-based Statistics (NBS) and graph theoretical analysis can be applied to assess structural connectivity, which is represented by anatomical connections formed by WM axonal fiber tracts to understand whether these connections underpin functional network connections. Network topology can be characterized by the application of graph modeling of the structural or functional links characterizing the interregional neuronal connections [21]. Only one study to date has compared DTI-based structural connectomic measures between the two ADHD subtypes and found decreased white matter connectivity in ADHD-C compared to ADHD-I, involving mostly right-hemispheric frontal, cingulate and supplementary motor regions [22].

This study utilized DTI data to assess whether structural WM microstructural properties and connectivity may distinguish the ADHD- C and ADHD-I types, and comparatively to neurotypical controls. Firstly, we applied tract-based spatial statistics (TBSS), using DTI derived scalar measures [23] of fractional anisotropy (FA), mean (MD), radial (RD), and axial (AD) diffusivity as indices of water diffusion properties in WM tracts. Secondly, we applied tractography analyses linking 84 brain regions distributed throughout the brain and used network-based Statistics (NBS) to explore connectomic differences and brain network organization, and graph theoretical analysis to capture topographic properties, which may underpin these subtypes.

Methods

Participant characteristics and study procedure

Participants were recruited as part of the International Study to Predict Optimized Treatment in ADHD (iSPOT-A) study. A detailed account of the inclusion/exclusion criteria protocols for participant recruitment, diagnostic measures, and procedures for the iSPOT-A study has been previously published [24] and is described briefly below. DTI data collected at Westmead Hospital, Sydney, as part of the baseline MRI data collection for the iSPOT-A study, were available for 37 participants with ADHD (mean = 13.30 ± 2.56; range 8–17 years) and 28 age and gender-matched typically developing controls (mean = 13.09 ± 2.63; range 8–17 years). Data from 2 participants were discarded due to excessive artifacts in the DTI dataset.

Confirmation of ADHD diagnosis (DSM-IV criteria) and subtype (i.e. presentation in DSM-V) was measured by the Mini International Neuropsychiatric Interview (MINI Kid) [25], and the Attention Deficit/Hyperactivity Disorder Rating Scale (ADHD-RS-IV) [26] with symptom severity assessed using the ADHD-RS-IV scores (requires a score of >1 on 6 or more subscale items on the Inattentive and/or Hyperactive/Impulsive subscales) and the Conners’ Parent Rating Scale-Revised: Long Version (CPRS-LV) [24]. Of the 37 ADHD participants, 19 met diagnostic criteria for ADHD-C type (mean = 13.25 ± 2.53; 4 females), while 18 met diagnostic criteria for the ADHD-I type (mean = 13.35 ± 2.65; 4 females). Seven ADHD-C participants and three ADHD-I participants were diagnosed with comorbid oppositional defiant disorder. All ADHD participants were medication-free at the time of testing; 20 were medication naïve; 17 were treatment-experienced with methylphenidate (n = 11 ADHD-C; n = 5 ADHD-I) and were withdrawn from methylphenidate for at least 5 half-lives. Participants were all fluent in English and had no history of brain injury, any significant medical condition affecting brain function (e.g., epilepsy), or any contraindications for MRI. All participants and/or their guardians provided written informed consent to participate in the research, in accordance with the National Health and Medical Research Council guidelines.

DTI image acquisition and preprocessing

Magnetic resonance images were acquired using a 3.0 Tesla GE Signa HDx scanner (GE Healthcare, Milwaukee, WI) using an 8-channel head coil. Diffusion tensor images were acquired using a spin-echo DTI-echo planar imaging sequence. Seventy contiguous 2.5mm slices were acquired in an axial orientation with an in-plane resolution of 1.72mm x 1.72mm and a 128 x 128 matrix (repetition time (TR) = 17000 ms; echo time (TE) = 95 ms; fat saturation: on; number of excitations (NEX) = 1; frequency direction: right/left). A baseline image (b = 0) and 42 different diffusion orientations were acquired with a b-value of 1250. Total acquisition time for the DTI protocol was 13 min 36s. Three-dimensional (3-D) T1-weighted magnetic resonance images were also acquired in the sagittal plane using a 3D SPGR sequence (TR = 8.3 ms; TE = 3.2 ms; flip Angle = 11°; TI = 500 ms; NEX = 1; ASSET = 1.5; Frequency direction: S/I). A total of 180 contiguous 1 mm slices were acquired with a 256 × 256 matrix, with an in-plane resolution of 1 mm × 1 mm resulting in isotropic voxels.

Diffusion tensor imaging data pre-processing and analytic methods utilized the Oxford Centre for Functional MRI of the Brain (FMRIB) diffusion toolbox and TBSS software tools as part of the FMRIB Software Library release 4.1.3 (http://www.fmrib.ox.ac.uk/fsl) [23,27]. Raw DTI data were first corrected for head movement and eddy current distortions. Diffusion tensor models were then fitted for each voxel within the brain mask and skeletonized FA, MD, AD, and RD images were generated for each participant. A detailed account of the steps applied prior to running voxel-wise statistical analysis which involves aligning the FA, MD, AD and RD images from each individual using non-linear registration and projecting this onto the mean FA skeleton mask for the cohort, has been published in our previous study [28].

Tract-based spatial statistical (TBSS) analysis of DTI data

We have previously described in greater detail, the methods employed for the TBSS Analysis of the DTI data [28]. In brief, group comparisons were performed for each WM microstructural metric using the Randomise (v2.1) permutation testing software in FSL. We performed three sets of comparisons: (1) ADHD-C versus ADHD-I group, (2) ADHD-C versus controls and (3) ADHD-I versus controls. Permutation testing was performed using 5,000 permutations with the threshold-free cluster enhancement option. We also extracted average DTI values for 46 white matter tract regions defined using the white matter JHU atlas and performed a 3 (group) x 1 (DTI derived FA values) one-way between-groups analysis of variance (ANOVA) [29]. We corrected for the number of tracts tested by applying a Bonferroni correction for multiple comparisons, i.e., p < 0.001 (0.05/46).

Network analysis of DTI data

DTI based structural connectome matrices for each individual participant were generated using probabilistic tractography [30]. Tractography analysis was performed using each brain region as seed and the remaining region labels as targets. One thousand sample tracts were generated from each voxel within the seed region, and only tracts that reached the target region were retained. The tracts were terminated once they reached a particular target region. The Desikan-Killiany atlas [31] was utilized to define the 84 brain regions using freesurfer (v4.3) (http://surfer.nmr.mgh.harvard.edu/) analysis of the T1 MRI scan. This resulted in an 84 x 84 interregional connectivity matrix of number of probabilistic tracts for each participant. Based on standard protocols and methods for MRI cortical parcellation described in our previous study [32], Freesurfer (v4.3) (http://surfer.nmr.mgh.harvard.edu) standard pipeline segmentation of the 3-D T1-weighted structural images, using the Deskilian- Killiany [31] atlas a two-dimensional cortical surface was calculated and automatically divided into 35 gyral-based anatomically labeled areas for each hemisphere. Cortical segmentation and anatomical labels were checked by manual inspection allowing the utislization of all the cortial regions and 7 subcortical structures for the network analysis to generate the connectivity matrix [32]. Furthermore, after manual checks for anomalies in whole brain parcellation we didn’t exclude any participants due to anomalies from the analysis.

Network-based statistical (NBS) analyses

Network-based statistical Analyses (NBS) was then applied to examine regional connectivity differences in the interregional 84 x 84 connectivity matrix between the ADHD-C (n = 19) and ADHD-I (n = 18) types and controls (n = 26). This method has been previously described in detail in Korgaonkar, Fornito [32].

Graph theory analyses

Graph theoretical analyses were performed on the inter-regional 84 x 84 connectivity matrices using the Brain Connectivity Toolbox (http://www.brain-connectivity-toolbox.net/) [33]. Global topological properties of the brain were estimated for each individual using: 1) characteristic path length (mean number of connections on the shortest path between any two regions in the network), 2) the clustering coefficient (quantification of the probability that two nodes connected to an index node are also connected to each other). Regional nodal characteristics were measured using the nodal degree (number of connections that a node has with the rest of the network). The matrices were thresholded at a range of network densities in 0.01 steps (0.05–0.30) to allow a comparison of network properties between the groups and avoid biases associated with using a single threshold as is typically done. Area under the curve was calculated to examine group differences across the full range of sparsity thresholds, with assessments for both global regional measures performed using a p value corrected for a number of global measures, i.e., p < 0.025 (.05/2), and for the number of nodes in regional measures, i.e., p < 0.0006 (.05/84). This method applied has been previously described in detail [32].

Correlations between FA values, network and clinical measures

Correlation analyses were performed to measure any associations between: FA values (for the JHU WM atlas tracts) and the global and regional graph network measures, and the dimensional measures of the ADHD-RS scores; total inattention items, total hyperactive/impulsive items, and total item scores, while controlling for age and gender.

Results

Demographic and clinical characteristics for the ADHD-C type, ADHD-I type, and control participants are summarized in Table 1. No significant differences were present between the three groups in terms of age and gender. Medication treatment history, comorbid disorders, or the ADHD-RS-IV (inattentive symptom items) sum of items 1–9 did not significantly differ between subtypes. Characteristically, ADHD-C type significantly differed from ADHD-I on the sum of items 10–18 (hyperactive/impulsive symptom items) and total item scores on the ADHD-RS-IV, in concordance with combined type criteria and severity (p < .05). Of the ADHD-I type participants, one participant qualified for seven items out of 9 hyperactive/impulsive subscale items, less than four 4 items (n = 4), 3 items (n = 2), 2 items (n = 1) and ≤1 item (n = 11).

[Figure omitted. See PDF.]

Table 1. Participant demographic and clinical characteristics.

https://doi.org/10.1371/journal.pone.0245028.t001

Tract-based statistical analyses and atlas based region of interest analysis

No significant differences were found between the subtypes or relative to controls for any of the DTI scalar measures FA, AD, RD, or MD using voxel-wise analysis. No significant differences were observed for the FA average values for the 46 defined WM regions, however findings from the ROI analyses which were significant at the uncorrected level is shown in Fig 1.

[Figure omitted. See PDF.]

Fig 1. Group comparison of FA values using atlas-based region of interest analysis between the combined (ADHD-C) and inattentive (ADHD-I) subtypes, and controls.

https://doi.org/10.1371/journal.pone.0245028.g001

Forrest plots showing FA values for 46 defined White Matter Regions using region of interest analysis comparing the combined and inattentive ADHD types (Fig 1A), Combined and controls (Fig 1B), and inattentive relative to controls (Fig 1C). The dotted line represents the .05 p-value threshold, and the solid line represents the Bonferroni corrected p-value threshold.

Network-based statistical analysis

NBS analysis also did not identify any inter-regional connectivity differences between the ADHD-C and ADHD-I types or when compared to controls after correcting for multiple comparisons.

Graph theoretical analysis

Neither of the global topological measures: clustering coefficient or global characteristic path length, showed significant differences between the ADHD-C and ADHD-I groups or compared to controls. Group comparison of the regional nodal degree revealed a single significant finding between the ADHD-C and control group, (p < .05 corrected for multiple comparisons) in the right insula (corrected p = .029; ADHD-C < Controls). There were no significant differences found between the two ADHD subtypes, or the ADHD-I type compared to controls. Regional nodal degree measures which were significant but did not survive after correcting for multiple comparisons are illustrated in Fig 2A–2C.

[Figure omitted. See PDF.]

Fig 2. Group comparison of regional nodal degree measures using graph theoretical analysis between the combined (ADHD-C) and inattentive (ADHD-I) subtypes, and controls.

https://doi.org/10.1371/journal.pone.0245028.g002

Forrest plots showing p-values for 84 brain regions comparing the combined and inattentive ADHD types (Fig 2A), Combined and controls (Fig 2B), and inattentive relative to controls (Fig 2C). The dotted line represents the .05 p-value threshold, and the solid line represents the Bonferroni corrected p-value threshold.

Correlations between the ROI, network and clinical measures

While no significant associations were found following Bonferroni correction, associations were observed at the uncorrected level between the sum of items 1–9 and FA in the left anterior limb of the internal capsule; p = 0.039, both the sum of items 10–18 and total item score with FA in the right superior cerebellar peduncle at p = 0.032 and p = 0.049, respectively. None of the global or regional network measures were correlated with the clinical rating measures of the ADHD-RS-IV. Associations observed between regional nodal degree measures and clinical measures at an uncorrected threshold have been summarized in S1–S3 Tables in the supplementary material.

Discussion

This study investigated whether structural WM microstructural properties and inter-regional white matter connectivity distinguished the combined (ADHD-C) and inattentive (ADHD-I) DSM-V presentations of ADHD. TBSS analysis of FA, AD, MD, and RD values, ROI analysis of FA values for 46 white matter tracts defined by the JHU WM atlas and NBS analyses of inter-regional connectivity of 84 cortical-subcortical brain regions revealed no significant differences between the subtypes or compared to controls. Of all the white matter measures, only graph properties of nodal degree for the insula were found to significantly differ in the ADHD-C type relative to controls after controlling for multiple comparisons. Studies of WM microstructural properties between the subtypes of ADHD are limited. In line with the only two previous TBSS studies [16,17] available in the literature, our findings did not find differences in FA, between the two ADHD subtypes. However, both the Svatkova [16] and Ercan [17] studies did observe differences in RD, MD, and AD between the two ADHD types. An earlier study [18] demonstrated FA, RD, and AD differences between the two subtypes, although the analytical approach employed in that study is no longer accepted in current DTI studies which favor TBSS, as a sophisticated model for measures of DTI scalar values [34]. While we did not find any significant white matter tractography differences between the combined and inattentive subtypes of ADHD, only one other study exists that has reported on white matter connectivity differences in the frontal, cingulate and supplementary motor regions between patients with ADHD- C and ADHD-I type using NBS [22]. As such, further research is warranted to determine differences in WM structural integrity between the hyperactive/impulsive subtype relative to the other subtypes.

Another possible reason for null findings in our study may be due to a number of treatment-experienced participants in our study compared to the previous subtype-based studies, which recruited participants who were treatment-naive or with treatment history < 6 months [17,18,22]. Svatkova, Nestrasil [16] had mostly treatment-experienced participants (n = 21, 63%) without washout prior to scanning. Participants in our study who were treatment-experienced (n = 17, 46%) commenced scanning procedures after entering a washout period of 7 days. Interestingly, previous subtype inclusive studies which compared ADHD participants to controls and found significant differences also had a majority of treatment naïve participants [35–38]. However, it is important to note that these previous studies are not comparable with our study in terms of methodological approaches. Furthermore, a recent DTI study investigated a medication naïve ADHD group of children and adults and found no differences to controls in the child group but reported significant findings in the treatment naïve adult group [39]. Nevertheless, it may be likely that white matter differences in ADHD may be influenced by stimulant treatment effects but may also be due to neurodevelopmental periods in line with the proposed maturational lag theory in ADHD [40].

Consistent with the two previous subtype comparison studies utilizing TBSS, we found no differences in FA values [16,17] or AD and MD values, distinguishing the two types [16]. Notably, while both studies employed cohorts that were similar to our study in terms of clinical characteristics and sample size, the Ercan, Suren [17] study included a treatment naïve sample. Interestingly, increased FA values were found relative to controls, in the bilateral cingulum bundle in ADHD-C, and the anterior thalamic radiation, inferior longitudinal fasciculus and left corticospinal tract involving regions linked to fronto-striatal-thalamic circuits in ADHD-I [16]. Increased RD in the forceps minor in ADHD- I relative to ADHD-C [16] and increased RD bilaterally and increased AD in brain regions mostly on the left side linked to fronto-striato-cerebellar regions in ADHD- C relative to ADHD-I [17] have also been reported. In contrast, also using a TBSS approach, Rossi et al. [41] measured FA values for 20 fiber tracts in ADHD-I children compared to controls and found no significant difference between the groups.

Additionally, we performed a region of interest (ROI) analysis and did not find significant differences in FA values between the two types. Despite the absence of significant differences between groups at the corrected thresholds, increased FA values in ADHD-C compared to ADHD-I and controls were observed at uncorrected thresholds. That is, the bilateral posterior corona radiata characterized the ADHD-C type from ADHD-I and controls, the left posterior limb of internal capsule and left anterior limb of internal capsule in ADHD-C relative to ADHD-I, and the left uncinate fasciculus in ADHD-C compared to controls. Interestingly, similar findings from a previous study comparing an ADHD group reported reduced FA values in the ADHD groups in the posterior corona radiata, the uncinate fasciculus with reduced MD in the posterior limb of internal capsule, relative to controls [42].

In our study, we also used tractography to evaluate white matter connectomic differences. Our study did not find WM connectivity differences between the ADHD-C and ADHD-I subtypes or compared to controls. However, previous findings from Hong, Zalesky [22], the only other study evaluating connectome differences between the subtypes, observed decreased WM connectivity differences in ADHD-C relative to ADHD-I connecting 17 right hemispheric brain regions including the superior frontal gyrus, anterior cingulate gyrus, and supplementary motor areas. Additionally, they found significant negative correlations between FA values and continuous performance task scores to differentiate ADHD-C and ADHD-I. Also, we did not find any significant differences between the two ADHD subtypes for global topological characteristics or regional nodal graph measures. However, a single significant finding was observed which showed reduced nodal degree involving the right insula in the ADHD- C group when compared to controls. Whilst speculative, similar findings involving the insula have been noted in previous structural [43–45] and functional connectivity [46,47] studies in ADHD and warrant further research. The insula is considered part of the salience network which interacts with the default mode network. Reduced degree is this region may account for attentional processing and goal-directed action deficits and tied to mechanisms of executive functioning and sensorimotor dysregulation associated with ADHD-C type [48]. Furthermore, this finding may have implications for attentional and inhibitory emotional processing in ADHD [45]. At the uncorrected thresholds, regional network measures revealed some interesting results; with increased nodal degree in regions involving the occipital temporal, and parietal lobes in ADHD-I relative to ADHD-C and controls and increased nodal degree in regions of the frontal lobe and the basal ganglia in ADHD-C compared to ADHD-I and controls. These findings are consistent with the Hong, Zalesky [22] study that also revealed decreased connectivity in ADHD-C, relative to ADHD-I, in frontal regions, which are involved in regulating inhibitory response and motor activity, both of which are linked to the ADHD-C type.

Although the availability of DTI studies which specifically examine subtype differences are sparse, significant findings have been reported, albeit equivocal. Undoubtedly, further studies are warranted to establish structural connectivity alterations in ADHD types, however, emerging evidence from DTI studies examining an all-inclusive ADHD group with controls have reported significant findings. For example, studies investigating basal ganglia and thalamic connectivity have found lower FA values in ADHD-C relative to controls [19,20]. Fall and colleagues [49] indirectly support these findings reporting mean reaction times correlated with MD values in the striatum and thalamus in ADHD-C compared to controls. Interestingly, these results are consistent with studies using structural gray matter and resting state functional connectivity, reported in a recent review [50] which have shown disruptions in the motor network in ADHD-C. Whether these structural connectivity disruptions or the absence of WM abnormalities attribute to functional deficits associated with the subtypes of ADHD, remain unconfirmed. And so, while the present study focused on whether hard-wired networks may shed insight that distinguishes the subtypes, further exploration of the functional dynamics of brain network organization may potentially provide greater insight toward the underlying neurobiology of the ADHD subtypes. Importantly, the rationale for the DSM-V based diagnostic criteria adopting ‘presentation type’ in place of ‘subtypes’ was to try to convey the somewhat transient nature to these classifications and how these clinical symptoms evolve throughout development [50,51]. Perhaps this is another potential reason for the null findings as ADHD presentation types are not stable across time, and thus they may be better linked with functional measures.

Limitations

There were several limitations to the study. It is quite likely that the lack of significant findings in our study is due to the small available sample size and should any differences be present, then these would be of minimal size, and only detectable in larger cohorts. Using the trend findings from the comparison analysis between the two ADHD subtypes, a power analysis indicated that at least n = 35, and n = 49 for the FA derived atlas-based region of interest analysis and regional nodal degree, respectively, would be required would be required for effects to be significant at the Bonferroni corrected threshold of 0.001.

This limits the statistical power of our results and thus may explain why we were not able to detect differences between groups after appropriate correcting for multiple comparisons. We have reported uncorrected findings in the supplementary section and replication studies with larger sample sizes utilizing these measures are warranted to further explore possible differences in microstructural integrity between the ADHD types. In a previous study utilizing the same cohort, we did observe differences in structural covariance between the two ADHD subtypes. It may be that interregional structural covariance relates better to functional networks and it is likely that functional differences are more pronounced between the ADHD types [6,52]. It is, however, important to note that some of the previous studies have reported significant white matter differences with similar sample sizes. It is likely these results could be cohort-specific, stressing the need for replication in larger cohorts.

We applied standard established analytical methods and hence our null findings are important to consider in the context of the paucity of DTI studies comparing ADHD presentations. However, an important consideration relates to common limitations of DTI analyses, in particular tract-based spatial statistics (TBSS) where TBSS is subject to inaccuracies relating to FA-based projections in regions of crossing fibers and incapable of obtaining a complete measure of the total number of white matter axons; combining within voxel microscopic fiber density and macroscopy [53]. The TBSS processing pipeline involved taking FA images from each participant which were then aligned to the FMRIB58_FA template and transformed into Montreal Neurological Institute 152 1 mm3 standard space using the nonlinear registration tool FNIRT [54,55] which may impact the validity and outcome of the TBSS findings [56–58]. However, TBSS is considered a “state of the art” [56] leading voxel wise DTI analysis technique and this approach is typically used in the field for voxel-wise comparisons of diffusion data. We employed TBSS as one of our measures, given its the one that is the most common for the field. Additionally, for comprehensiveness we employed measures of tractography to account for methodological limitations of TBSS. Furthermore, due to the paucity of DTI studies which specifically examine the ADHD subtypes, these measures were selected as valid and novel techniques, but also to allow as much as possible, a comparison with the limited available studies in the literature that have employed the same techniques. Future research would benefit from the inclusion of more sensitive measures such as fixel-based analysis and/or calculation of tract length of each WM tract.

Our study examined the combined and inattentive type only, as data for the predominantly hyperactive-impulsive type was not available, thereby analysis between all three types was not possible. Medication effects and stimulant treatment history may bias the findings from this study; however, this confounding effect is difficult to eliminate. Although, recent ADHD DTI research studies which investigated possible medication effects on WM structure and connectivity produced inconclusive findings [39]. Changes in FA may be influenced by variable factors such as familial vulnerability, ADHD symptom count, and neurodevelopmental periods as it is well established that FA increases from 10–12 years up to 40 years of age in all WM tracts [59,60]. Consequently, these factors are not uniformly accounted for in ADHD research and may limit the generalizability of the results of our study. While this study did match participants for age and gender, analyses could not be explored by developmental periods to explore possible FA differences due to the small sample size. In the absence of formal IQ measures we cannot determine whether our results were mediated by group differences in IQ. Previous studies have shown IQ differences between ADHD participants and typically developing healthy individuals [61]. While we do not have a formal reliable measure of IQ, we used the Spot the Word Test [62], collected as part of the iSPOT-A cognitive battery. This measure of verbal memory has been validated as a proxy estimate of intelligence [63]. We did not find any significant differences in this IQ estimate score, but future work should consider exploring the effects of IQ between ADHD subtypes. Finally, DTI lacks any information on functional connectivity of the brain and as such replication of these findings will follow in another study which will involve resting-state fMRI to determine whether similarities are found in functional interregional connections between the two subtypes.

Conclusion

In summation, we did not find white matter microstructural properties or network connectivity to differentiate the two subtypes from each other or relative to controls. However, we did find the ADHD-C group to differ in regional nodal degree in the insula relative to controls. While the overall results of this study may be inconsistent with the findings of previous ADHD subtype DTI studies, the absence of evidence for differences in WM microstructural properties relative to FA values between the two subtypes is in line with previous research. To the best of our knowledge, this is the first study to utilize both voxel-wise analysis of scalar white matter measures in addition to tractography incorporating network and graph theoretical measures to examine differences between these two ADHD subtypes.

Supporting information

S1 Table. Correlations between the global network measures and the ADHD-RS IV scores.

https://doi.org/10.1371/journal.pone.0245028.s001

(DOCX)

S2 Table. Correlations between averaged FA values of the 46 white matter tracts and the ADHD-RS IV scores.

https://doi.org/10.1371/journal.pone.0245028.s002

(DOCX)

S3 Table. Correlations between regional nodal degree of the 84 white matter regions and the ADHD-RS IV scores.

https://doi.org/10.1371/journal.pone.0245028.s003

(DOCX)

Acknowledgments

We thank Dr. Lavier Gomes and Ms. Sheryl Foster and the Department of Radiology at Westmead Hospital for their substantial contributions to magnetic resonance imaging data acquisition. We thank Tracey Tsang who served as the iSPOT-A trial coordinator, along with Sariah Hobby, Yennie Hyunh and Jodie Logan who assisted in data acquisition. We also thank the individuals who gave their time to participate in the study.

Citation: Saad JF, Griffiths KR, Kohn MR, Braund TA, Clarke S, Williams LM, et al. (2021) No support for white matter connectivity differences in the combined and inattentive ADHD presentations. PLoS ONE 16(5): e0245028. https://doi.org/10.1371/journal.pone.0245028

References

1. Polanczyk GV, Salum GA, Sugaya LS, Caye A, Rohde LA. Annual research review: A meta-analysis of the worldwide prevalence of mental disorders in children and adolescents. J Child Psychol Psychiatry. 2015;56(3):345–65. pmid:25649325

2. DSM-V. Diagnostic and Statistical Manual of Mental Disorders (DSM-5®): American Psychiatric Publishing; 2013.

3. Solanto M, Gilbert S, Raj A, Zhu J, Pope-Boyd Sb, Stepak B, et al. Neurocognitive Functioning in AD/HD, Predominantly Inattentive and Combined Subtypes. J Abnorm Child Psychol. 2007;35(5):729–44. pmid:17629724

4. Williams LM. Precision psychiatry: a neural circuit taxonomy for depression and anxiety. The Lancet Psychiatry. 2016;3(5):472–80. pmid:27150382

5. Bush G. Attention-deficit/hyperactivity disorder and attention networks. Neuropsychopharmacology: official publication of the American College of Neuropsychopharmacology. 2010;35(1):278–300. pmid:19759528

6. Saad JF, Griffiths KR, Kohn MR, Clarke S, Williams LM, Korgaonkar MS. Regional brain network organization distinguishes the combined and inattentive subtypes of Attention Deficit Hyperactivity Disorder. NeuroImage Clinical. 2017;15:383–90. pmid:28580295

7. Vilgis V, Sun L, Chen J, Silk TJ, Vance A. Global and local grey matter reductions in boys with ADHD combined type and ADHD inattentive type. Psychiatry Research-Neuroimaging. 2016;254:119–26. pmid:27399309

8. Pineda DA, Restrepo MA, Sarmiento RJ, Gutierrez JE, Vargas SA, Quiroz YT, et al. Statistical analyses of structural magnetic resonance imaging of the head of the caudate nucleus in Colombian children with attention-deficit hyperactivity disorder. J Child Neurol. 2002;17(2):97–105. pmid:11952084

9. Carmona S, Proal E, Hoekzema EA, Gispert J, Picado M, Moreno I, et al. Ventro-striatal reductions underpin symptoms of hyperactivity and impulsivity in attention-deficit/hyperactivity disorder. Biological psychiatry. 2009;66(10):972–7. pmid:19576573

10. Carmona S, Vilarroya O, Bielsa A, Tremols V, Soliva JC, Rovira M, et al. Global and regional gray matter reductions in ADHD: a voxel-based morphometric study. Neuroscience letters. 2005;389(2):88–93. pmid:16129560

11. Semrud-Clikeman M, Fine JG, Bledsoe J, Zhu DC. Regional Volumetric Differences Based on Structural MRI in Children With Two Subtypes of ADHD and Controls. Journal of attention disorders. 2014. pmid:25488955

12. Anderson A, Douglas PK, Kerr WT, Haynes VS, Yuille AL, Xie J, et al. Non-negative matrix factorization of multimodal MRI, fMRI and phenotypic data reveals differential changes in default mode subnetworks in ADHD. NeuroImage. 2014;102 Pt 1:207–19. pmid:24361664

13. Al-Amin M, Zinchenko A, Geyer T. Hippocampal subfield volume changes in subtypes of attention deficit hyperactivity disorder. Brain Research. 2018;1685:1–8. pmid:29427578

14. Matthews M, Nigg JT, Fair DA. Attention deficit hyperactivity disorder. Current topics in behavioral neurosciences. 2014;16:235–66. pmid:24214656

15. Alexander AL, Lee JE, Lazar M, Field AS. Diffusion Tensor Imaging of the Brain. Neurotherapeutics: the journal of the American Society for Experimental NeuroTherapeutics. 2007;4(3):316–29. pmid:17599699

16. Svatkova A, Nestrasil I, Rudser K, Goldenring Fine J, Bledsoe J, Semrud-Clikeman M. Unique white matter microstructural patterns in ADHD presentations-a diffusion tensor imaging study. Hum Brain Mapp. 2016. pmid:27159198

17. Ercan ES, Suren S, Bacanlı A, Yazıcı KU, Callı C, Ardic UA, et al. Altered structural connectivity is related to attention deficit/hyperactivity subtypes: A DTI study. Psychiatry Research: Neuroimaging. 2016. pmid:27130841

18. Lei D, Ma J, Du XX, Shen GH, Jin XM, Gong QY. Microstructural Abnormalities in the Combined and Inattentive Subtypes of Attention Deficit Hyperactivity Disorder: a Diffusion Tensor Imaging Study. Scientific Reports. 2014;4. pmid:25363043

19. Ashtari M, Kumra S, Bhaskar SL, Clarke T, Thaden E, Cervellione KL, et al. Attention-deficit/hyperactivity disorder: a preliminary diffusion tensor imaging study. Biol Psychiatry. 2005;57(5):448–55. pmid:15737658

20. Xia S, Li X, Kimball AE, Kelly MS, Lesser I, Branch C. Thalamic shape and connectivity abnormalities in children with attention-deficit/hyperactivity disorder. Psychiatry Res. 2012;204(2–3):161–7. pmid:23149038

21. Sporns O. Structure and function of complex brain networks. Dialogues in Clinical Neuroscience. 2013;15(3):247–62. pmid:24174898

22. Hong SB, Zalesky A, Fornito A, Park S, Yang YH, Park MH, et al. Connectomic Disturbances in Attention-Deficit/Hyperactivity Disorder: A Whole-Brain Tractography Analysis. Biological Psychiatry. 2014;76(8):656–63. pmid:24503470

23. Smith SM, Jenkinson M, Johansen-Berg H, Rueckert D, Nichols TE, Mackay CE, et al. Tract-based spatial statistics: Voxelwise analysis of multi-subject diffusion data. NeuroImage. 2006;31(4):1487–505. pmid:16624579

24. Elliott GR, Blasey C, Rekshan W, Rush AJ, Palmer DM, Clarke S, et al. Cognitive Testing to Identify Children With ADHD Who Do and Do Not Respond to Methylphenidate. Journal of attention disorders. 2014.

25. Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, et al. The Mini-International Neuropsychiatric Interview (MINI): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. Journal of clinical psychiatry. 1998. pmid:9881538

26. DuPaul GJ. ADHD Rating Scale-IV: Checklists, Norms, and Clinical Interpretation: Guilford Press; 1998.

27. Smith SM, Jenkinson M, Woolrich MW, Beckmann CF, Behrens TE, Johansen-Berg H, et al. Advances in functional and structural MR image analysis and implementation as FSL. NeuroImage. 2004;23 Suppl 1:S208–19. pmid:15501092

28. Korgaonkar MS, Grieve SM, Koslow SH, Gabrieli JD, Gordon E, Williams LM. Loss of white matter integrity in major depressive disorder: evidence using tract-based spatial statistical analysis of diffusion tensor imaging. Hum Brain Mapp. 2011;32(12):2161–71. pmid:21170955

29. Mori S, Wakana S, van Zijl PCM, Nagae-Poetscher LM. MRI Atlas of Human White Matter: Elsevier Science; 2005.

30. Hernandez-Fernandez M, Reguly I, Jbabdi S, Giles M, Smith S, Sotiropoulos SN. Using GPUs to accelerate computational diffusion MRI: From microstructure estimation to tractography and connectomes. NeuroImage. 2019;188:598–615. pmid:30537563

31. Desikan RS, Segonne F, Fischl B, Quinn BT, Dickerson BC, Blacker D, et al. An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. NeuroImage. 2006;31(3):968–80. pmid:16530430

32. Korgaonkar MS, Fornito A, Williams LM, Grieve SM. Abnormal structural networks characterize major depressive disorder: a connectome analysis. Biol Psychiatry. 2014;76(7):567–74. pmid:24690111

33. Rubinov M, Sporns O. Complex network measures of brain connectivity: Uses and interpretations. NeuroImage. 2010;52(3):1059–69. pmid:19819337

34. Van Hecke W, Leemans A, Emsell L. DTI Analysis Methods: Voxel-Based Analysis. In: Van Hecke W, Emsell L, Sunaert S, editors. Diffusion Tensor Imaging: A Practical Handbook. New York, NY: Springer New York; 2016. p. 183–203.

35. Silk T, Vance A, Rinehart N, Bradshaw JL, Cunnington R. White-matter abnormalities in attention deficit hyperactivity disorder: a diffusion tensor imaging study. Hum Brain Mapp. 2009;30(9):2757–65. pmid:19107752

36. Witt ST, Stevens MC. Relationship between white matter microstructure abnormalities and ADHD symptomatology in adolescents. Psychiatry Research-Neuroimaging. 2015;232(2):168–74. pmid:25795595

37. Hamilton LS, Levitt JG, O’Neill J, Alger JR, Luders E, Phillips OR, et al. Reduced white matter integrity in attention-deficit hyperactivity disorder. Neuroreport. 2008;19(17):1705–8. pmid:18841089

38. Wu ZM, Bralten J, Cao QJ, Hoogman M, Zwiers MP, An L, et al. White Matter Microstructural Alterations in Children with ADHD: Categorical and Dimensional Perspectives. Neuropsychopharmacology: official publication of the American College of Neuropsychopharmacology. 2017;42(2):572–80. pmid:27681441

39. Bouziane C, Caan MWA, Tamminga HGH, Schrantee A, Bottelier MA, de Ruiter MB, et al. ADHD and maturation of brain white matter: A DTI study in medication naive children and adults. NeuroImage: Clinical. 2018;17:53–9. pmid:29527472

40. Shaw P, Eckstrand K, Sharp W, Blumenthal J, Lerch JP, Greenstein D, et al. Attention-deficit/hyperactivity disorder is characterized by a delay in cortical maturation. Proceedings of the National Academy of Sciences of the United States of America. 2007;104(49):19649–54. pmid:18024590

41. Rossi AS, de Moura LM, de Mello CB, de Souza AA, Muszkat M, Bueno OF. Attentional Profiles and White Matter Correlates in Attention-Deficit/Hyperactivity Disorder Predominantly Inattentive Type. Front Psychiatry. 2015;6:122. pmid:26441684

42. Nagel BJ, Bathula D, Herting M, Schmitt C, Kroenke CD, Fair D, et al. Altered white matter microstructure in children with attention-deficit/hyperactivity disorder. J Am Acad Child Adolesc Psychiatry. 2011;50(3):283–92. pmid:21334568

43. Francx W, Llera A, Mennes M, Zwiers MP, Faraone SV, Oosterlaan J, et al. Integrated analysis of gray and white matter alterations in attention-deficit/hyperactivity disorder. NeuroImage Clinical. 2016;11:357–67. pmid:27298764

44. Sidlauskaite J, Sonuga-Barke E, Roeyers H, Wiersema JR. Altered intrinsic organisation of brain networks implicated in attentional processes in adult attention-deficit/hyperactivity disorder: a resting-state study of attention, default mode and salience network connectivity. European archives of psychiatry and clinical neuroscience. 2016;266(4):349–57. pmid:26260900

45. Lopez-Larson MP, King JB, Terry J, McGlade EC, Yurgelun-Todd D. Reduced insular volume in attention deficit hyperactivity disorder. Psychiatry research. 2012;204(1):32–9. pmid:23142193

46. Rubia K, Norman L, Lukito S, Alegria A, Wulff M. Neuroimaging of ADHD: From matter over mind to mind over matter. Frontiers in Human Neuroscience.

47. Fair DA, Nigg JT, Iyer S, Bathula D, Mills KL, Dosenbach NU, et al. Distinct neural signatures detected for ADHD subtypes after controlling for micro-movements in resting state functional connectivity MRI data. Frontiers in systems neuroscience. 2012;6:80. pmid:23382713

48. Zhao Q, Li H, Yu X, Huang F, Wang Y, Liu L, et al. Abnormal Resting-State Functional Connectivity of Insular Subregions and Disrupted Correlation with Working Memory in Adults with Attention Deficit/Hyperactivity Disorder. Frontiers in Psychiatry. 2017;8(200). pmid:29075206

49. Fall S, Querne L, Le Moing AG, Berquin P. Individual differences in subcortical microstructure organization reflect reaction time performances during a flanker task: a diffusion tensor imaging study in children with and without ADHD. Psychiatry Res. 2015;233(1):50–6. pmid:26025013

50. Saad JF, Griffiths KR, Korgaonkar MS. A Systematic Review of Imaging Studies in the Combined and Inattentive Subtypes of Attention Deficit Hyperactivity Disorder. Frontiers in Integrative Neuroscience. 2020;14(31). pmid:32670028

51. Willcutt EG, Nigg JT, Pennington BF, Solanto MV, Rohde LA, Tannock R, et al. Validity of DSM-IV attention deficit/hyperactivity disorder symptom dimensions and subtypes. J Abnorm Psychol. 2012;121(4):991–1010. pmid:22612200

52. Alexander-Bloch A, Giedd JN, Bullmore E. Imaging structural co-variance between human brain regions. Nature reviews Neuroscience. 2013;14(5):322–36. pmid:23531697

53. Raffelt DA, Tournier JD, Smith RE, Vaughan DN, Jackson G, Ridgway GR, et al. Investigating white matter fibre density and morphology using fixel-based analysis. NeuroImage. 2017;144(Pt A):58–73. pmid:27639350

54. Andersson J, Jenkinson M, Smith S. FMRIB technical report TR07JA2. FMRIB Analysis Group of the University of Oxford. 2007.

55. Jenkinson M, Beckmann CF, Behrens TE, Woolrich MW, Smith SM. FSL. NeuroImage. 2012;62(2):782–90. pmid:21979382

56. Bach M, Laun FB, Leemans A, Tax CMW, Biessels GJ, Stieltjes B, et al. Methodological considerations on tract-based spatial statistics (TBSS). NeuroImage. 2014;100:358–69. pmid:24945661

57. Schwarz CG, Reid RI, Gunter JL, Senjem ML, Przybelski SA, Zuk SM, et al. Improved DTI registration allows voxel-based analysis that outperforms tract-based spatial statistics. NeuroImage. 2014;94:65–78. pmid:24650605

58. de Groot M, Vernooij MW, Klein S, Ikram MA, Vos FM, Smith SM, et al. Improving alignment in Tract-based spatial statistics: evaluation and optimization of image registration. NeuroImage. 2013;76:400–11. pmid:23523807

59. Yap QJ, Teh I, Fusar-Poli P, Sum MY, Kuswanto C, Sim K. Tracking cerebral white matter changes across the lifespan: insights from diffusion tensor imaging studies. Journal of Neural Transmission. 2013;120(9):1369–95. pmid:23328950

60. Lebel C, Gee M, Camicioli R, Wieler M, Martin W, Beaulieu C. Diffusion tensor imaging of white matter tract evolution over the lifespan. NeuroImage. 2012;60(1):340–52. pmid:22178809

61. Beare R, Adamson C, Bellgrove MA, Vilgis V, Vance A, Seal ML, et al. Altered structural connectivity in ADHD: a network based analysis. Brain Imaging and Behavior. 2016:1–13. pmid:25749917

62. Baddeley A, Emslie H, Nimmo-Smith I. The Spot-the-Word test: a robust estimate of verbal intelligence based on lexical decision. Br J Clin Psychol. 1993;32(1):55–65. pmid:8467274

63. Silverstein SM, Berten S, Olson P, Paul R, Willams LM, Cooper N, et al. Development and validation of a World-Wide-Web-based neurocognitive assessment battery: WebNeuro. Behav Res Methods. 2007;39(4):940–9. pmid:18183911