The Potential of Trial-by-Trial Variabilities of Ongoing-EEG, Evoked Potentials, Event Related Potentials and fMRI as Diagnostic Markers for Neuropsychiatric Disorders

Abstract

The laser pulses excited Aδ and C nociceptors in the superficial skin layers and they produced a high amplitude response in the brain, which was processed with multiple linear regression approaches in the trial-by-trial analysis. According to Gluth and Rieskamp (2017), the trial-by-trial variability is a useful way to link variance in behavior that cannot be exclusively explained by cognitive modeling with ongoing-EEG, ERPs, and fMRI. [...]based on the abovementioned references, we could mention three essential aspects that highlight the importance of the trial-by-trial variability analysis of ongoing-EEG, ERPs, and fMRI responses of the impaired brain. [...]it makes possible the analysis of the relationships in the variation among the ongoing-EEG-prestimulus brain-states, the trial-by-trial ERPs, and the associated behavioral outcomes (De Blasio and Barry, 2018). [...]the combined use of ongoing-EEG, EPs, ERPs and fMRI signals with their trial-by-trial variability analysis could be of great importance in the mechanistic exploration of neuropsychiatric disorders.

The “Trial-by-trial Variability” and Synonyms

The theme of “neuronal variability” is of great importance in the characterization of the “inter and intra-subject variability,” a concept that is becoming crucial to understand neuronal adaptation and plasticity in aging, and many pathological conditions of the brain. There are many comprehensive reviews about the “inter and intra-subject variability” of many behavioral and brain signals, in patients with neurological disorders and healthy controls (MacDonald et al., 2006, 2009; Faisal et al., 2008; Fontanini and Katz, 2008; Geurts et al., 2008; Vanrullen et al., 2011; Dykiert et al., 2012; Tamm et al., 2012; Garrett et al., 2013; Kofler et al., 2013; Dinstein et al., 2015; Dhawale et al., 2017; Ouyang et al., 2017; Tian et al., 2017; Haigh, 2018). However, there are no reviews focused on the “inter and intra-subject variability” for evoked potentials (EP) or event related potentials (ERP).

We found that many authors employed different terms to describe neuronal variability as: trial-by-trial, trial-to-trial, single-trial, inter-trial, intertrial. Also, many references are as old as the origin of the EEG technique, introduced by Berger (1929). For instance, Adrian and Yamagiwa (1935) noted that there are considerable variations in one subject and another regarding the fluctuation in size of the ongoing-EEG waves. In the same way, Barlow (1959), Davis and Zerlin (1964) studied the variability of EPs with the idea that such variability could serve to obtain normative data, which could explain behavioral and developmental observations in patients with neuropsychiatric disorders and healthy controls (see also Zerlin and Davis, 1967; Cigánek, 1969; Lifshitz, 1969; Callaway et al., 1970).

A constructive critic for the reviews mentioned in the first paragraph of this section is that the authors of such reviews did not cite relevant studies of trial-by-trial EP variability. For example, they did not cite the study by Callaway and Halliday (1973), who found that the variability of visual and auditory EPs decreased with increased age. They also ignored an essential study by Callaway and Halliday (1973), who were the first to perform a test and retest of the EPs separated by a year, as recently was done by Arazi et al. (2017). Another point no discussed in such reviews is that the results by Williams et al. (2005) are consistent with the pioneer findings of Callaway and Halliday (1973). In fact, Williams et al. (2005) found that the behavioral variability (choice reaction time) falls as age increases. In the same way, Callaway and Halliday (1973) found that the EP variability also falls as age increases. This means that both variabilities change as a function of age.

A possible explanation for these omissions of the old literature could be the indiscriminate use of different terms to refer to the “neuronal variability.” Another possible explanation for such omissions is the use of different synonyms for the terms “inter and intra-subject variability.” The intra-subject variability is the variability in behavior, or in any signal, of a single individual measured across multiple time points. In contrast, inter-individual variability is defined as the variability in behavior or in any signal exhibited by multiple individuals at a single time point.

As a conclusion of this section, we highlight that the terminology should be defined in consensus to quickly localize the literature in this field and guarantee a fluid advance of new questions and clinical applications of the trial-by-trial variability.

The “Trial-by-trial Variabilities” of Ongoing-EEG, ERPs and fMRI Signals as Neuronal Markers of Neuropsychiatric Disorders. Origins and New Tendencies

Two pioneering studies motivated the analysis of the trial-by-trial variability of EPs. Barlow published the first in 1959, who invented an electronic device to detect the variance of EPs analogically. Zerlin and Davis published the second study in 1967, who presented compelling data from a subject with uniquely high amplitude auditory EPs (not averaged). Such reports, supported the fact that it is possible to obtain normative data for the variability of EPs, even that they only could be detected by averaging techniques (Cigánek, 1968), or by multiple linear regression approaches in the trial-by-trial analysis (Iannetti et al., 2005, 2006). In fact, Iannetti et al. (2005, 2006) also recorded EPs (not averaged) as Zerlin and Davis (1967). They were able to elicit trial-by-trial EPs related to a brief painful stimulus with a laser. The laser pulses excited Aδ and C nociceptors in the superficial skin layers and they produced a high amplitude response in the brain, which was processed with multiple linear regression approaches in the trial-by-trial analysis.

Jones et al. (1965), Marjerrison et al. (1968), Lifshitz (1969), and Callaway et al. (1970) published other pioneering studies, reporting that the EP variability correlates with neuropsychiatric disorders, behavior, and aging. Jones et al. (1965), Marjerrison et al. (1968), Lifshitz (1969), and Callaway et al. (1970) found that the variability of EPs is high both in schizophrenia and in young children. These seminal reports motivated the use of the trial-by-trial analysis of the EPs as a potential neuronal marker of disease, and they were a preamble for the analysis of behavioral variability across the lifespan (see review in MacDonald et al., 2006). In the case of schizophrenia, many studies reported that these patients tend to have smaller and more variable EPs than healthy subjects (Jones and Callaway, 1970; Saletu et al., 1971; Borge, 1973; Cohen, 1973; Rappaport et al., 1975; Buchsbaum and Coppola, 1979; Winterer and Weinberger, 2004; Jansen et al., 2010; Shin et al., 2015). Consistently, there is also a greater trial-by-trial variability in autism than in healthy controls (Milne, 2011; Dinstein et al., 2012; Haigh et al., 2014).

Furthermore, the concurrent or separated trial-by-trial analysis of the ongoing-EEG, EPs, ERPs and fMRI could be the future in the field of the inter- and intra-subject variability, in patients with schizophrenia, autism, and other neuropsychiatric disorders. For instance, Haigh et al. (2017) examined differential sensory fMRI signatures in autism and schizophrenia via an analysis of amplitude and trial-by-trial variability. These authors found that autism was associated with more considerable trial-by-trial variability, whereas schizophrenia was associated with smaller response amplitudes. Future studies will be necessary to explain whether the trial-by-trial variation in fMRI signatures could be used as a reliable biomarker for schizophrenia as the trial-by-trial analysis of EPs.

In this context, there are many reports obtained from healthy subjects that could be replicated in patients with neuropsychiatric disorders. In this paragraph, we describe some of them in chronological order. For example, Debener et al. (2005) demonstrated that the single trial-by-trial error-related negativity of the EEG reflects quick behavioral adjustments that predict the fMRI activity of brain regions engaged in the processing of response errors. Mantini et al. (2009), employed the trial-by-trial EEG/fMRI correlation approach and ERPs during target detection, to show that the ventral attention network was transiently activated by the occurrence of targets, while the dorsal attention network reflects sustained activity. In a similar way, Goldman et al. (2009), employed the trial-by-trial analysis of the combined use of ongoing-EEG, ERPs and fMRI to identify cortical areas associated with EEG components related to an oddball auditory paradigm. Also, Hesselmann et al. (2008), used the trial-by-trial analysis of the ongoing-EEG, ERPs, and fMRI to show that the fMRI signals in the inferior parietal lobe and precentral gyrus, exhibit a significant covariation with the P3 related activity. In another study, Mayhew et al. (2013), performed the simultaneous ongoing-EEG/fMRI recording during rest and during the noxious thermal stimulation, discovering an interesting interaction between the alpha-power, the fMRI activity, and the trial-by-trial variability of the behavioral report of pain. Other studies also combined the analysis of EEG/fMRI signals with the trial-by-trial variability in the context of neuronal connectivity (Goltz et al., 2015) and working memory maintenance (Scheeringa et al., 2016). According to Gluth and Rieskamp (2017), the trial-by-trial variability is a useful way to link variance in behavior that cannot be exclusively explained by cognitive modeling with ongoing-EEG, ERPs, and fMRI.

Therefore, based on the abovementioned references, we could mention three essential aspects that highlight the importance of the trial-by-trial variability analysis of ongoing-EEG, ERPs, and fMRI responses of the impaired brain. First, the increased variability in the behavioral and trial-by-trial ERPs is a universal feature in patients with schizophrenia and autism. Second, it makes possible the analysis of the relationships in the variation among the ongoing-EEG-prestimulus brain-states, the trial-by-trial ERPs, and the associated behavioral outcomes (De Blasio and Barry, 2018). This second aspect is particularly important because the study of such variation across the life-span may allow exploring mechanisms of healthy and pathological aging (De Blasio and Barry, 2018). Third, the combined use of ongoing-EEG, EPs, ERPs and fMRI signals with their trial-by-trial variability analysis could be of great importance in the mechanistic exploration of neuropsychiatric disorders.

As a conclusion of this section, we highlight that the trial-by-trial variability analysis of ongoing-EEG, EPs, ERPs, and fMRI signals could be considered as a potential neuronal marker for neuropsychiatric disorders, as schizophrenia, Alzheimer disease, depression, borderline personality disorder, autism, and attention deficit hyperactivity disorder (Schwartz et al., 1989; Hogan et al., 2006; Geurts et al., 2008; Kaiser et al., 2008; Dinstein et al., 2015; Saville et al., 2015).

Limitations and Recommendations for the Analysis of Trial-by-trial Variability of EEG/fMRI Signals

A limitation of the use of the trial-by-trial variability as a diagnostic marker includes the requirement of a high number of single trials to extract reliable ERPs and behavioral patterns (Boudewyn et al., 2018). On the other hand, most of the patients are medicated and this could add a possible bias in the data interpretation. Castellanos et al. (2005) and Kaiser et al. (2008), mention that a serious confound may arise from the dopamine receptor blocking medication in patients with schizophrenia, because an impairment of catecholaminergic neurotransmission has been postulated as a cause of increased intra-subject variability. Another limitation is that we know little about the physiological mechanisms for the relationships between the trial-by-trial variability of the ongoing-EEG, EPs, ERPs, fMRI, and behavior. Future work will be necessary to disentangle such relationships in patients with diverse neuropsychiatric disorders and healthy controls (MacDonald et al., 2006; Kaiser et al., 2008). Another limitation is the cooperation of the same subjects in the EEG aging paradigms across the lifespan. For instance, it would be complicated to obtain a similar U-like curve for the trial-by-trial EPs, as the curve obtained by Williams et al. (2005) for the reaction time across the lifespan (see also MacDonald et al., 2006). Such type of studies would require the development of new wearable EEG systems that could be used across the lifespan (Tian et al., 2019). Such systems would require new technologies for the development of low power, small size, and imperceptible weight, and they are in state of the art for wearable EEG solutions.

Because in the literature there are many terms for the “trial-by-trial variability” and the “inter and intra-subject” variability, we recommend the standardized use of these terms in future publications and a consensus statement. In the same context, we recommend the writing of a systematic review and meta-analysis about this issue, following similar methods as Dykiert et al. (2012).

Author Contributions

All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.

Funding

The Dorothy Feiss Scientific Research Grant, Deutsche Forschungsgemeinschaft (DFG)-Sonderforschungsbereich (SFB) 1280. We thank the support by the Cátedra Marcos Moshinsky (EM) and VIEP-BUAP (AG-R and VL-C). The publication of this article was funded by the Open Access Fund of the Leibniz Association.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

Adrian, E. D., and Yamagiwa, K. (1935). The origin of the Berger rhythm. Brain 58, 323–351. doi: 10.1093/brain/58.3.323

Arazi, A., Gonen-Yaacovi, G., and Dinstein, I. (2017). The magnitude of trial-by-trial neural variability is reproducible over time and across tasks in humans. Eneuro 4, 1–12. doi: 10.1523/ENEURO.0292-17.2017

Barlow, J. S. (1959). A small electronic analogue averager and variance computer for evoked potentials of the brain. Med. Electron. 1959, 113–119.

Berger, H. (1929). Über das elektrenkephalogramm des menschen. Archiv. Psychiatr. Nervenkrankheiten 87, 527–570. doi: 10.1007/BF01797193

Borge, G. F. (1973). Perceptual modulation and variability in psychiatric patients. Arch. Gen. Psychiatry 29, 760–763. doi: 10.1001/archpsyc.1973.04200060046006

Boudewyn, M. A., Luck, S. J., Farrens, J. L., and Kappenman, E. S. (2018). How many trials does it take to get a significant ERP effect? It depends. Psychophysiology 55:e13049. doi: 10.1111/psyp.13049

Buchsbaum, M. S., and Coppola, R. (1979). “Signal-to-noise ration and response variability in affective disorders and schizophrenia,” in Evoked Brain Potentials and Behavior. The Downstate Series of Research in Psychiatric and Psychology, Vol. 2, eds H. Begleiter (Boston, MA: Springer). doi: 10.1007/978-1-4684-3462-0_20

Callaway, E., and Halliday, R. A. (1973). Evoked potential variability: effects of age, amplitude and methods of measurement. Electroencephalogr. Clin. Neurophysiol. 34, 125–133. doi: 10.1016/0013-4694(73)90039-4

Callaway, I. I. I. E, Jones, R. T., and Donchin, E. (1970). Auditory evoked potential variability in schizophrenia. Electroencephalogr. Clin. Neurophysiol. 29, 421–428. doi: 10.1016/0013-4694(70)90059-3

Castellanos, F. X., Sonuga-Barke, E. J., Scheres, A., Di Martino, A., Hyde, C., and Walters, J. R. (2005). Varieties of attention-deficit/ hyperactivity disorder-related intra-individual variability. Biol. Psychiatry 57, 1416–1423. doi: 10.1016/j.biopsych.2004.12.005

Cigánek, L. (1968). Some problems of measurement of the evoked potentials. Electroencephalogr. Clin. Neurophysiol. 24:294. doi: 10.1016/0013-4694(68)90209-5

Cigánek, L. (1969). Variability of the human visual evoked potential: normative data. Electroencephalogr. Clin. Neurophysiol. 27, 35–42. doi: 10.1016/0013-4694(69)90106-0

Cohen, R. (1973). The Influence of Task-Irrelevant Stimulus Variations on the Reliability of Auditory Evoked Responses in Schizophrenia. Average Evoked Responses and Their Conditioning in Normal Subjects and Psychiatric Patients. Paris: INSERM, 373–388.

Davis, H., and Zerlin, S. (1964). The variability of evoked responses. Physiologist 7:114.

De Blasio, F. M., and Barry, R. J. (2018). Prestimulus delta and theta contributions to equiprobable Go/NoGo processing in healthy ageing. Int. J. Psychophysiol. 130, 40–52. doi: 10.1016/j.ijpsycho.2018.05.005

Debener, S., Ullsperger, M., Siegel, M., Fiehler, K., von Cramon, D. Y., and Engel, A. K. (2005). Trial-by-trial coupling of concurrent electroencephalogram and functional magnetic resonance imaging identifies the dynamics of performance monitoring. J. Neurosci. 25, 11730–11737. doi: 10.1523/JNEUROSCI.3286-05.2005

Dhawale, A. K., Smith, M. A., and Ölveczky, B. P. (2017). The role of variability in motor learning. Ann. Rev. Neurosci. 40, 479–498. doi: 10.1146/annurev-neuro-072116-031548

Dinstein, I., Heeger, D. J., and Behrmann, M. (2015). Neural variability: friend or foe? Trends Cogn. Sci. 19, 322–328. doi: 10.1016/j.tics.2015.04.005

Dinstein, I., Heeger, D. J., Lorenzi, L., Minshew, N. J., Malach, R., and Behrmann, M. (2012). Unreliable evoked responses in autism. Neuron 75, 981–991. doi: 10.1016/j.neuron.2012.07.026

Dykiert, D., Der, G., Starr, J. M., and Deary, I. J. (2012). Age differences in intra-individual variability in simple and choice reaction time: systematic review and meta- analysis. PLoS ONE 7:e45759. doi: 10.1371/journal.pone.0045759

Faisal, A. A., Selen, L. P., and Wolpert, D. M. (2008). Noise in the nervous system. Nat. Rev. Neurosci. 9:292. doi: 10.1038/nrn2258

Fontanini, A., and Katz, D. B. (2008). Behavioral states, network states, and sensory response variability. J. Neurophysiol. 100, 1160–1168. doi: 10.1152/jn.90592.2008

Garrett, D. D., Samanez-Larkin, G. R., MacDonald, S. W., Lindenberger, U., McIntosh, A. R., and Grady, C. L. (2013). Moment-to-moment brain signal variability: a next frontier in human brain mapping? Neurosci. Biobehav. Rev. 37, 610–624. doi: 10.1016/j.neubiorev.2013.02.015

Geurts, H. M., Grasman, R. P., Verté, S., Oosterlaan, J., Roeyers, H., van Kammen, S. M., et al. (2008). Intra-individual variability in ADHD, autism spectrum disorders and Tourette's syndrome. Neuropsychologia 46, 3030–3041. doi: 10.1016/j.neuropsychologia.2008.06.013

Gluth, S., and Rieskamp, J. (2017). Variability in behavior that cognitive models do not explain can be linked to neuroimaging data. J. Math. Psychol. 76, 104–116. doi: 10.1016/j.jmp.2016.04.012

Goldman, R. I., Wei, C. Y., Philiastides, M. G., Gerson, A. D., Friedman, D., Brown, T. R., et al. (2009). Single-trial discrimination for integrating simultaneous EEG and fMRI: Identifying cortical areas contributing to trial-to-trial variability in the auditory oddball task. Neuroimage 47, 136–147. doi: 10.1016/j.neuroimage.2009.03.062

Goltz, D., Gundlach, C., Nierhaus, T., Villringer, A., Müller, M., and Pleger, B. (2015). Connections between intraparietal sulcus and a sensorimotor network underpin sustained tactile attention. J. Neurosci. 35, 7938–7949. doi: 10.1523/JNEUROSCI.3421-14.2015

Haigh, S. M. (2018). Variable sensory perception in autism. Eur. J. Neurosci. 47, 602–609. doi: 10.1111/ejn.13601

Haigh, S. M., Gupta, A., Barb, S. M., Glass, S. A., Minshew, N. J., Dinstein, I., et al. (2017). Differential sensory fMRI signatures in autism and schizophrenia: analysis of amplitude and trial-to-trial variability. Schizophr. Res. 175, 12–19. doi: 10.1016/j.schres.2016.03.036

Haigh, S. M., Heeger, D. J., Dinstein, I., Minshew, N., and Behrmann, M. (2014). Cortical variability in the sensory-evoked response in autism. J. Autism Dev. Disord. 45, 1176–1190. doi: 10.1007/s10803-014-2276-6

Hesselmann, G., Kell, C. A., Eger, E., and Kleinschmidt, A. (2008). Spontaneous local variations in ongoing neural activity bias perceptual decisions. Proc. Natl. Acad. Sci. U.S.A. 105, 10984–10989. doi: 10.1073/pnas.0712043105

Hogan, M. J., Carolan, L., Roche, R. A., Dockree, P. M., Kaiser, J., Bunting, B. P., et al. (2006). Electrophysiological and information processing variability predicts memory decrements associated with normal age-related cognitive decline and Alzheimer's disease (AD). Brain Res. 1119, 215–226. doi: 10.1016/j.brainres.2006.08.075

Iannetti, G. D., Zambreanu, L., Cruccu, G., and Tracey, I. (2005). Operculoinsular cortex encodes pain intensity at the earliest stages of cortical processing as indicated by amplitude of laser-evoked potentials in humans. Neuroscience 131, 199–208. doi: 10.1016/j.neuroscience.2004.10.035

Iannetti, G. D., Zambreanu, L., and Tracey, I. (2006). Similar nociceptive afferents mediate psychophysical and electrophysiological responses to heat stimulation of glabrous and hairy skin in humans. J. Physiol. 577 (Pt 1), 235–248. doi: 10.1113/jphysiol.2006.115675

Jansen, B. H., Hu, L., and Boutros, N. N. (2010). Auditory evoked potential variability in healthy and schizophrenia subjects. Clin. Neurophysiol. 121, 1233–1239. doi: 10.1016/j.clinph.2010.03.006

Jones, R. T., Blacker, K. H., Callaway, E., and Laine, R. S. (1965). The auditory evoked response as a diagnostic and prognostic measure in schizophrenia. Am. J. Psychiatry 122, 33–41. doi: 10.1176/ajp.122.1.33

Jones, R. T., and Callaway, E. (1970). Auditory evoked response in schizophrenia: a reassessment. Biol. Psychiatry 2, 291–298

Kaiser, S., Roth, A., Rentrop, M., Friederich, H. C., Bender, S., and Weisbrod, M. (2008). Intra-individual reaction time variability in schizophrenia, depression and borderline personality disorder. Brain Cogn. 66, 73–82. doi: 10.1016/j.bandc.2007.05.007

Kofler, M. J., Rapport, M. D., Sarver, D. E., Raiker, J. S., Orban, S. A., Friedman, L. M., et al. (2013). Reaction time variability in ADHD: a meta-analytic review of 319 studies. Clin. Psychol. Rev. 33, 795–811. doi: 10.1016/j.cpr.2013.06.001

Lifshitz, K. (1969). Intra and inter individual variability of the averaged evoked potential in normal and chronic schizophrenic subjects. Electroencephalogr. Clin. Neurophysiol. 27, 688–689. doi: 10.1016/0013-4694(69)91311-X

MacDonald, S. W., Li, S. C., and Bäckman, L. (2009). Neural underpinnings of within-person variability in cognitive functioning. Psychol. Aging 24, 792–808. doi: 10.1037/a0017798

MacDonald, S. W., Nyberg, L., and Bäckman, L. (2006). Intra-individual variability in behavior: links to brain structure, neurotransmission and neuronal activity. Trends Neurosci. 29, 474–480. doi: 10.1016/j.tins.2006.06.011

Mantini, D., Corbetta, M., Perrucci, M. G., Romani, G. L., and Del Gratta, C. (2009). Large-scale brain networks account for sustained and transient activity during target detection. Neuroimage 44, 265–274. doi: 10.1016/j.neuroimage.2008.08.019

Marjerrison, G., Krause, A. E., and Keogh, R. P. (1968). Variability of the EEG in schizophrenia: quantitative analysis with a modulus voltage integrator. Electroencephalogr. Clin. Neurophysiol. 24, 35–41. doi: 10.1016/0013-4694(68)90064-3

Mayhew, S. D., Hylands-White, N., Porcaro, C., Derbyshire, S. W., and Bagshaw, A. P. (2013). Intrinsic variability in the human response to pain is assembled from multiple, dynamic brain processes. Neuroimage 75, 68–78. doi: 10.1016/j.neuroimage.2013.02.028

Milne, E. (2011). Increased intra-participant variability in children with autistic spectrum disorders: evidence from single-trial analysis of evoked EEG. Front. Psychol. 2:51. doi: 10.3389/fpsyg.2011.00051

Ouyang, G., Hildebrandt, A., Sommer, W., and Zhou, C. (2017). Exploiting the intra-subject latency variability from single-trial event-related potentials in the P3 time range: a review and comparative evaluation of methods. Neurosci. Biobehav. Rev. 75, 1–21. doi: 10.1016/j.neubiorev.2017.01.023

Rappaport, M., Hopkins, H. K., Hall, K., Belleza, T., and Hall, R. A. (1975). Schizophrenia and evoked potentials: maximum amplitude, frequency of peaks, variability, and phenothiazine effects. Psychophysiology 12, 196–207. doi: 10.1111/j.1469-8986.1975.tb01276.x

Saletu, B., Itil, T. M., and Saletu, M. (1971). Auditory evoked response, EEG, and thought process in schizophrenics. Am. J. Psychiatry 128, 336–344. doi: 10.1176/ajp.128.3.336

Saville, C. W., Feige, B., Kluckert, C., Bender, S., Biscaldi, M., Berger, A., et al. (2015). Increased reaction time variability in attention-deficit hyperactivity disorder as a response-related phenomenon: evidence from single-trial event-related potentials. J. Child Psychol. Psychiatry 56, 801–813. doi: 10.1111/jcpp.12348

Scheeringa, R., Koopmans, P. J., van Mourik, T., Jensen, O., and Norris, D. G. (2016). The relationship between oscillatory EEG activity and the laminar-specific BOLD signal. Proc. Natl. Acad. Sci. U.S.A. 113, 6761–6766. doi: 10.1073/pnas.1522577113

Schwartz, F., Carr, A. C., Munich, R. L., Glauber, S., Lesser, B., and Murray, J. (1989). Reaction time impairment in schizophrenia and affective illness: the role of attention. Biol. Psychiatry 25, 540–548. doi: 10.1016/0006-3223(89)90214-X

Shin, K. S., Kim, J. S., Kim, S. N., Hong, K. S., O'Donnell, B. F., Chung, C. K., et al. (2015). Intraindividual neurophysiological variability in ultra-high-risk for psychosis and schizophrenia patients: single-trial analysis. NPJ Schizophr. 1:15031. doi: 10.1038/npjschz.2015.31

Tamm, L., Narad, M. E., Antonini, T. N., O'Brien, K. M., Hawk, L. W. Jr., and Epstein, J. N. (2012). Reaction time variability in ADHD: a review. Neurotherapeutics 9, 500–508. doi: 10.1007/s13311-012-0138-5

Tian, L., Zimmerman, B., Akhtar, A., Yu, K. J., Moore, M., Wu, J., et al. (2019). Large-area MRI-compatible epidermal electronic interfaces for prosthetic control and cognitive monitoring. Nat. Biomed. Eng. 3, 194–205. doi: 10.1038/s41551-019-0347-x

Tian, Q., Chastan, N., Bair, W. N., Resnick, S. M., Ferrucci, L., and Studenski, S. A. (2017). The brain map of gait variability in aging, cognitive impairment and dementia-A systematic review. Neurosci. Biobehav. Rev. 74, 149–162. doi: 10.1016/j.neubiorev.2017.01.020

Vanrullen, R., Busch, N. A., Drewes, J., and Dubois, J. (2011). Ongoing EEG phase as a trial-by-trial predictor of perceptual and attentional variability. Front. Psychol. 2:60. doi: 10.3389/fpsyg.2011.00060

Williams, B. R., Hultsch, D. F., Strauss, E. H., Hunter, M. A., and Tannock, R. (2005). Inconsistency in reaction time across the lifespan. Neuropsychology 19, 88–96. doi: 10.1037/0894-4105.19.1.88

Winterer, G., and Weinberger, D. R. (2004). Genes, dopamine and cortical signal-to-noise ratio in schizophrenia. Trends Neurosci. 27, 683–690. doi: 10.1016/j.tins.2004.08.002

Zerlin, S., and Davis, H. (1967). The variability of single evoked vertex potentials in man. Electroencephalogr. Clin. Neurophysiol. 23, 468–472. doi: 10.1016/0013-4694(67)90190-3