EEG microstate features for schizophrenia classification

Abstract

[...]it is suitable for studying complex cognitive functions. [...]EEG can provide unique information that is otherwise difficult to obtain using imaging modalities. The features based on those microstates show differences between patients with schizophrenia and other groups and allow interpretation from a neuroscience perspective. [...]four archetype microstates were used not only in schizophrenia [24, 37–39, 46] but also in general medical conditions, such as physical exercise [54], insomnia [55], hearing loss [56]. With multivariate analysis, we can simultaneously analyze multiple dependent and independent variables to improve reliability and validity. [...]multivariate analysis can utilize all microstate-feature information and identify new patterns to improve understanding [58, 59]. Machine-learning techniques (e.g., classification using kernel method) accomplish multivariate analyses that catalog distinct observations and allocate new observations to previously defined groups [60]. [...]by applying machine-learning-based algorithms to microstate features, we can distinguish between EEG recordings of patients diagnosed with schizophrenia and those of healthy (control) subjects and present a practical application.

About the Authors:

Kyungwon Kim

Roles Conceptualization, Formal analysis, Investigation, Methodology, Project administration, Writing – original draft

Affiliations Department of Biomedical Science and Engineering (BMSE), Institute Integrated Technology (IIT), Gwangju Institute of Science and Technology (GIST), Cheomdan-gwagiro, Gwangju, South Korea, Department of Psychiatry and Biomedical Research Institute, Pusan National University Hospital, Busan, South Korea

Nguyen Thanh Duc

Roles Methodology, Writing – original draft

Affiliations Department of Biomedical Science and Engineering (BMSE), Institute Integrated Technology (IIT), Gwangju Institute of Science and Technology (GIST), Cheomdan-gwagiro, Gwangju, South Korea, Department of Neurology and Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Canada, McConnel Brain Imaging Center, Montreal Neurological Institute, McGill University, Montreal, Canada, Ludmer Centre for Neuroinformatics and Mental Health, McGill University, Montreal, Canada

Min Choi

Roles Methodology, Software, Writing – original draft

Affiliation: Department of Biomedical Science and Engineering (BMSE), Institute Integrated Technology (IIT), Gwangju Institute of Science and Technology (GIST), Cheomdan-gwagiro, Gwangju, South Korea

Boreom Lee

Roles Supervision, Writing – review & editing

* E-mail: [email protected]

Affiliation: Department of Biomedical Science and Engineering (BMSE), Institute Integrated Technology (IIT), Gwangju Institute of Science and Technology (GIST), Cheomdan-gwagiro, Gwangju, South Korea

ORCID logo https://orcid.org/0000-0002-7233-5833

Introduction

Schizophrenia is a mental illness whose fundamental nature is not fully understood. Although Emil Kraepelin [1] and Eugen Bleuler [2] developed and advocated conceptualizations of schizophrenia, they alone do not explain its various manifestations. Several studies since then [3, 4] have documented changes in cognitive function as core symptoms of schizophrenia, and with advances in neuroscience modalities, many researchers have attempted to reveal clinical symptoms of schizophrenia objectively. Their work has improved understanding of the illness, thereby enabling causes to be identified and treatments to be developed. Nonetheless, objective research of schizophrenia continues to be very important.

Electroencephalography (EEG) measures electrical activity of the brain and is one of the most widely used modalities in schizophrenia research. EEG can analyze neural synchrony associated with the pathophysiology of schizophrenia [5]. EEG-measured neural activity reflects self-generated oscillation and large-scale synchronization with responses to external information [6]. Moreover, EEG present excellent time resolution. Thus, it is suitable for studying complex cognitive functions. Therefore, EEG can provide unique information that is otherwise difficult to obtain using imaging modalities.

Many features derived from EEG were exploited for schizophrenia classification. The power spectra has long been the most widely used feature [7–10] and has been applied to other psychotic disorders as well [11]. Johannesen et al. used frequency domain features in the working memory task to differentiate between patients with schizophrenia and healthy controls [12]. Schizophrenia classification was also performed using statistical features including mean, skewness, and kurtosis [13, 14]. It has been reported that the statistical descriptor of the variability within the EEG signal, the entropy measure and the fractal dimension related with it, are useful for schizophrenia classification [15–18]. In addition, candidate features that may be applied to schizophrenia classification was exploited in other research area [19, 20]. Unfortunately, no “gold standard” has been established for analyzing EEG data to diagnose and treat schizophrenia. There is no clear definition of the pathological levels, and, in terms of clinical EEG interpretation, intra- and interrater reliabilities have proven insufficient [21] compared to other diagnostic tools. Although many researchers have proposed various features for schizophrenia classification, such features do not resolve inconsistencies or ambiguities in EEG interpretations. Furthermore, such features have failed to utilize the greatest advantage of EEG: its millisecond temporal resolution. Therefore, it is important to develop diagnostic features that can exploit it.

Several attempts have been made to use EEG microstate analysis to overcome this limitation. For example, by segmenting spontaneous EEG at the sub-second level, Lehmann et al. (1987) demonstrated the existence of a quasi-stable microstate, producing stable and evenly patterned results at 80–120 ms intervals [22]. We can also assume that EEG captures the manifestations and changes of such microstates.

Many previous studies have revealed microstate changes in various diseases and mental states as “atoms of thought” [23] or “building blocks of mentation” [24]. Characteristic alterations in EEG microstates have been reported in tasks involving specific cognitive functions and sensory inputs [25] and in tasks requiring abstract reasoning [26]. Yuan et al. (2012) reported the association between microstates and known large scale networks using functional magnetic resonance imaging (fMRI) [27]. Studies have supported this association structurally [28, 29] and functionally [30]. Microstate analysis was performed under several psychiatric conditions (e.g., sleep [31], anxiety disorder [32], mood disorder [33], and neurodegenerative disorder [34–36]. In particular, the characteristic microstate change has been investigated [8, 24, 37–44] and the change in microstates C and D has been well-known [45–47] in schizophrenia. These studies have been performed in various areas related to schizophrenia, such as clinical symptoms [38, 39, 43, 48], genetic vulnerability [40], and medication [42] as well as presence of disease [24, 36, 37, 44]. Recently, the potential as a state and trait biomarker has been reported that can suggest the progress [49] and genetic underpinnings [41] of illness. Although clustering algorithms and methodological choices for microstate analysis are diverse [50, 51], EEG analysis method using a cluster of k-means proposed by Pascual-Marqui et al. [52] is widely used in schizophrenia researches. It was exploited in early microstate study [37] and provided four archetype microstates based on study with a large sample of subjects [53]. This method enables the use of four archetype microstates with high interpretability, where functional significance has been revealed based on resting-state fMRI networks [27, 30]. The functions of the four type microstates (i.e., A, B, C, and D) are known to be associated with auditory, visual, default mode, and dorsal attention, respectively [25, 27]. The features based on those microstates show differences between patients with schizophrenia and other groups and allow interpretation from a neuroscience perspective. Therefore, four archetype microstates were used not only in schizophrenia [24, 37–39, 46] but also in general medical conditions, such as physical exercise [54], insomnia [55], hearing loss [56]. In summary, the microstate analysis method using the four archetypes is suitable for schizophrenia research.

Although there have been microstate feature-based studies on schizophrenia, they were implemented using univariate, instead of multivariate, analysis. Univariate analysis is useful for validating hypotheses according to each variable. However, it does not fully exploit microstate features for building a classification or treatment model. Univariate analysis is a special case of the multivariate model that is straightforward and neatly structured, whereas multivariate analysis applies to complete or general cases [57]. With multivariate analysis, we can simultaneously analyze multiple dependent and independent variables to improve reliability and validity. Therefore, multivariate analysis can utilize all microstate-feature information and identify new patterns to improve understanding [58, 59]. Applying multivariate analysis to the microstate feature, we can create a more powerful model to reveal differences that are not detected in univariate analysis. Models having improved reliability and validity possess more advantages of interpreting and generalizing results. Machine-learning techniques (e.g., classification using kernel method) accomplish multivariate analyses that catalog distinct observations and allocate new observations to previously defined groups [60]. Thus, by applying machine-learning-based algorithms to microstate features, we can distinguish between EEG recordings of patients diagnosed with schizophrenia and those of healthy (control) subjects and present a practical application. In summary, machine-learning-based microstate analysis is a novel approach to schizophrenia classification.

Considering the characteristics of schizophrenia and microstate analysis, we hypothesize that multivariate microstate features based on four microstate archetypes are useful for schizophrenia classification. We first obtain microstate and conventional EEG features from recordings of patients diagnosed with schizophrenia and those of healthy (control) subjects. Next, we obtain classification accuracy for differentiating between diagnosed patients and controls using three sets of features: microstate, conventional EEG, and combined. Finally, we compare performances of the three features sets.

Materials and methods

Dataset

A publicly accessible EEG dataset was used in our study [61]. Study protocol was approved by the Ethics Committee of the Institute of Psychiatry and Neurology in Warsaw. All participants received a written description of the protocol and provided written consent to take part in this study. The dataset was obtained from 14 patients diagnosed with schizophrenia and those from 14 healthy (control) subjects. The patients’ group comprised seven males (27.9±3.3 years) and seven females (28.3±4.1 years) who were diagnosed with paranoid schizophrenia according to the International Classification of Diseases (ICD)-10-CM criteria F20.0 and who showed prominent positive symptoms. Other inclusion criteria were washout periods of more than 1 week. Early-stage patients, such as those exhibiting their first episodes, were excluded. Other exclusion criteria were as follows: pregnancy, organic brain pathology, severe neurological diseases (e.g. epilepsy, Alzheimer’s, or Parkinson disease), and presence of a general medical condition. Patient and control groups were matched by age and gender. Nineteen channel EEG were recorded in accordance with the International 10/20 EEG system with a sampling frequency of 250 Hz for 15 min each during an eyes-closed resting state. Detailed information about the dataset can be found at the repository and from its related article [61].

EEG data preprocessing

To obtain artifact-free EEG features for microstate analyses, the EEG dataset was pre-processed using the FieldTrip toolbox and EEGLAB [62, 63]. We followed the well-established preprocessing pipeline that have been widely used in some previous studies [24, 25, 37, 53] to use the archetype microstates, which are proposed from the studies of schizophrenia and known for their functional significances. Fig 1 shows the entire process of extracting EEG features from raw data. EEG were re-referenced to the common average electrode and were filtered using a 2–20-Hz band-pass filter. Continuous EEG data were segmented into 5-s non-overlapping epochs evaluated based on variance, so that the epoch containing artifacts could be removed. From the continuous EEG dataset, we extracted 1,362 artifact-free epochs of patients diagnosed with schizophrenia and 1,225 artifact-free epochs of healthy (control) subjects. The benefit of this approach was that the all artifact-free data points could be analyzed instead of selecting and analyzing parts. All epochs were implemented for classification to distinguish whether each epoch was of the patients diagnosed with schizophrenia or healthy (control) subjects. To prevent the statistical power from being exaggerated in comparing the features between the two groups, the comparison was performed using only the first 20 artifact-free epochs as in previous studies [24, 36, 37, 53, 64]. Cohen’s D was reported with p-value to evaluate the effect size. The comparison performed with all epochs is presented in the S1 Table.

[Figure omitted. See PDF.]

Fig 1. Overview of the process for extracting microstate and conventional EEG features.

https://doi.org/10.1371/journal.pone.0251842.g001

Microstate analysis

Global Field Power (GFP).

GFP can be used to represent the global pattern of neuroelectrical activity and is defined as follows:(1)where xi(n) and represent the instantaneous and mean potentials across N electrodes at time n.

Because GFP instantaneously measured electric-field activity, it was typically used to characterize the dynamic fluctuations of brain activity. Local GFP maxima described instances of the strongest brain potentials and the highest topographic signal-to-noise ratios. In microstate analysis, topographies of the electric field at local GFP maxima were discrete EEG states, and signal evolution was a series of such states [47]. Then, successive microstates, which were discrete states of the EEG analyzed based on local maxima of the GFP, were derived. Using clustering methods, all microstates can be assigned according to archetype microstates. The archetypes have enabled many studies which uncovered their function and applied them to various diseases [45]. In this study, modified k-means clustering were used for it. The clustering method were described in detail in the following sections and the parameters for microstate analysis can be found at http://www.thomaskoenig.ch.

Most microstate analysis studies have reported the same four archetypal microstate topographies to represent brain activity measured using a resting-state EEG. These four topographies included right-frontal left-posterior, left-frontal right-posterior, midline frontal-occipital, and midline frontal and are typed as A, B, C, and D, respectively [37, 53]. Importantly, single topography remained quasi-stable for intervals of about 80–120 ms before dynamically transitioning to another topography (i.e., microstates) [47]. Therefore, when an EEG was considered to be a series of topographies of electric potentials that evolve, the entire recording can be studied using a set of topography that dynamically fluctuate amongst themselves at discrete time points.

EEG microstate segmentation methods.

Microstate analysis was used to segment entire EEG recordings into successive topographies, including spatiotemporal information of global brain network, that remained quasi-stable for a short period of time (80–120 ms). An early microstate analysis method proposed by Lehmann et al. segmented EEG signals using adaptive segmentation algorithms, wherein the topography at each successive GFP peak was compared to those at preceding ones, and a new microstate was considered when centroid locations of segmented potentials changed by more than a predefined interval [22]. Thus, the overall length and general topographic characteristics of microstates could be computed. However, early attempts employing adaptive segmentation rarely grouped microstates into archetypes.

More recently, Pascual-Marqui et al. [52] proposed a statistical approach that directly considered the topographies of entire EEG recording. This method utilized k-means clustering analysis, which iteratively combined nested high spatial-correlation topographies and identified a representative topography that best explained the variance in each cluster. Various methods of clustering, such as agglomerative hierarchical clustering [65], principal component analysis [66, 67], independent component analysis [68], a mixture of Gaussian algorithms [69], and Markov process-based decomposition [70, 71], recently developed for factor analysis, can be used to segment the most dominant spatial components in series of topography.

In this work, we used the modified k-means clustering algorithm to segment microstates obtained from EEG recordings [45] to determine the optimal sorting order of clusters using the cross-validation criterion. Previous well-established microstate studies using multi-modality data including EEG and resting-state fMRI [27, 30, 37, 53] have revealed that four topographies across all subjects were optimal to fully represent whole brain activity and allow interpretation from a neuroscience perspective. The four topographies included right-frontal left-posterior, left-frontal right-posterior, midline frontal-occipital, and midline frontal and are typed as A, B, C, and D, respectively. Therefore, in this work, we adopted these four most common microstate topographies A, B, C, and D as the number of optimal archetype templates. In addition, the sorting of microstates (A, B, C, D sorting) was performed using cross-validation strategies on the training EEG trials. The sorting order by k-means clustering, which led to the lowest free energy on training EEG inputs, is selected.

Once we determined the optimal number of microstates (four archetype microstates), next step is to sort and label them into a sequence by using modified k-means clustering algorithm and Global Explain Variance (GEV) criteria [65]. The setting parameters for k-means algorithms are re-run and iterations as explained following. In principle, by re-running the stochastic k-means algorithm multiple times (in this analysis, we set re-run parameter to 20 times), we are able to test multiple segmentations on the same dataset and select the best re-run based on the GEV criteria [65]. GEV is a measure of how similar each EEG sample is to the microstate prototype it has been assigned to. The higher the GEV the better [65]. More importantly, we are able to reach the global minimum among 20 local minimums (20 re-runs). After 20 re-runs, the one that maximizes the GEV is selected. However, the number of re-run is a trade-off between computation time and how likely we are to converge on the same optimal solution. In the Microstate EEGlab toolbox [50, 51] and its Python package in which we have applied for our analysis select 10 re-run as a default value. In addition, Thomas Koenig’s manual (http://www.thomaskoenig.ch) has recommended that a range from 20 to 50 re-runs could be sufficient for a proper analysis. Furthermore, we have found that there are several existing EEG microstate analysis literatures that set 10 re-runs [52] as well as papers that use 30 [26] as a proper re-run number.

Another parameter for k-means clustering is iteration, which means that in each re-run the k-means algorithm keeps iterating until some stopping criteria (convergence threshold) are satisfied. In this analysis, we used the convergence threshold, which stops the algorithms when the relative error change between subsequence iterations is below the threshold. Here, we set the threshold at 10−6. The maximum number of iterations set to 1000 which means the algorithm can stop if the maximum iteration is reached before convergence for computation time efficiency.

Conventional EEG feature extraction

We compared classification performances obtained using microstate and conventional EEG features. First, electrodes were clustered into five regions of interest (ROI): left anterior (Fp1, F7, F3), right anterior (Fp2, F4, F8), left posterior (T7, C3, P7, P3, O1), right posterior (C4, T8, P4, P8, O2), and central (Fz, Cz, Pz). Then, conventional EEG features were extracted from each ROI. Based on numerous features extracted from previous studies [15, 20], 31 features suitable for resting-state EEG recordings were selected. A summary of all the defined features is presented in Table 1. The frequency band of the spectral analysis was defined as follows: δ (2–4 Hz), θ (4–8 Hz), α (8–13 Hz), and low-β (13–20 Hz). Finally, the analysis was performed using 155 features.

[Figure omitted. See PDF.]

Table 1. Summary of conventional EEG features obtained for patients diagnosed with schizophrenia and healthy (control) subjects.

https://doi.org/10.1371/journal.pone.0251842.t001

Classification

To assess whether microstate features were appropriate for schizophrenia classification, we used several machine-learning algorithms to compare classification performances of microstate and conventional EEG features. We used tenfold cross-validation to assess classification accuracy. Considering high inter-subject variability of EEG, the fold was generated by subject label so that all epochs of one subject was included in the training or test set. Four parameters were used to evaluate classifier performance: accuracy, area under the curve (AUC), sensitivity, and specificity. Permutation tests were implemented to evaluate the competency of classifiers. In this study, we performed 1,000 repetitions to obtain a p-value to verify whether or not the observed accuracy was obtained by chance or whether it represented results. The above procedures were performed using MATLAB® software (MathWorks, Inc., Natick, MA, USA).

The classifier was selected based on stability and simplicity among various known classifiers from a previous EEG study [72] and other biomedical engineering research [73, 74]: the support vector machine (SVM) [75], Linear Discriminant Analysis [76], Naïve Bayes (NB) [77], random forest (RF) [78], and the k-nearest neighbor (KNN) [79]. To compare classification performances of microstate and conventional EEG features, we applied all classifiers to combined features to obtain accuracy.

Feature selection.

In the neuroimaging machine-learning community, it was widely known that feature selection was an important step required prior to training classifiers [80]. Feature selection was essential, because it was used to select the most informative features and to discard the noise and artifacts, which helped enhance the classification performance and reduce complex computations and overfitting problems. In this work, we employed univariate t-tests and multivariate recursive feature elimination (RFE) as feature selection techniques. Univariate t-tests were performed on individual features to identify significant differences between groups, whereas multivariate RFE was used to investigate mutual relationships between multiple features.

Univariate feature selection.

Univariate statistical t-tests have been used in many neuroimaging studies to show abnormalities in average signals for one or more brain features in an illness group compared to normal average signals for those in a healthy group. Recent discrimination studies have used such t-tests to select the most informative features for machine learning in neuroimaging [80, 81]. The key results of statistical test-based analyses were usually expressed as p-values. Subsequently, the optimal p-value cut-off for selecting relevant features was determined via cross-validation, and the features selected were used in the subsequent machine-learning analysis. In this work, we applied t-test-based feature selection techniques to machine-learning-based schizophrenia classification. Using t-tests on training data, we generated a result that required retaining only those features presenting significant changes in any of the feature measures (i.e., microstate measures) between the two groups at threshold p-values (p<0.05, p<0.01, p<0.005, and p<0.001). Bonferroni correction was applied for multiple comparisons.

Recursive feature elimination.

Although the univariate t-test did not consider interactions between multiple patterns, RFE was a multivariate wrapper-based feature selection algorithm that ranked features based on their effects on classification [82]. The RFE ranking procedure was closely related to that of the SVM model. In each RFE iteration, an SVM model was trained. The lowest-ranking features were then removed, because they have the least effect on classification accuracy, whereas the remaining features were used for the SVM model in the next iteration. The sequential process was repeated until all features have been discarded. The features were then ranked based on the elimination sequence. A detailed explanation of the application of RFE algorithms to neuroimaging can be found in previous studies [80, 82]. In this work, we implemented a univariate t-test/multivariate RFE hybrid feature selection technique.

Statistical analysis

The microstate topographies between groups were compared using topographic ANOVA (TANOVA) in the Ragu software [83]. TANOVA was a non-parametric randomization test that computed statistical differences using global field power of difference topographies [84]. Similarity was assessed for each microstate type as in the method of Koenig et al. [37] and Grieder et al. [64]. Microstate features were analyzed using independent t-tests. To prevent type-Ⅰ errors caused by the multiple-comparison problem, we evaluated significance using Bonferroni correction. Paired t-tests were performed to assess the significance of classification results obtained from the three feature sets. Four p-values (p<0.05, p<0.01, p<0.005, and p<0.001) were used to select univariate features. Statistical analyses were performed using MATLAB® software.

Results

Microstate analysis

Normalized microstate scalp topographies of a patient diagnosed with schizophrenia and a healthy (control) subject are shown in Fig 2. Types A and B were dominant in the unilateral frontal area, type C was symmetrical, and type D was somewhat dominant in the occipital area. These four microstates were similar to those shown in previous studies [37, 53]. Only type D topography showed significant difference between the two groups after Bonferroni correction. This was also consistent with the previous result [37]. Thus, the same archetype was applied.

[Figure omitted. See PDF.]

Fig 2. Four microstates of resting-state EEG recordings obtained for patients diagnosed with schizophrenia and healthy (control) subjects.

Type D topography was significantly different between the two groups even after Bonferroni correction (p = 0.0012). Asterisk (*) show a significant difference. SC: patients diagnosed with schizophrenia, HC: healthy control.

https://doi.org/10.1371/journal.pone.0251842.g002

Results obtained from microstate analysis conducted using 19 microstate features are presented in Table 2. There was no difference in total time between the two groups after Bonferroni correction. For types B and C in patients diagnosed with schizophrenia, coverage increased with increasing duration or occurrence. For type D in patients diagnosed with schizophrenia, coverage decreased with decreasing duration and occurrence. On the other hand, all type A features did not differ between patients diagnosed with schizophrenia and healthy (control) subjects. For across type features, there was no difference between the two groups in both mean duration and mean occurrence after Bonferroni correction. Although there was a difference in mean global field power (GFP) for types B, there was no difference for types A, C, and D. Among the significant features after Bonferroni correction, none of the features showed a small effect with Cohen’s d less than 0.2. On the contrary, all features that were not significant showed small effects with Cohen’s d less than 0.3. To summarize, eight microstate features showed significant differences between patients diagnosed with schizophrenia and healthy (control) subjects.

[Figure omitted. See PDF.]

Table 2. 19 microstate features obtained for patients diagnosed with schizophrenia and healthy (control) subjects.

https://doi.org/10.1371/journal.pone.0251842.t002

EEG analysis using conventional EEG features

Using the conventional EEG features presented in Table 1, we investigated differences between patients diagnosed with schizophrenia and healthy (control) subjects. A total of 155 features were tested using post hoc analysis in five ROIs at significance levels 0.05, 0.01, 0.005, and 0.001. Differences were found in 42, 31, 22, and 14 features, respectively. Among the statistical features, mean, kurtosis, skewness, and coefficient of variation did not differ in all ROIs. 13 of 55 statistical features showed significant differences. Each frequency-domain feature showed significant differences in at least one ROI, and 19 of 65 features were different between groups. 10 of 35 time-domain features showed differences, and 3 of the 7 features did not differ significantly in all ROIs. Approximate and sample entropy features showed a very low correlation with schizophrenia. Overall, although some insignificant features were included, the conventional EEG feature set consisted of several features useful for schizophrenia classification.

Classification performances obtained using microstate and conventional EEG features

To compare performances obtained using microstate and conventional EEG features in classifying patients diagnosed with schizophrenia and healthy (control) subjects, we applied several classifiers to each feature set. The classification accuracy obtained by tenfold cross-validation using several classifiers is presented in the S2 Table, and the result of classifier showing the highest classification accrual is presented in Table 3. The highest performance was 75.64%, obtained by applying the SVM quadratic kernel to microstate features. The conventional EEG feature also showed the highest average accuracy of 67.62% with the SVM. The highest accuracy with SVM was 72.93% when using microstate and conventional EEG features, which was significantly higher than when using only conventional EEG ones. The performances of the other classifier were found in the S2 Table. In summary, in the schizophrenia classification, the addition of microstate features to conventional features increased the accuracy of classification.

[Figure omitted. See PDF.]

Table 3. Classification accuracies with quadratic SVM classifier using 19 microstate and 155 conventional EEG features obtained from the EEG dataset for patients diagnosed with schizophrenia and healthy (control) subjects.

https://doi.org/10.1371/journal.pone.0251842.t003

For accuracy calculated by tenfold cross-validation, microstate features showed higher accuracy than conventional EEG with NB, RF, KNN, and SVM classifers. The model of the highest accuracy is shown in Table 4 as the quadratic-kernel SVM using the microstate feature. The accuracy of each fold was evaluated using a permutation test with 1,000 repetitions. It was statistically significant in all folds.

[Figure omitted. See PDF.]

Table 4. The classification results of tenfold cross-validation with quadratic SVM classifier using 19 microstate features.

https://doi.org/10.1371/journal.pone.0251842.t004

Classification performances with feature selection methods

Table 5 shows whether the superiority of microstate features was maintained after the feature selection method was applied. The results of the other classifier were listed in the S3 Table. For the microstate feature, only four features were removed: total time, duration, and coverage of type A and the coverage of type B. For conventional EEG features, including numerous insignificant ones, the highest accuracy was achieved when only RFE was applied. After feature selection was applied, the highest accuracies of the microstate and conventional EEG feature sets were 76.62 and 68.89%, respectively. Conventional feature set showed the highest classification accuracy using 136 features by removing 19 features with RFE. The highest classification accuracy of combined feature set was 76.85%, which was higher than that of conventional EEG ones alone. It was obtained using 39 features by removing 133 features with univariate feature selection and 2 features with RFE. Even in a subset of features showing the highest classification accuracy, microstate features outperformed conventional EEG ones. However, the addition of microstate features helped improve classification accuracy.

[Figure omitted. See PDF.]

Table 5. Classification accuracies with quadratic SVM classifier achieved for different methods of selecting features from an EEG dataset for patients diagnosed with schizophrenia and healthy (control) subjects.

https://doi.org/10.1371/journal.pone.0251842.t005

Discussion

This study investigates the usefulness of EEG microstate features for schizophrenia classification. To validate microstate features, we first used a modified k-means method to extract them from EEG recordings of patients diagnosed with schizophrenia and those from healthy (control) subjects. Considering research reproducibility, we utilized all but some of the epochs, which consist of the remaining epochs after the epochs containing artefact were removed in the preprocessing step. Next, we applied feature selection methods to conventional EEG features introduced in a previous study [7–20] to compare classification performances. The method would prevent classification performance of conventional EEG features from being underestimated in multivariate analysis. Finally, after classification performances were obtained using microstate features, conventional EEG features, and combined features with tenfold cross-validation, average accuracies were compared. It showed higher classification accuracy than when using only convention features, both when only using the microstate feature and when combining microstate and convention features. Therefore, the results of this study suggest that microstate features are useful for distinguishing between schizophrenic and healthy (control) subjects. Combining microstate features with conventional EEG features is, thus, a proper application that can achieve higher classification accuracy than using only conventional ones.

The results of the microstate analysis, represented in Fig 2. and Table 2, showed characteristic changes in features for patients diagnosed with schizophrenia. The topography of type D showed a significant difference between the two groups, and the others did not. While this is consistent with previous schizophrenia studies [37], it differs from the result of semantic dementia and Alzheimer’s disease studies, which differed in types B and C [64]. Changes of duration and occurrence led to changes in coverage, mean duration, and mean occurrence. For patients diagnosed with schizophrenia, duration increased for type C and decreased for type D. According to Milz et al., each microstate can be associated with a specific function [30], and several features of the four types may change in each illness group [45]. The changes in EEG recordings of types C and D microstate features are in line with those reported in previous studies [24, 36, 40, 44]. Specifically, the type-D microstate feature showed reduced duration when a subject experienced hallucinations [39]. In agreement with results of previous studies, duration was reduced in the group of patients diagnosed with paranoid schizophrenia who showed prominent positive symptoms in this study. Type B should be interpreted with caution, because, although coverage [36] and occurrence [44] were consistent with those shown in previous studies, duration decrerased in the previous study [24, 36] increased in our study. In contrast to earlier findings that showed no change in occurrence or reduced occurrence [24, 36, 44], occurrence and coverage of type A increased. However, the features of types A and B had smaller effect sizes considering Cohen’s d. Thus, it is difficult to accept that the features actually differed between schizophrenia and healthy (control) subjects. Type A and B features, known to be associated with a sensory task, do not present remarkable differences. Considering the function of each type [27, 30], it is natural that types C and D, attributed to default mode and dorsal attention, respectively [25, 27], are more reliable diagnostic features than type A and B features in schizophrenia classification. Results consistent with previous studies have established that microstate features are suitable for schizophrenia classification.

Several studies reported abnormal functional brain network in schizophrenia and other psychiatric disorders. For example, depressive disorders, including major depressive disorders, have been examined in comparison with schizophrenia. They are classified as different diseases [85], however, share risk factors and may show similar clinical manifestations in the early or severe stage of the disorder [86]. The dysregulation of left inferior parietal cortex [87] and decreased convergent and divergent network connectivity [88] were consistent in both disorders. Lydon-Staley and Bassett showed the similarity of reduced small-world properties and differences in regional activation on two disorders [89]. In some regions or networks, the difference in activation was demonstrated [90–92], and the opposite activation was identified [93–95]. Taken together, previous studies of functional networks revealed similar and different alterations depending on the nature or areas of the feature applied. Considering that microstate features are associated with the functional network, these may also show common and distinct changes in various psychiatric disorders. The development of various features that can fully exploit the advantages of temporal resolution enables the microstate feature to be a diagnostic biomarker reflecting functional brain network.

Machine-learning based multivariate analysis provides an opportunity to understand a system by analyzing many features simultaneously. Therefore, we can reduce type Ⅰ errors and obtain optimized models with multivariate analysis. In other words, features extracted from EEG recordings containing some information can be utilized with multivariate analysis to develop a model having higher reliability and validity. This is an important process in the classification of schizophrenia for which EEG microstate analysis is a valuable method. Several studies have already reported that EEG features are useful in emotion recognition [20], neurological disease [96], and schizophrenia classification [16, 97]. To validate performances of microstate features in schizophrenia classification, we examined performances of conventional EEG features for comparison. Our results suggest that microstate features reflect schizophrenia characteristics and show better classification performance than conventional EEG features. The microstate feature was not only used alone, it also showed better performance when combined with conventional EEG features. We can improve the classification accuracy by combining microstates with conventional features. In other words, we can infer that microstate features contain information that is difficult to obtain by traditional EEG analysis methods. We expect microstate analysis to be synergistic with conventional EEG analysis methods. Taken together, microstate features show potential for schizophrenia classification.

Prior studies have applied multivariate analysis to various EEG features to classify patients diagnosed with schizophrenia from healthy (control) subjects [16, 98–101]. Although classification accuracies obtained in those studies were quite high, they may not have been sufficient to generalize results. For example, some studies were conducted without validation [98, 99], and only some channels were selected in other studies [16, 101]. Another study only used EEG event-related potentials generated during a specific task [100]. In the present work, because accuracy was calculated by applying tenfold cross-validation to resting-state-EEG data, classification performance is reliable. Furthermore, although machine learning has been applied to EEG microstate features for patients diagnosed with neurodegenerative diseases [102] and neurological disorders [103], it has not been applied yet to schizophrenia microstate studies. Although machine learning of microstate features is an alternative to existing methods of classification, it will likely develop into a more robust method when combined with other neuroimaging modalities, such as fMRI [45]. Therefore, our research investigating the classification accuracy of microstate features is essential to verify the feasibility of microstate analysis in schizophrenia classification. Regarding the performance of the microstate features used in this study, our method resulted in accurate classification and can be applied to various areas of neuroscience and clinical fields.

The strength of this study is that it has more generalizability than previous studies. First, we minimized bias caused by data selection. In previous studies, EEG recordings shorter than 10 s were selected from those longer than a few minutes [37, 53] for microstate analysis, which may have affected results. We divided all data into epochs and used all EEG recordings from all subjects to obtain overall classification performance. Therefore, classification performance in this study showed generality, because it considered EEG data for all subjects. Second, we applied cross-validation. In particular, considering inter-subject variability of EEG data [104], we chose tenfold cross-validation to prevent EEG epochs from having the same subjects belonging to both training and testing datasets. Because of these two reasons, despite the application of RFE to many of the features referenced in the literature, classification performance of conventional EEG features was relatively low. On the contrary, microstate features showed better performance, although the methods were not applied in all previous studies. Our reproducible results are likely to be replicated in future studies, suggesting that microstate features are promising diagnostic features for schizophrenia classification.

Some study limitations must also be discussed. This study used publicly available data instead of data obtained from cohort studies, and the number of subjects was small. Although age and gender were matched, it was difficult to obtain enough data to represent the general population. Thus, the results of this study should be interpreted carefully. Therefore, classification performances of microstate features should be analyzed using data obtained from a larger cohort to improve classification accuracy and to develop an accurate model representing the general population. Our study was performed using modified k-means clustering only with a k value of 4. In general, higher classification accuracy can be achieved by using other clustering methods or a higher k value [25, 105]. Although our study was performed only with a k value of 4, it is reliable to show a sufficient explained variance of patients diagnosed with schizophrenia (76.4±4.4%) and those of healthy (control) subjects (75.8±5.8%). However, using a different k value in future research may contribute to improvement of classification accuracy. Furthermore, the 19 microstate features used in this study may not fully reflect information obtained from microstate analysis. Although the number of features was not small, the included information sometimes overlapped. For example, the duration can be calculated from the total time, coverage, and occurrence. Thus, only the 19 microstate features used in this study did not fully exploit the strength of microstate analysis. Nevertheless, because microstate features showed promising classification performance, it is very probable that classification performance would be improved by developing new microstate features.

Conclusions

In this study, we provided evidence for usefulness of EEG microstate features in schizophrenia classification. When comparing classification accuracies obtained using microstate features, conventional EEG features, and combined features, microstate features and combined features outperformed conventional ones, suggesting that EEG microstate features are appropriate for schizophrenia classification.

Supporting information

S1 Table. 19 microstate features obtained for patients diagnosed with schizophrenia and healthy (control) subjects.

https://doi.org/10.1371/journal.pone.0251842.s001

(DOCX)

S2 Table. Classification accuracies obtained using microstate and conventional EEG features obtained from the EEG dataset for patients diagnosed with schizophrenia and healthy (control) subjects.

https://doi.org/10.1371/journal.pone.0251842.s002

(DOCX)

S3 Table. Classification accuracies achieved for different methods of selecting features from an EEG dataset for patients diagnosed with schizophrenia and healthy (control) subjects.

https://doi.org/10.1371/journal.pone.0251842.s003

(DOCX)

Citation: Kim K, Duc NT, Choi M, Lee B (2021) EEG microstate features for schizophrenia classification. PLoS ONE 16(5): e0251842. https://doi.org/10.1371/journal.pone.0251842

References

1. Kraepelin E, Barclay RM, Robertson GM. Dementia præcox and paraphrenia. Edinburgh,: E. & S. Livingstone; 1919. 1 p. l., x. 331 p. p.

2. Bleuler E. Dementia praecox. New York,: International Universities Press; 1950. 548 p. p.

3. Barch DM, Ceaser A. Cognition in schizophrenia: core psychological and neural mechanisms. Trends Cogn Sci. 2012;16(1):27–34. Epub 2011/12/16. PubMed Central PMCID: PMC3860986. pmid:22169777

4. Taylor MA, Abrams R. Cognitive impairment in schizophrenia. Am J Psychiatry. 1984;141(2):196–201. Epub 1984/02/01. pmid:6691479.

5. Phillips KG, Uhlhaas PJ. Neural oscillations as a translational tool in schizophrenia research: rationale, paradigms and challenges. J Psychopharmacol. 2015;29(2):155–68. Epub 2015/01/09. pmid:25567552.

6. Uhlhaas PJ, Singer W. Abnormal neural oscillations and synchrony in schizophrenia. Nat Rev Neurosci. 2010;11(2):100–13. Epub 2010/01/21. pmid:20087360.

7. Karson CN, Coppola R, Morihisa JM, Weinberger DR. Computed electroencephalographic activity mapping in schizophrenia. The resting state reconsidered. Arch Gen Psychiatry. 1987;44(6):514–7. Epub 1987/06/01. pmid:3495249.

8. Sueyoshi K, Sumiyoshi T. Electrophysiological Evidence in Schizophrenia in Relation to Treatment Response. Front Psychiatry. 2018;9:259. Epub 2018/06/29. pmid:29951008; PubMed Central PMCID: PMC6008315.

9. Galderisi S, Mucci A, Volpe U, Boutros N. Evidence-based medicine and electrophysiology in schizophrenia. Clin EEG Neurosci. 2009;40(2):62–77. Epub 2009/06/19. pmid:19534300.

10. Kam JW, Bolbecker AR, O’Donnell BF, Hetrick WP, Brenner CA. Resting state EEG power and coherence abnormalities in bipolar disorder and schizophrenia. J Psychiatr Res. 2013;47(12):1893–901. Epub 2013/10/05. pmid:24090715; PubMed Central PMCID: PMC4015517.

11. Newson JJ, Thiagarajan TC. EEG Frequency Bands in Psychiatric Disorders: A Review of Resting State Studies. Front Hum Neurosci. 2018;12:521. Epub 2019/01/29. pmid:30687041; PubMed Central PMCID: PMC6333694.

12. Johannesen JK, Bi J, Jiang R, Kenney JG, Chen CA. Machine learning identification of EEG features predicting working memory performance in schizophrenia and healthy adults. Neuropsychiatr Electrophysiol. 2016;2:3. Epub 2016/07/05. pmid:27375854; PubMed Central PMCID: PMC4928381.

13. Alimardani F, Cho J, Boostani R, Hwang H. Classification of Bipolar Disorder and Schizophrenia Using Steady-State Visual Evoked Potential Based Features. IEEE Access. 2018;6:40379–88.

14. Tikka SK, Singh BK, Nizamie SH, Garg S, Mandal S, Thakur K, et al. Artificial intelligence-based classification of schizophrenia: A high density electroencephalographic and support vector machine study. Indian J Psychiatry. 2020;62(3):273–82. Epub 2020/08/11. pmid:32773870; PubMed Central PMCID: PMC7368447.

15. Sabeti M, Katebi S, Boostani R. Entropy and complexity measures for EEG signal classification of schizophrenic and control participants. Artif Intell Med. 2009;47(3):263–74. Epub 2009/05/01. pmid:19403281.

16. Sabeti M, Katebi SD, Boostani R, Price GW. A new approach for EEG signal classification of schizophrenic and control participants. Expert Systems with Applications. 2011;38(3):2063–71. WOS:000284863200086.

17. Portnova GV, Atanov MS. Nonlinear EEG parameters of emotional perception in patients with moderate traumatic brain injury, coma, stroke and schizophrenia. AIMS Neurosci. 2018;5(4):221–35. Epub 2018/11/07. pmid:32341963; PubMed Central PMCID: PMC7179336.

18. Racz FS, Stylianou O, Mukli P, Eke A. Multifractal and Entropy-Based Analysis of Delta Band Neural Activity Reveals Altered Functional Connectivity Dynamics in Schizophrenia. Front Syst Neurosci. 2020;14:49. Epub 2020/08/15. pmid:32792917; PubMed Central PMCID: PMC7394222.

19. Li X, Song D, Zhang P, Zhang Y, Hou Y, Hu B. Exploring EEG Features in Cross-Subject Emotion Recognition. Front Neurosci. 2018;12:162. Epub 2018/04/05. pmid:29615853; PubMed Central PMCID: PMC5867345.

20. Jenke R, Peer A, Buss M. Feature Extraction and Selection for Emotion Recognition from EEG. Ieee T Affect Comput. 2014;5(3):327–39. WOS:000344589600012.

21. Grant AC, Abdel-Baki SG, Weedon J, Arnedo V, Chari G, Koziorynska E, et al. EEG interpretation reliability and interpreter confidence: A large single-center study. Epilepsy & Behavior. 2014;32:102–7. WOS:000332398600020. pmid:24531133

22. Lehmann D, Ozaki H, Pal I. EEG alpha map series: brain micro-states by space-oriented adaptive segmentation. Electroencephalography and Clinical Neurophysiology. 1987;67(3):271–88. pmid:2441961

23. Lehmann D, Strik WK, Henggeler B, Koenig T, Koukkou M. Brain electric microstates and momentary conscious mind states as building blocks of spontaneous thinking: I. Visual imagery and abstract thoughts. International Journal of Psychophysiology. 1998;29(1):1–11. WOS:000073934400001. pmid:9641243

24. Lehmann D, Faber PL, Galderisi S, Herrmann WM, Kinoshita T, Koukkou M, et al. EEG microstate duration and syntax in acute, medication-naive, first-episode schizophrenia: a multi-center study. Psychiatry Res. 2005;138(2):141–56. Epub 2005/03/16. pmid:15766637.

25. Seitzman BA, Abell M, Bartley SC, Erickson MA, Bolbecker AR, Hetrick WP. Cognitive manipulation of brain electric microstates. Neuroimage. 2017;146:533–43. Epub 2016/10/25. pmid:27742598; PubMed Central PMCID: PMC5321823.

26. Zappasodi F, Perrucci MG, Saggino A, Croce P, Mercuri P, Romanelli R, et al. EEG microstates distinguish between cognitive components of fluid reasoning. NeuroImage. 2019;189:560–73. pmid:30710677

27. Britz J, Van De Ville D, Michel CM. BOLD correlates of EEG topography reveal rapid resting-state network dynamics. Neuroimage. 2010;52(4):1162–70. WOS:000280695200002. pmid:20188188

28. Milz P, Pascual-Marqui RD, Achermann P, Kochi K, Faber PL. The EEG microstate topography is predominantly determined by intracortical sources in the alpha band. Neuroimage. 2017;162:353–61. Epub 2017/08/30. pmid:28847493.

29. Yuan H, Zotev V, Phillips R, Drevets WC, Bodurka J. Spatiotemporal dynamics of the brain at rest—exploring EEG microstates as electrophysiological signatures of BOLD resting state networks. Neuroimage. 2012;60(4):2062–72. Epub 2012/03/03. pmid:22381593.

30. Milz P, Faber PL, Lehmann D, Koenig T, Kochi K, Pascual-Marqui RD. The functional significance of EEG microstates-Associations with modalities of thinking. Neuroimage. 2016;125:643–56. WOS:000366647500058. pmid:26285079

31. Brodbeck V, Kuhn A, von Wegner F, Morzelewski A, Tagliazucchi E, Borisov S, et al. EEG microstates of wakefulness and NREM sleep. Neuroimage. 2012;62(3):2129–39. Epub 2012/06/05. pmid:22658975.

32. Kikuchi M, Koenig T, Munesue T, Hanaoka A, Strik W, Dierks T, et al. EEG microstate analysis in drug-naive patients with panic disorder. PloS one. 2011;6(7):e22912. Epub 2011/08/11. pmid:21829554; PubMed Central PMCID: PMC3146502.

33. Strik WK, Dierks T, Becker T, Lehmann D. Larger topographical variance and decreased duration of brain electric microstates in depression. J Neural Transm Gen Sect. 1995;99(1–3):213–22. Epub 1995/01/01. pmid:8579806.

34. Hatz F, Hardmeier M, Benz N, Ehrensperger M, Gschwandtner U, Ruegg S, et al. Microstate connectivity alterations in patients with early Alzheimer’s disease. Alzheimers Res Ther. 2015;7:78. Epub 2016/01/01. pmid:26718102; PubMed Central PMCID: PMC4697314.

35. Musaeus CS, Nielsen MS, Hogh P. Microstates as Disease and Progression Markers in Patients With Mild Cognitive Impairment. Front Neurosci. 2019;13:563. Epub 2019/07/03. pmid:31263397; PubMed Central PMCID: PMC6584800.

36. Nishida K, Morishima Y, Yoshimura M, Isotani T, Irisawa S, Jann K, et al. EEG microstates associated with salience and frontoparietal networks in frontotemporal dementia, schizophrenia and Alzheimer’s disease. Clin Neurophysiol. 2013;124(6):1106–14. Epub 2013/02/14. pmid:23403263.

37. Koenig T, Lehmann D, Merlo MC, Kochi K, Hell D, Koukkou M. A deviant EEG brain microstate in acute, neuroleptic-naive schizophrenics at rest. pmid:10449596 1999;249(4):205–11. Epub 1999/08/17. 10449596.

38. Strelets V, Faber PL, Golikova J, Novototsky-Vlasov V, Koenig T, Gianotti LR, et al. Chronic schizophrenics with positive symptomatology have shortened EEG microstate durations. Clin Neurophysiol. 2003;114(11):2043–51. Epub 2003/10/29. pmid:14580602.

39. Kindler J, Hubl D, Strik WK, Dierks T, Koenig T. Resting-state EEG in schizophrenia: auditory verbal hallucinations are related to shortening of specific microstates. Clin Neurophysiol. 2011;122(6):1179–82. Epub 2010/12/03. pmid:21123110.

40. Tomescu MI, Rihs TA, Roinishvili M, Karahanoglu FI, Schneider M, Menghetti S, et al. Schizophrenia patients and 22q11.2 deletion syndrome adolescents at risk express the same deviant patterns of resting state EEG microstates: A candidate endophenotype of schizophrenia. Schizophr Res Cogn. 2015;2(3):159–65. Epub 2015/05/27. pmid:29379765; PubMed Central PMCID: PMC5779300.

41. da Cruz JR, Favrod O, Roinishvili M, Chkonia E, Brand A, Mohr C, et al. EEG microstates are a candidate endophenotype for schizophrenia. Nat Commun. 2020;11(1):3089. Epub 2020/06/20. pmid:32555168; PubMed Central PMCID: PMC7303216.

42. Mackintosh AJ, Borgwardt S, Studerus E, Riecher-Rössler A, de Bock R, Andreou C. EEG Microstate Differences in Medicated vs. Medication-Naïve First-Episode Psychosis Patients. Frontiers in Psychiatry. 2020;11:1320. pmid:33329154

43. Andreou C, Faber PL, Leicht G, Schoettle D, Polomac N, Hanganu-Opatz IL, et al. Resting-state connectivity in the prodromal phase of schizophrenia: insights from EEG microstates. Schizophr Res. 2014;152(2–3):513–20. Epub 2014/01/07. pmid:24389056.

44. Kikuch M, Koenig T, Wada Y, Higashima M, Koshino Y, Strik W, et al. Native EEG and treatment effects in neuroleptic-naive schizophrenic patients: Time and frequency domain approaches. Schizophrenia Research. 2007;97(1–3):163–72. WOS:000251742000019. pmid:17703921

45. Michel CM, Koenig T. EEG microstates as a tool for studying the temporal dynamics of whole-brain neuronal networks: A review. Neuroimage. 2018;180:577–93. WOS:000443271100021. pmid:29196270

46. Rieger K, Diaz Hernandez L, Baenninger A, Koenig T. 15 Years of Microstate Research in Schizophrenia—Where Are We? A Meta-Analysis. Front Psychiatry. 2016;7:22. Epub 2016/03/10. pmid:26955358; PubMed Central PMCID: PMC4767900.

47. Khanna A, Pascual-Leone A, Michel CM, Farzan F. Microstates in resting-state EEG: current status and future directions. Neurosci Biobehav Rev. 2015;49:105–13. Epub 2014/12/21. pmid:25526823; PubMed Central PMCID: PMC4305485.

48. Ranlund S, Nottage J, Shaikh M, Dutt A, Constante M, Walshe M, et al. Resting EEG in psychosis and at-risk populations—a possible endophenotype? Schizophr Res. 2014;153(1–3):96–102. Epub 2014/02/04. pmid:24486144; PubMed Central PMCID: PMC3969576.

49. de Bock R, Mackintosh AJ, Maier F, Borgwardt S, Riecher-Rossler A, Andreou C. EEG microstates as biomarker for psychosis in ultra-high-risk patients. Transl Psychiatry. 2020;10(1):300. Epub 2020/08/26. pmid:32839449; PubMed Central PMCID: PMC7445239.

50. Khanna A, Pascual-Leone A, Farzan F. Reliability of resting-state microstate features in electroencephalography. PloS one. 2014;9(12):e114163. Epub 2014/12/06. pmid:25479614; PubMed Central PMCID: PMC4257589.

51. Poulsen A, Pedroni A, Langer N, Hansen L. Microstate EEGlab toolbox: An introductory guide. 2018.

52. Pascual-Marqui RD, Michel CM, Lehmann D. Segmentation of brain electrical activity into microstates: model estimation and validation. IEEE Trans Biomed Eng. 1995;42(7):658–65. Epub 1995/07/01. pmid:7622149.

53. Koenig T, Prichep L, Lehmann D, Sosa PV, Braeker E, Kleinlogel H, et al. Millisecond by millisecond, year by year: normative EEG microstates and developmental stages. Neuroimage. 2002;16(1):41–8. Epub 2002/04/24. pmid:11969316.

54. Spring JN, Bourdillon N, Barral J. Resting EEG Microstates and Autonomic Heart Rate Variability Do Not Return to Baseline One Hour After a Submaximal Exercise. Front Neurosci. 2018;12:460. Epub 2018/07/26. pmid:30042654; PubMed Central PMCID: PMC6048261.

55. Wei Y, Ramautar JR, Colombo MA, Te Lindert BHW, Van Someren EJW. EEG Microstates Indicate Heightened Somatic Awareness in Insomnia: Toward Objective Assessment of Subjective Mental Content. Front Psychiatry. 2018;9:395. Epub 2018/09/22. pmid:30237769; PubMed Central PMCID: PMC6135918.

56. Cai Y, Chen S, Chen Y, Li J, Wang CD, Zhao F, et al. Altered Resting-State EEG Microstate in Idiopathic Sudden Sensorineural Hearing Loss Patients With Tinnitus. Front Neurosci. 2019;13:443. Epub 2019/05/28. pmid:31133786; PubMed Central PMCID: PMC6514099.

57. Tabachnick BG, Fidell LS. Using multivariate statistics. 6th ed. Boston: Pearson Education; 2013. xxxi, 983 p. p.

58. Grootswagers T, Wardle SG, Carlson TA. Decoding Dynamic Brain Patterns from Evoked Responses: A Tutorial on Multivariate Pattern Analysis Applied to Time Series Neuroimaging Data. Journal of Cognitive Neuroscience. 2017;29(4):677–97. WOS:000395122700008. pmid:27779910

59. Pereira F, Mitchell T, Botvinick M. Machine learning classifiers and fMRI: A tutorial overview. Neuroimage. 2009;45(1):S199–S209. WOS:000263862600018. pmid:19070668

60. Johnson RA, Wichern DW. Applied multivariate statistical analysis. 6th ed. Upper Saddle River, N.J.: Pearson Prentice Hall; 2007. xviii, 773 p. p.

61. Olejarczyk E, Jernajczyk W. EEG in schizophrenia. RepOD; 2017.

62. Oostenveld R, Fries P, Maris E, Schoffelen JM. FieldTrip: Open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Comput Intell Neurosci. 2011;2011:156869. Epub 2011/01/22. pmid:21253357; PubMed Central PMCID: PMC3021840.

63. Delorme A, Makeig S. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods. 2004;134(1):9–21. Epub 2004/04/23. pmid:15102499.

64. Grieder M, Koenig T, Kinoshita T, Utsunomiya K, Wahlund LO, Dierks T, et al. Discovering EEG resting state alterations of semantic dementia. Clin Neurophysiol. 2016;127(5):2175–81. Epub 2016/04/14. pmid:27072087.

65. Murray MM, Brunet D, Michel CM. Topographic ERP analyses: a step-by-step tutorial review. Brain Topogr. 2008;20(4):249–64. Epub 2008/03/19. pmid:18347966.

66. Pourtois G, Delplanque S, Michel C, Vuilleumier P. Beyond conventional event-related brain potential (ERP): exploring the time-course of visual emotion processing using topographic and principal component analyses. Brain Topogr. 2008;20(4):265–77. Epub 2008/03/14. pmid:18338243.

67. Spencer KM, Dien J, Donchin E. Spatiotemporal analysis of the late ERP responses to deviant stimuli. Psychophysiology. 2001;38(2):343–58. Epub 2001/05/12. pmid:11347879.

68. Makeig S, Debener S, Onton J, Delorme A. Mining event-related brain dynamics. Trends Cogn Sci. 2004;8(5):204–10. Epub 2004/05/04. pmid:15120678.

69. Lucia M, Michel C, Clarke S, Murray M. Single-subject EEG analysis based on topographic information. International Journal of Bioelectromagnetism wwwijbemorg. 2007;9:168–71.

70. Hadriche A, Pezard L, Nandrino JL, Ghariani H, Kachouri A, Jirsa VK. Mapping the dynamic repertoire of the resting brain. Neuroimage. 2013;78:448–62. Epub 2013/04/27. pmid:23618603.

71. Duc NT, Lee B. Microstate functional connectivity in EEG cognitive tasks revealed by a multivariate Gaussian hidden Markov model with phase locking value. J Neural Eng. 2019;16(2):026033. Epub 2019/01/24. pmid:30673644.

72. Tahernezhad-Javazm F, Azimirad V, Shoaran M. A review and experimental study on the application of classifiers and evolutionary algorithms in EEG-based brain-machine interface systems. J Neural Eng. 2018;15(2):021007. Epub 2017/07/19. pmid:28718779.

73. Park C, Took CC, Seong JK. Machine learning in biomedical engineering. Biomed Eng Lett. 2018;8(1):1–3. Epub 2019/01/04. pmid:30603186; PubMed Central PMCID: PMC6208556.

74. Lahmiri S, Dawson DA, Shmuel A. Performance of machine learning methods in diagnosing Parkinson’s disease based on dysphonia measures. Biomed Eng Lett. 2018;8(1):29–39. Epub 2017/10/12. pmid:30603188; PubMed Central PMCID: PMC6208554.

75. Cortes C, Vapnik V. Support-Vector Networks. Machine Learning. 1995;20(3):273–97. WOS:A1995RX35400003.

76. Fisher RA. THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS. Annals of Eugenics. 1936;7(2):179–88.

77. Rish I. An Empirical Study of the Naïve Bayes Classifier. IJCAI 2001 Work Empir Methods Artif Intell. 2001;3.

78. Breiman L. Random Forests. Machine Learning. 2001;45(1):5–32.

79. Zhang ML, Zhou ZH. ML-KNN: A lazy learning approach to multi-label leaming. Pattern Recognition. 2007;40(7):2038–48. WOS:000246332300016.

80. Nguyen DT, Ryu S, Qureshi MNI, Choi M, Lee KH, Lee B. Hybrid multivariate pattern analysis combined with extreme learning machine for Alzheimer’s dementia diagnosis using multi-measure rs-fMRI spatial patterns. PLoS One. 2019;14(2):e0212582. Epub 2019/02/23. pmid:30794629; PubMed Central PMCID: PMC6386400.

81. Nguyen DUC. Mild cognitive impairment diagnosis using extreme learning machine combined with multivoxel pattern analysis on multi-biomarker resting-state fMRI2019.

82. Duc NT, Ryu S, Qureshi MNI, Choi M, Lee KH, Lee B. 3D-Deep Learning Based Automatic Diagnosis of Alzheimer’s Disease with Joint MMSE Prediction Using Resting-State fMRI. Neuroinformatics. 2019. pmid:31093956

83. Koenig T, Kottlow M, Stein M, Melie-Garcia L. Ragu: a free tool for the analysis of EEG and MEG event-related scalp field data using global randomization statistics. Comput Intell Neurosci. 2011;2011:938925. Epub 2011/03/16. pmid:21403863; PubMed Central PMCID: PMC3049349.

84. Strik WK, Fallgatter AJ, Brandeis D, Pascual-Marqui RD. Three-dimensional tomography of event-related potentials during response inhibition: evidence for phasic frontal lobe activation. Electroencephalogr Clin Neurophysiol. 1998;108(4):406–13. Epub 1998/08/26. pmid:9714383.

85. Diagnostic and statistical manual of mental disorders: DSM-5. American Psychiatric A, American Psychiatric Association DSMTF, editors. Arlington, VA: American Psychiatric Association; 2013.

86. Upthegrove R, Marwaha S, Birchwood M. Depression and Schizophrenia: Cause, Consequence, or Trans-diagnostic Issue? Schizophr Bull. 2017;43(2):240–4. Epub 2016/07/17. pmid:27421793; PubMed Central PMCID: PMC5605248.

87. Muller VI, Cieslik EC, Laird AR, Fox PT, Eickhoff SB. Dysregulated left inferior parietal activity in schizophrenia and depression: functional connectivity and characterization. Front Hum Neurosci. 2013;7:268. Epub 2013/06/20. pmid:23781190; PubMed Central PMCID: PMC3679482.

88. Wu XJ, Zeng LL, Shen H, Yuan L, Qin J, Zhang P, et al. Functional network connectivity alterations in schizophrenia and depression. Psychiatry Res Neuroimaging. 2017;263:113–20. Epub 2017/04/04. pmid:28371656.

89. Lydon-Staley DM, Bassett DS. Network Neuroscience: A Framework for Developing Biomarkers in Psychiatry. Curr Top Behav Neurosci. 2018;40:79–109. Epub 2018/04/08. pmid:29626337.

90. Penner J, Ford KA, Taylor R, Schaefer B, Theberge J, Neufeld RW, et al. Medial Prefrontal and Anterior Insular Connectivity in Early Schizophrenia and Major Depressive Disorder: A Resting Functional MRI Evaluation of Large-Scale Brain Network Models. Front Hum Neurosci. 2016;10:132. Epub 2016/04/12. pmid:27064387; PubMed Central PMCID: PMC4811885.

91. Hugdahl K, Rund BR, Lund A, Asbjornsen A, Egeland J, Ersland L, et al. Brain activation measured with fMRI during a mental arithmetic task in schizophrenia and major depression. Am J Psychiatry. 2004;161(2):286–93. Epub 2004/02/03. pmid:14754778.

92. Shao J, Meng C, Tahmasian M, Brandl F, Yang Q, Luo G, et al. Common and distinct changes of default mode and salience network in schizophrenia and major depression. Brain Imaging Behav. 2018;12(6):1708–19. Epub 2018/02/21. pmid:29460166.

93. Hafner H, Maurer K, Trendler G, an der Heiden W, Schmidt M, Konnecke R. Schizophrenia and depression: challenging the paradigm of two separate diseases—a controlled study of schizophrenia, depression and healthy controls. Schizophr Res. 2005;77(1):11–24. Epub 2005/07/12. pmid:16005381.

94. Kuhn S, Gallinat J. Resting-state brain activity in schizophrenia and major depression: a quantitative meta-analysis. Schizophr Bull. 2013;39(2):358–65. Epub 2011/11/15. pmid:22080493; PubMed Central PMCID: PMC3576173.

95. Wei Y, Chang M, Womer FY, Zhou Q, Yin Z, Wei S, et al. Local functional connectivity alterations in schizophrenia, bipolar disorder, and major depressive disorder. J Affect Disord. 2018;236:266–73. Epub 2018/05/12. pmid:29751242.

96. Yuan S, Liu J, Shang J, Kong X, Yuan Q, Ma Z. The earth mover’s distance and Bayesian linear discriminant analysis for epileptic seizure detection in scalp EEG. Biomed Eng Lett. 2018;8(4):373–82. Epub 2019/01/04. pmid:30603222; PubMed Central PMCID: PMC6209084.

97. Chu WL, Huang MW, Jian BL, Cheng KS. Analysis of EEG entropy during visual evocation of emotion in schizophrenia. Ann Gen Psychiatry. 2017;16:34. Epub 2017/10/13. pmid:29021815; PubMed Central PMCID: PMC5613505.

98. Kim JW, Lee YS, Han DH, Min KJ, Lee J, Lee K. Diagnostic utility of quantitative EEG in un-medicated schizophrenia. Neurosci Lett. 2015;589:126–31. Epub 2015/01/18. pmid:25595562.

99. Won GH, Kim JW, Choi TY, Lee YS, Min KJ, Seol KH. Theta-phase gamma-amplitude coupling as a neurophysiological marker in neuroleptic-naive schizophrenia. Psychiatry Res. 2018;260:406–11. Epub 2017/12/19. pmid:29253805.

100. Bak N, Ebdrup BH, Oranje B, Fagerlund B, Jensen MH, During SW, et al. Two subgroups of antipsychotic-naive, first-episode schizophrenia patients identified with a Gaussian mixture model on cognition and electrophysiology. Transl Psychiatry. 2017;7(4):e1087. Epub 2017/04/12. pmid:28398342; PubMed Central PMCID: PMC5416700.

101. Sabeti M, Boostani R, Katebi SD, Price GW. Selection of relevant features for EEG signal classification of schizophrenic patients. Biomedical Signal Processing and Control. 2007;2(2):122–34. WOS:000205637100005.

102. Babiloni C, Triggiani AI, Lizio R, Cordone S, Tattoli G, Bevilacqua V, et al. Classification of Single Normal and Alzheimer’s Disease Individuals from Cortical Sources of Resting State EEG Rhythms. Front Neurosci. 2016;10:47. Epub 2016/03/05. pmid:26941594; PubMed Central PMCID: PMC4763025.

103. Raj VK, Rajagopalan SS, Bhardwaj S, Panda R, Reddam VR, Chaitanya G, et al. Machine learning detects EEG microstate alterations in patients living with temporal lobe epilepsy. Seizure-Eur J Epilep. 2018;61:8–13. WOS:000447477500003.

104. Goncalves SI, de Munck JC, Pouwels PJ, Schoonhoven R, Kuijer JP, Maurits NM, et al. Correlating the alpha rhythm to BOLD using simultaneous EEG/fMRI: inter-subject variability. Neuroimage. 2006;30(1):203–13. Epub 2005/11/18. pmid:16290018.

105. D’Croz-Baron DF, Baker M, Michel CM, Karp T. EEG Microstates Analysis in Young Adults With Autism Spectrum Disorder During Resting-State. Front Hum Neurosci. 2019;13:173. Epub 2019/06/28. pmid:31244624; PubMed Central PMCID: PMC6581708.