1. Introduction
Neurological diseases, such as amyotrophic lateral sclerosis, brainstem strokes, and spinal cord injuries, could lead to locked-in syndrome (LIS), which makes it impossible for LIS patients to have full voluntary muscle control [1]. Recently, interest in brain-computer interfaces (BCIs) has increased because they can replace the function of LIS patients’ impaired bodies.
BCI allows LIS patients to communicate with the external environment using only brain signals without any voluntary movements [2]. In other words, BCI systems can provide alternative communication channels for LIS patients [3]. BCIs can be realized using various neuroimaging modalities, such as electrocorticography (ECoG) [4,5], electroencephalography (EEG) [6,7], magnetoencephalography (MEG) [8,9], functional near-infrared spectroscopy (fNIRS) [10,11], and functional magnetic resonance imaging (fMRI) [12,13]. An invasive signal recording technique, such as ECoG, requires a surgical operation to place recording electrodes on the cortex. In contrast, non-invasive techniques, such as EEG, MEG, fNIRS, and fMRI, can provide safe recording of brain activity without surgery. Thus, the majority of BCI systems have been implemented based on non-invasive neuroimaging modalities [14]. Among the noninvasive modalities, EEG has been widely used to develop BCIs because it has high portability, reasonable cost, and high temporal resolution compared to other noninvasive recording modalities [14].
Conventional EEG-based BCI systems have used brain signals measured using scalp electrodes with conductive gels for accurate measurement of EEGs. However, the conventional measurement system is relatively bulky because it consists of several components, such as an external amplifier, cap, and electrode, which limits the value of BCI applications in terms of practical use. Recently, to overcome the limitation of conventional EEG-based BCI systems, EEGs measured by electrodes attached around the ears, called ear-EEGs, have been proposed as an alternative to the classical scalp-EEG [15,16]. It has been proven that ear-EEG can be set-up within several minutes, and used for successive days [17,18,19]. Most importantly, the feasibility of ear-EEG-based BCI systems has been demonstrated even though their performance is still lower than that of conventional EEG-based systems in general [18,20].
There are two approaches to develop EEG-based BCI systems according to whether external stimuli are used or not [21]. The first is based on an exogenous BCI paradigm that uses external stimuli to induce specific brain patterns, such as auditory steady-state responses (ASSRs) [22,23], event-related potentials (ERPs) [17,24,25,26,27,28], and steady-state visual evoked potentials (SSVEPs) [29]. The other approach is based on an endogenous BCI paradigm that uses self-modulated EEG patterns without external stimuli [30,31,32,33], such as sensorimotor rhythms [34] and slow cortical potential (SCP) [35]. Irrespective of if an exogenous or endogenous BCI system is developed, the most important factor to consider is practicality for clinical use and daily applications. Most previous BCI studies based on ear-EEG have used exogenous paradigms with external stimuli because of relatively high signal-to-noise ratio (SNR) compared to the SNR of endogenous paradigms. However, repetitive presentation of external stimuli used in exogenous BCI paradigms can cause user fatigue [36,37,38], and additional hardware is required to provide visual, auditory, or tactile external stimuli. The mentioned disadvantages would make it difficult to use an exogenous BCI system over a long period in daily life. Therefore, an endogenous BCI can be an alternative to the exogenous BCI in terms of practical use, but the feasibility of using ear-EEG to develop an endogenous BCI system has rarely been evaluated in previous BCI studies.
The objective of this study was to demonstrate the feasibility of ear-EEG for the development of an endogenous BCI system. To this end, EEGs were measured around the ears as well as on the scalp while subjects performed mental arithmetic (MA) and light cognitive (LC) tasks (imagining vocalization of English letters from A to Z). We compared event-related (de)synchronization (ERD/ERS) patterns induced by the MA and LC tasks between scalp-EEG and ear-EEG, and the classification accuracies of the MA and LC tasks to test the feasibility of an ear-EEG-based endogenous BCI system. As a result, similar ERD/ERS patterns were obtained between scalp- and ear-EEG during both MA and LC tasks. Also, the mean classification accuracy of ear-EEG was comparable to those of other scalp areas, and it was above 70% required for effective binary communication, demonstrating that the feasibility of using ear-EEG for the development of an endogenous BCI system.
2. Materials and Methods
2.1. Subjects
Eighteen healthy individuals were recruited for this study (21–31 years of age; mean 24.5 ± 2.67 years, 10 males and eight females). They reported no history of neurological or psychiatric conditions. All subjects were informed about the experimental procedure, and written consent was obtained from each subject before the experiment. All subjects received financial reimbursement after the experiment. The experimental protocol of this study was approved by the Institutional Review Board (IRB) of Kumoh National Institute of Technology (No. 6250). EEG data analysis was conducted for fifteen subjects, excluding three subjects who reported excessive fatigue during the experiment, and drinking alcohol in the previous day, which could affect experimental results.
2.2. EEG Measurement
EEG data were recorded in a sound-proof room, and subjects were seated in a comfortable armchair in front of a 21-inch monitor about 1 m away. During the experiment, the subjects were asked to refrain from any body movement to minimize physiological artifacts. A binaural audio system (Britz, BR-1000A, Cuve Black2, Paju-si, South Korea) was placed on both sides of the monitor, and it provided the subjects with auditory cues during the experiment. EEG data were recorded using thirty-one electrodes of a multi-channel EEG apparatus (Brain Products, GmbH, Gilching, Germany). Scalp-EEGs were measured using twenty-five electrodes attached to the scalp according to the international 10–20 system (Fp1–2, Fz, F3–4, 7-8, FC5–6, Cz, C3–4, T7–8, CP1–2, Pz, P3–4, 7–8, PO7–8, O1, and O2), while ear-EEG data were measured using six electrodes attached behind the ears (three electrodes for each ear). In order to measure ear-EEG, we first cleaned the skin behind the ears using an alcohol, a double-sided sticker was attached on the skin, a rubber ring holder was mounted on the sticker, and an electrode was inserted into the holder. Same types of electrodes were used to measure both scalp- and ear-EEG. The detailed information of electrode positions for scalp-EEG and ear-EEG is illustrated in Figure 1. Reference and ground electrodes were attached at FCz and Fpz, respectively. The sampling rate was 1000 Hz, and impedance was maintained below 10 kΩ during the entire experiment. The scalp- and ear-EEG were independently re-referenced before the analysis (see Section 2.4 for details).
Enlarge this image.Figure 1. Electrode positions used to record EEG data. (A) The brain area is divided into four regions of interests (ROIs) for data analysis (frontal, central, occipital, and ear area). (B) The electrode placement for ear-EEG.
2.3. Experimental Paradigm
Before the main experiment, we measured EEGs when the subjects kept their eyes closed (EC) and eyes opened (EO) for 30 s each, which was repeated six times. The preliminary experiment was conducted to check the feasibility of using ear-EEG by confirming the well-known neurophysiological phenomenon that there is a significant increase in alpha power (8–13 Hz) when the eyes are closed compared to when they are open.
Figure 2 shows the schematic diagram of the main experiment conducted to confirm whether an endogenous BCI paradigm can be realized using only ear-EEG. Two cognitive tasks, MA and LC, were employed in the main experiment. During MA, the subjects were instructed to sequentially subtract a single-digit number (between 5 and 9) from a three-digit number (e.g., 594 − 8) [31]. Fifty pairs of 3-digit and 1-digit numbers were randomly selected and presented to each subject, but the order of the 50 MA tasks was same between the subjects. We asked subjects to perform the consecutive subtraction as quickly as possible during the task period. During LC, they were instructed to mentally imagine vocalization of English letters from A to Z at 1 Hz without vocalization. The LC task was designed to maintain a steady and constant level of light cognitive load, which was introduced because subjects tend to randomly think something that might disturb the low loading state in a conventional resting state [30,31]. All subjects completed five sessions. Each experimental session started with an initial resting state in which a blank was first displayed for 5 s, which was followed by resting state where the string ‘ABC’ with an asterisk were presented. The subjects imagined vocalization of the English alphabet for 10 s while focusing on the asterisk at the center of a monitor screen that was used as a fixation mark to prevent severe ocular movement. A single trial was comprised of a task instruction of 5 s, followed by a task period of 10 s with a black fixation cross, and a variable resting period ranging from 10–15 s. During the task instruction period, either the MA problem or the ‘ABC’ string (LC) was randomly displayed on the screen for 5 s. The task started by presenting a black fixation cross that lasted for 10 s, during which the subjects performed either MA or LC according to an instruction presented on the monitor. The task period was followed by rest. A short beep was presented for 300 ms at every screen transition to provide subjects with explicit information of screen transition. Each session consisted of 10 MA and 10 LC trials, and a short break was given for several minutes between sessions. All subjects performed 50 MA and 50 LC trials (10 trials × 5 sessions).
Enlarge this image.Figure 2. Experimental paradigm of one session used in the main experiment. In the beginning of each session, a rest period of 10 s is performed. The string ‘ABC’ and an asterisk are presented to indicate a rest period and the subject is asked to fix the eyes to the asterisk to minimize ocular movement. After the rest period, either mental arithmetic (MA) or light cognitive (LC) task is randomly performed. For MA, a pair of a three-digit number and a single-digit number between 5 and 9 is randomly presented, and the subject is asked to sequentially subtract the single-digit number from the three-digit number (e.g., 477 − 8) for 10 s. For LC, the string ‘ABC’ is presented, and the subject is asked to internally imagine vocalization of the English alphabet from A to Z with a 1 Hz speed for 10 s. Both MA and LC are performed ten times in each session, and each subject completes five sessions (50 MA and 50 LC in total). A short beep (300 ms) is presented at every screen transition (red speaker icons).
2.4. EEG Data Analysis
EEG data were analyzed using EEGLAB [39,40] and BBCI toolbox [39,40] based on MATLAB (MathWorks, Natick, MA, USA). The EEG data were first bandpass-filtered between 1 and 50 Hz using a zero-phase fourth-order Butterworth filter and then down-sampled to 200 Hz in order to avoid introducing unwanted artifacts after downsampling. To compare neural characteristics and classification performance of different brain areas, we divided the brain area into four regions of interests (ROIs): frontal, central, occipital, and ear area (see Figure 1 for more detail about the four ROIs). For fair comparison, we selected six electrodes for each ROI, as follows: frontal (Fp1–2, F3–4, 7–8); central (FC5–6, C3–4, T7–8); occipital (P3–4, PO7–8, O1–2); and ear area (R1–3, L1–3). All ROIs were re-referenced to remove the impact of the original reference electrode (FCz). Three scalp ROIs were independently re-referenced using a common average reference (CAR) method with six electrodes each [41] while ear-area was re-referenced using a modified CAR in which the mean of three electrodes attached on an opposite ear area was used as a reference signal [16]. Trials contaminated by eye blinks and body movements were removed based on a peak-to-peak amplitude thresholding method [42]. Two samples showing the lowest and highest amplitudes were first found for each component caused by eye blink and the difference between them was calculated. If the difference was higher than a certain threshold at least once in the trial, a corresponding trial was removed. A threshold differed between subjects because a baseline amplitude varied from one subject to others (126.67 ± 22.95 μV for MA and LC; 196 ± 51.34 for EO/EC). The subjects blinked the eyes relatively strongly in the preliminary experiment as compared to in the main experiment, in particular, during the transition between EC and EO to get vision back. Note that EC was first performed, followed by EO. Thus, the threshold for EO/EC was set to be higher than that of MA and LC. The numbers of rejected trials for MA, LC, and EO/EC were 4 ± 1.66, 3.66 ± 1.06, 1.27 ± 0.77 in average, respectively.
For the EEG data measured in the preliminary experiment with EC and EO, time-frequency analysis was performed using a short-time Fourier transformation (window size: 1 s, 50% overlap). Spectral powers in a frequency band from 5–15 Hz, containing the α-band (8–13 Hz), were used to investigate alpha power changes between the EC and EO conditions. Relative changes of mean alpha power from EO to EC (signal-to-noise ratio: SNR) were estimated in decibels for the four ROIs (frontal, central, occipital, and ear area). The SNR is simply given as:
SNR=10×log10(alphaECalphaEO)
For the EEG data measured during MA and LC in the main experiment, ERD/ERS analysis was first performed [43], for which epochs from −2–10 s based on the task onset of MA and LC were extracted. Baseline correction was performed by subtracting the mean value of the EEG data recorded between −2–0 s from each data point. ERD/ERS patterns induced during MA and LC were calculated for each channel, and averaged over the six channels contained in each ROI. Classification of MA and LC was performed for each ROI. For feature extraction, a multi-band common spatial pattern (CSP) was applied to the epochs of MA and LC [44,45], where five frequency bands were used: δ-band (1–3 Hz), θ-band (4–7 Hz), α-band (8–13 Hz), β-band (14–29 Hz), and γ-band (30–50 Hz). The log-variances of the two first and last CSP components were calculated in each band as features for classification. Note that the multi-band CSP was applied to each ROI individually. To estimate classification accuracy, ten-fold cross-validation was performed ten times using shrinkage linear discriminant analysis (sLDA) [46,47]. All statistics were carried out using a Friedman test, and a Wilcoxon signed-rank sum test was used for post-hoc analysis with Bonferroni correction (i.e., p = 0.05/number of post-hoc tests).
3. Results
3.1. Alpha Power Changes during EC and EO
Figure 3 illustrates grand-average time-frequency maps of scalp- and ear-EEG in the frequency band from 5–15 Hz. As is well documented, a more significant increase in alpha power—around 10 Hz—is observed in the occipital area during EC than during EO (Figure 3C). Interestingly, a similar pattern is also shown for ear-EEG (Figure 3D), which can be explained by the fact that ear-EEG is measured close to the occipital lobe that mainly shows an alpha power increase during EC. Frontal and central areas also show an alpha power increase during EC, but which is not as strong as occipital and ear areas. The SNRs of the frontal, central, occipital, and ear area (EC/EO) are 0.74 ± 0.40, 1.55 ± 0.54, 4.20 ± 3.38, and 2.48 ± 1.32, respectively. As expected, the mean SNR of the occipital area is the highest and is significantly higher than the other ROIs. The SNR of the ear area is also statistically higher than both frontal and central areas (Wilcoxson signed rank sum test; occipital > ear > central > frontal, corrected p < 0.05).
Enlarge this image.Figure 3. Grand average time-frequency maps with eyes closed (EC) and eyes opened (EO) for (A) frontal, (B) central, (C) occipital, and (D) ear area. The color scale was chosen to fit the range for (D) ear area.
3.2. ERD/ERS Pattern Maps during MA and LC
Figure 4 shows grand average ERD/ERS maps of all electrodes during MA. Prominent ERS is observed in the α-band around 10 Hz at most electrodes, while a wide ERD is observed in the β- and γ-bands [32,33]. ERS is stronger in occipital and ear areas than fronto-central areas; an opposite trend is observed for ERD. Grand average ERD/ERS patterns of all electrodes during LC are shown in Figure 5. There are no distinct ERD/ERS patterns compared to those induced during MA. However, natural α-oscillation is visibly observed around the parieto-occipital areas and their adjacent ear areas, but it is not as strong as the alpha ERS induced during MA. Figure 6 presents grand average ERD/ERS pattern maps during MA and LC along with the difference between MA and LC (denoted by ‘MA-LC’) for the four ROIs.
Enlarge this image.Figure 4. Grand average ERD/ERS maps of all electrodes during MA. The four regions of interest (ROIs), frontal, central, occipital, and ear area, are denoted by four different colored lines and titles for each map (green, orange, red, and gray), respectively. The x- and y-axis of each map indicate the task time from −2–10 s based on task onset (t = 0 s), and the frequency band ranging from 1 to 50 Hz, respectively. ERD and ERS are presented in blue and red, respectively. Note that scalp- and ear-EEG are independently re-referenced using a CAR and a modified CAR, respectively.
Enlarge this image.Figure 5. Grand average ERD/ERS maps of all electrodes during LC. The four regions of interest (ROIs), frontal, central, occipital, and ear area, are denoted by four different colored lines and titles for each map (green, orange, red, and gray), respectively. The x- and y-axis of each map indicate the task time from −2–10 s based on task onset (t = 0 s), and the frequency band ranging from 1 to 50 Hz, respectively. ERD and ERS are presented in blue and red, respectively. Note that scalp- and ear-EEG are independently re-referenced using a CAR and a modified CAR, respectively.
Enlarge this image.Figure 6. Grand average ERD/ERS maps of each ROI during MA and LC, and their differences (MA-LC). The ERD/ERS maps of ear area (denoted by ‘Ear’) are obtained by averaging the six electrodes attached behind both ears (three electrode for each side). The x- and y-axis of each map indicate the task time from −2–10 s based on task onset (t = 0 s), and the frequency band ranging from 1 to 50 Hz, respectively. ERD and ERS are presented in blue and red, respectively.
The ERD/ERS patterns shown in Figure 4 and Figure 5 can be similarly observed for each ROI, and the difference between MA and LC is clearly seen (see ‘MA-LC’), leading to reasonable classification between the two conditions. Interestingly, ERD/ERS patterns for the ear area are very similar to those for the central and occipital areas adjacent to the ear area, demonstrating the feasibility of using ear-EEG to develop cognitive-task-based endogenous BCIs.
3.3. Classification Performance
Figure 7 shows the electrode positions of each ROI and their mean classification accuracies. The classification accuracy attained using all electrodes re-referenced using CAR, excluding the six ear electrodes, is also presented as a reference accuracy. The mean accuracies of the frontal, central, occipital, and ear areas are 81.26 ± 9.72, 80.78 ± 8.60, 84.55 ± 5.75 and 78.36 ± 10.36%, respectively (all electrodes: 91.80 ± 5.60%). The classification accuracy of the occipital area is higher than those of the other ROIs, but there is no statistically significant difference between the four ROIs (Friedman test; p = 0.63).
Enlarge this image.Figure 7. (A) Electrode positions used to create each ROI, and (B) the mean classification accuracies of the four ROIs with that obtained using all electrodes (‘Scalp’), excluding the six ear electrodes. Each ROI was individually re-referenced, where a CAR was used for the scalp ROIs (‘Scalp’, ‘Frontal’, ‘Central’, and ‘Occipital’) while the mean of three electrodes attached on an opposite ear area was used as a reference signal for ear ROI (‘Ear’). Error bars indicate standard deviations of the estimated classification accuracies of each ROI. There is no significant difference between the four ROIs (Friedman test; p = 0.63).
4. Discussion
In recent years, ear-EEG has been introduced to develop a more practical BCI system as an alternative to the classical scalp-EEG; its feasibility has been demonstrated in many previous studies [15,16,17,23,25,26,27,48,49,50,51,52,53]. Most of previous studies have used exogenous BCI paradigms that take advantage of brain activity evoked by external stimuli, such as ASSR [16,18,23,48,54,55,56,57], SSVEP [23,51,55,57], and ERP [15,17,23,25,26,27,48,49,58], while few studies based on ear-EEG have developed endogenous BCI systems based on self-modulated brain activity. Development of an endogenous BCI system is important because continuous external visual or auditory stimulation can readily cause fatigue [29,59,60], and the performance of these BCI systems is significantly degraded when the user has deficits in visual or auditory function [59,61]. The goal of this study was to investigate whether ear-EEG can be used to develop an endogenous BCI system with reliable performance. To this end, we conducted preliminary and main experiments to check the feasibility of using an ear-EEG based on the EC/EO task and an endogenous BCI paradigm (MA vs. LC), respectively.
In the preliminary experiment, we compared changes in alpha activity induced during EC and EO for each ROI, and we confirmed that the SNR of ear-EEG was comparable even though it was smaller than that of the occipital area, which showed the highest increase of alpha activity during EC. This result demonstrated the feasibility of using ear-EEG as compared to traditional scalp-EEG, and is in line with previous reports [17,25,57]. Even though the previous studies investigated alpha power changes induced during EC and EO, they only checked them using ear-EEG without direct comparison between ear-EEG and scalp-EEG. So, it should be noted that this study is first to quantitatively compare the SNR of ear-EEG with those of other brain areas in terms of alpha activity increase related to EC. The comparison presented in this study could provide more detailed information about the degree of utilization of ear-EEG as compared to scalp-EEG in developing EEG applications based on alpha activity, such as brain authentication based on the resting state [62], attention monitoring [63,64], and sleep detection [65].
In the main experiment, we used a MA task that is one of the widely employed cognitive tasks for developing an endogenous BCI system [31,33,66,67,68,69]. A significant alpha ERS and a wide-band ERD in the β- and γ-bands were observed for most electrodes during MA, which is consistent with previous studies [32,33]. Most importantly, similar ERD/ERS patterns were also shown for ear-EEG during MA. Also, the ERD/ERS pattern maps of the ear area were more similar to those of centro-occipital areas than those of frontal areas, which was also indirectly proved in a previous study showing that centro-occipital EEGs can be more predicted using ear-EEG than frontal EEG [70]. As documented in related studies [33,71], the strongest alpha ERS was observed during MA on occipital areas that are most sensitive to cognitive tasks [33], thereby showing the highest classification accuracy between MA and LC (84.55 ± 5.75%). Importantly, the classification performance of the ear area was comparable to the other ROIs from the statistical point of view (Friedman test; p = 0.63). In addition, the mean classification accuracy of ear-EEG exceeded the marginal accuracy of 70% that determines whether a binary BCI system can be practically used [72]. The ERD/ERS and classification results could demonstrate that ear-EEG can be used to develop an endogenous BCI system with acceptable performance.
In this study, we used MA and LC (mental vocalization task) tasks to confirm the feasibility of ear-EEG for the development of an endogenous BCI system. So far, various mental tasks have been used to develop endogenous BCI systems, such as motor imagery [34,73], mental rotation [74], spatial navigation [75], mental vocalization [76,77], and word association [78]. Thus, further studies for other mental tasks should be performed to more generally address the feasibility of ear-EEG for developing an endogenous BCI system, which could also provide the information about proper combinations of mental tasks for implementing ear-EEG-based BCIs. On the other hand, we used one of the language tasks as a LC task because it was confirmed that the LC task used in this study can maintain constant level of low cognitive loading [31]. However, considering that a mental subtraction task was used as a main task in this study in order to actively induce self-modulated EEGs, an easy subtraction task (e.g., 100 − 1) might be a more proper selection as a LC task instead of the language task for the sake of task unity. Investigating the difference between the language task used in this study and an easy subtraction task will be considered in terms of neural characteristics and task performance in future studies.
A multi-band CSP and sLDA were used for feature extraction and classification, respectively, in this study. However, because there is no guarantee that sLDA is always most suitable for classification of given EEG data, another classifier can be considered. For example, random forest, k-nearest neighbor, and Gaussian mixture model showed better classification performance than LDA for sleep EEGs [79]. Because a most suitable classifier highly depends on the characteristics of given dataset, trial and error with different classifiers should be undergone to find a better classifier [80]. In this study, sLDA was a better choice than other two classifiers tested, random forest and support vector machine, in terms of classification accuracy (not shown here in detail).
As mentioned above, most ear-EEG studies have used external auditor/visual stimuli and have demonstrated that ear-EEG can reliably capture evoked brain potentials, such as SSVEP [23,51,55,57], ASSR [16,18,23,48,54,55,56,57], and ERP [15,17,23,25,26,27,48,49,58]. In this study, we demonstrated that ear-EEG can also measure self-modulated brain activity (Figure 6). Thus, it can be thought that spontaneous brain activity generated without the repetitive use of external stimuli and the execution of mental tasks might be captured using ear-EEG. Thus, investigation of spontaneous brain activity measured around the ears would be an interesting future research topic to expand the application areas of ear-EEG, such as emotion recognition and epileptic seizure detection.
Author Contributions
S.-I.C., H.-J.H., and C.-H.I. designed the experiment, S.-I.C., and G.-Y.C. acquired the data, and S.-I.C., and J.S. performed data analysis. C.-H.H., C.-H.I., and K.S.S. reviewed the draft. All authors wrote the manuscript.
Funding
This work was supported by Institute for Information & Communications Technology Promotion (IITP) grant funded by the Korea government (MSIT) (No. 2017-0-00451), and by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIP; Ministry of Science, ICT & Future Planning) (No. 2017R1C1B5017909).
Conflicts of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be constructed as a potential conflict of interest.
1. Bauer, G.; Gerstenbrand, F.; Rumpl, E. Varieties of the locked-in syndrome. J. Neurol. 1979, 221, 77–91.
2. He, S.; Zhang, R.; Wang, Q.; Chen, Y.; Yang, T.; Feng, Z.; Zhang, Y.; Shao, M.; Li, Y. A P300-based threshold-free brain switch and its application in wheelchair control. IEEE Trans. Neural Syst. Rehabil. Eng. 2017, 25, 715–725.
3. Allison, B.Z.; Dunne, S.; Leeb, R.; Millán, J.; Nijholt, A. Recent and upcoming BCI progress: Overview, analysis, and recommendations. In Towards Practical Brain-Computer Interfaces: Bridging the Gap from Research to Real-World Applications; Springer: Berlin, Germany, 2013; pp. 1–13.
4. Eric, C.L.; Gerwin, S.; Jonathan, R.W.; Jeffrey, G.O.; Daniel, W.M. A brain-computer interface using electrocorticographic signals in humans. J. Neural Eng. 2004, 1, 63.
5. Alan, D.D.; Shivayogi, V.H.; Ying, Y.; Stephen, F.; Jennifer, L.C.; Michael, B.; Elizabeth, C.T.-K.; Wei, W. Remapping cortical modulation for electrocorticographic brain-computer interfaces: A somatotopy-based approach in individuals with upper-limb paralysis. J. Neural Eng. 2018, 15, 026021.
6. Wolpaw, J.R.; Birbaumer, N.; McFarland, D.J.; Pfurtscheller, G.; Vaughan, T.M. Brain-computer interfaces for communication and control. Clin. Neurophysiol. 2002, 113, 767–791.
7. Shu, X.; Chen, S.; Yao, L.; Sheng, X.; Zhang, D.; Jiang, N.; Jia, J.; Zhu, X. Fast recognition of BCI-inefficient users using physiological features from EEG signals: A screening study of stroke patients. Front. Neurosci. 2018, 12.
8. Mellinger, J.; Schalk, G.; Braun, C.; Preissl, H.; Rosenstiel, W.; Birbaumer, N.; Kubler, A. An MEG-based brain-computer interface (BCI). NeuroImage 2007, 36, 581–593.
9. Corsi, M.-C.; Chavez, M.; Schwartz, D.; Hugueville, L.; Khambhati, A.N.; Bassett, D.S.; Fallani, F.D.V. Integrating EEG and MEG signals to improve motor imagery classification in brain-computer interface. Int. J. Neural Syst 2018, 38, 1850014.
10. Sitaram, R.; Zhang, H.; Guan, C.; Thulasidas, M.; Hoshi, Y.; Ishikawa, A.; Shimizu, K.; Birbaumer, N. Temporal classification of multichannel near-infrared spectroscopy signals of motor imagery for developing a brain-computer interface. NeuroImage 2007, 34, 1416–1427.
11. Naseer, N.; Hong, K.-S. fNIRS-based brain-computer interfaces: A review. Front. Hum. Neurosci. 2015, 9, 3.
12. Weiskopf, N.; Mathiak, K.; Bock, S.W.; Scharnowski, F.; Veit, R.; Grodd, W.; Goebel, R.; Birbaumer, N. Principles of a brain-computer interface (BCI) based on real-time functional magnetic resonance imaging (fMRI). IEEE Trans. Biomed. Eng. 2004, 51, 966–970.
13. Sokunbi, M.O. Using real-time fMRI brain-computer interfacing to treat eating disorders. J. Neurol. Sci. 2018, 388, 109–114.
14. Hwang, H.-J.; Kim, S.; Choi, S.; Im, C.-H. EEG-based brain-computer interfaces: A thorough literature survey. Int. J. Hum. Comput. Interact. 2013, 29, 814–826.
15. Bleichner, M.G.; Lundbeck, M.; Selisky, M.; Minow, F.; Jager, M.; Emkes, R.; Debener, S.; De Vos, M. Exploring miniaturized EEG electrodes for brain-computer interfaces. An EEG you do not see? Physiol. Rep. 2015, 3, e12362.
16. Mikkelsen, K.B.; Kappel, S.L.; Mandic, D.P.; Kidmose, P. EEG recorded from the ear: Characterizing the ear-EEG method. Front. Neurosci. 2015, 9, 438.
17. Debener, S.; Emkes, R.; De Vos, M.; Bleichner, M. Unobtrusive ambulatory EEG using a smartphone and flexible printed electrodes around the ear. Sci. Rep. 2015, 5, 16743.
18. Looney, D.; Kidmose, P.; Park, C.; Ungstrup, M.; Rank, M.; Rosenkranz, K.; Mandic, D. The in-the-ear recording concept: User-centered and wearable brain monitoring. IEEE Pulse 2012, 3, 32–42.
19. Zibrandtsen, I.; Kidmose, P.; Otto, M.; Ibsen, J.; Kjaer, T.W. Case comparison of sleep features from ear-EEG and scalp-EEG. Sleep Sci. 2016, 9, 69–72.
20. Mikkelsen, K.B.; Villadsen, D.B.; Otto, M.; Kidmose, P. Automatic sleep staging using ear-EEG. Biomed. Eng. Online 2017, 16, 111.
21. Nicolas-Alonso, L.F.; Gomez-Gil, J. Brain computer interfaces, a review. Sensors 2012, 12, 1211–1279.
22. Kim, D.W.; Hwang, H.J.; Lim, J.H.; Lee, Y.H.; Jung, K.Y.; Im, C.H. Classification of selective attention to auditory stimuli: Toward vision-free brain-computer interfacing. J. Neurosci. Methods 2011, 197, 180–185.
23. Kidmose, P.; Looney, D.; Ungstrup, M.; Rank, M.L.; Mandic, D.P. A study of evoked potentials from ear-EEG. IEEE Trans. Biomed. Eng. 2013, 60, 2824–2830.
24. Hwang, H.-J.; Ferreria, V.Y.; Ulrich, D.; Kilic, T.; Chatziliadis, X.; Blankertz, B.; Treder, M. A gaze independent brain-computer interface based on visual stimulation through closed eyelids. Sci. Rep. 2015, 5, 15890.
25. Norton, J.J.S.; Lee, D.S.; Lee, J.W.; Lee, W.; Kwon, O.; Won, P.; Jung, S.-Y.; Cheng, H.; Jeong, J.-W.; Akce, A.; et al. Soft, curved electrode systems capable of integration on the auricle as a persistent brain-computer interface. Proc. Natl. Acad. Sci. USA 2015, 112, 3920–3925.
26. Bleichner, M.G.; Mirkovic, B.; Debener, S. Identifying auditory attention with ear-EEG: cEEGrid versus high-density cap-EEG comparison. J. Neural Eng. 2016, 13, 066004.
27. Fiedler, L.; Wostmann, M.; Graversen, C.; Brandmeyer, A.; Lunner, T.; Obleser, J. Single-channel in-ear-EEG detects the focus of auditory attention to concurrent tone streams and mixed speech. J. Neural Eng. 2017, 14, 036020.
28. Won, D.O.; Hwang, H.J.; Kim, D.M.; Müller, K.R.; Lee, S.W. Motion-based rapid serial visual presentation for gaze-independent brain-computer interfaces. IEEE Trans. Neural Syst. Rehabil. Eng. 2018, 26, 334–343.
29. Hwang, H.J.; Lim, J.H.; Jung, Y.J.; Choi, H.; Lee, S.W.; Im, C.H. Development of an SSVEP-based BCI spelling system adopting a QWERTY-style LED keyboard. J. Neurosci. Methods 2012, 208, 59–65.
30. Power, S.D.; Kushki, A.; Chau, T. Towards a system-paced near-infrared spectroscopy brain-computer interface: Differentiating prefrontal activity due to mental arithmetic and mental singing from the no-control state. J. Neural Eng. 2011, 8, 066004.
31. Shin, J.; Müller, K.R.; Hwang, H.J. Near-infrared spectroscopy (NIRS)-based eyes-closed brain-computer interface (BCI) using prefrontal cortex activation due to mental arithmetic. Sci. Rep. 2016, 6, 36203.
32. Friedrich, E.V.; Scherer, R.; Neuper, C. The effect of distinct mental strategies on classification performance for brain-computer interfaces. Int. J. Psychophysiol. 2012, 84, 86–94.
33. Shin, J.; Müller, K.-R.; Schmitz, C.H.; Kim, D.-W.; Hwang, H.-J. Evaluation of a compact hybrid brain-computer interface system. Biomed. Res. Int. 2017, 2017, 11.
34. Hwang, H.-J.; Kwon, K.; Im, C.-H. Neurofeedback-based motor imagery training for brain-computer interface (BCI). J. Neurosci. Methods 2009, 179, 150–156.
35. Mensh, B.D.; Werfel, J.; Seung, H.S. BCI competition 2003-data set Ia: Combining gamma-band power with slow cortical potentials to improve single-trial classification of electroencephalographic signals. IEEE Trans. Biomed. Eng. 2004, 51, 1052–1056.
36. Kim, Y.-J.; Lee, E.C. EEG based comparative measurement of visual fatigue caused by 2D and 3D displays. In Proceedings of the International Conference on Human-Computer Interaction, Berlin, Germany, 9–14 July 2011; pp. 289–292.
37. Khan, M.J.; Hong, K.S. Passive BCI based on drowsiness detection: An fNIRS study. Biomed. Opt. Express 2015, 6, 4063–4078.
38. Myrden, A.; Chau, T. Effects of user mental state on EEG-BCI performance. Front. Hum. Neurosci. 2015, 9, 308.
39. Delorme, A.; Makeig, S. EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J. Neurosci. Methods 2004, 134, 9–21.
40. Blankertz, B.; Acqualagna, L.; Dähne, S.; Haufe, S.; Schultze-Kraft, M.; Sturm, I.; Ušćumlic, M.; Wenzel, M.A.; Curio, G.; Müller, K.-R. The Berlin Brain-Computer Interface: Progress beyond communication and control. Front. Neurosci. 2016, 10, 530.
41. McFarland, D.J.; McCane, L.M.; David, S.V.; Wolpaw, J.R. Spatial filter selection for EEG-based communication. Electroencephalogr. Clin. Neurophysiol. 1997, 103, 386–394.
42. Zhang, J.; Lau, E.Y.Y. Sleep deprivation compromises resting-state emotional regulatory processes: An EEG study. J. Sleep Res. 2018, 27.
43. Pfurtscheller, G. Graphical display and statistical evaluation of event-related desynchronization (ERD). Electroencephalogr. Clin. Neurophysiol. 1977, 43, 757–760.
44. Lemm, S.; Blankertz, B.; Curio, G.; Muller, K.R. Spatio-spectral filters for improving the classification of single trial EEG. IEEE Trans. Biomed. Eng. 2005, 52, 1541–1548.
45. Ramoser, H.; Muller-Gerking, J.; Pfurtscheller, G. Optimal spatial filtering of single trial EEG during imagined hand movement. IEEE Trans. Rehabil. Eng. 2000, 8, 441–446.
46. Peck, R.; Ness, J.V. The use of shrinkage estimators in linear discriminant analysis. IEEE Trans. Pattern Anal. Mach. Intell. 1982, 4, 530–537.
47. Schäfer, J.; Strimmer, K. A shrinkage approach to large-scale covariance matrix estimation and implications for functional genomics. Stat. Appl. Genet. Mol. Biol. 2005, 4, 32.
48. Kidmose, P.; Looney, D.; Mandic, D.P. Auditory evoked responses from Ear-EEG recordings. In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), San Diego, CA, USA, 28 August–1 September 2012; pp. 586–589.
49. Fiedler, L.; Obleser, J.; Lunner, T.; Graversen, C. Ear-EEG allows extraction of neural responses in challenging listening scenarios—A future technology for hearing aids? In Proceedings of the 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA, 16–20 August 2016; pp. 5697–5700.
50. Mirkovic, B.; Bleichner, M.G.; De Vos, M.; Debener, S. Target speaker detection with concealed EEG around the ear. Front. Neurosci. 2016, 10, 349.
51. Wang, Y.T.; Nakanishi, M.; Wang, Y.; Wei, C.S.; Cheng, C.K.; Jung, T.P. An online brain-computer interface based on SSVEPs measured from non-hair-bearing areas. IEEE Trans. Neural Syst. Rehabil. Eng. 2017, 25, 11–18.
52. Floriano, A.; Diez, P.F.; Bastos-Filho, T.F. Evaluating the influence of chromatic and luminance stimuli on SSVEPs from behind-the-ears and occipital areas. Sensors 2018, 18, 615.
53. Wei, C.S.; Wang, Y.T.; Lin, C.T.; Jung, T.P. Toward drowsiness detection using non-hair-bearing EEG-based brain-computer interfaces. IEEE Trans. Neural Syst. Rehabil. Eng. 2018, 26, 400–406.
54. Kidmose, P.; Looney, D.; Jochumsen, L.; Mandic, D.P. Ear-EEG from generic earpieces: A feasibility study. In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Osaka, Japan, 3–7 July 2013; pp. 543–546.
55. Goverdovsky, V.; Looney, D.; Kidmose, P.; Mandic, D.P. In-ear EEG from viscoelastic generic earpieces: Robust and unobtrusive 24/7 monitoring. IEEE Sens. J. 2016, 16, 271–277.
56. Bech Christensen, C.; Harte, J.M.; Lunner, T.; Kidmose, P. Ear-EEG based objective hearing threshold estimation evaluated on normal hearing subjects. IEEE Trans. Biomed. Eng. 2017, 99, 28796603.
57. Goverdovsky, V.; von Rosenberg, W.; Nakamura, T.; Looney, D.; Sharp, D.J.; Papavassiliou, C.; Morrell, M.J.; Mandic, D.P. Hearables: Multimodal physiological in-ear sensing. Sci. Rep. 2017, 7, 6948.
58. Pacharra, M.; Debener, S.; Wascher, E. Concealed around-the-ear EEG captures cognitive processing in a visual simon task. Front. Hum. Neurosci. 2017, 11, 290.
59. Hong, B.; Lou, B.; Guo, J.; Gao, S. Adaptive active auditory brain computer interface. In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Minneapolis, MN, USA, 3–6 September 2009; pp. 4531–4534.
60. Allison, B.; Luth, T.; Valbuena, D.; Teymourian, A.; Volosyak, I.; Graser, A. BCI demographics: How many (and what kinds of) people can use an SSVEP BCI? IEEE Trans. Neural Syst. Rehabil. Eng. 2010, 18, 107–116.
61. Brunner, P.; Joshi, S.; Briskin, S.; Wolpaw, J.R.; Bischof, H.; Schalk, G. Does the ‘P300’ speller depend on eye gaze? J. Neural Eng. 2010, 7, 056013.
62. Choi, G.Y.; Choi, S.I.; Hwang, H.J. Individual identification based on resting-state EEG. In Proceedings of the 6th International Conference on Brain-Computer Interface (BCI), Seoul, Korea, 15–17 January 2018; pp. 1–4.
63. Benedek, M.; Schickel, R.J.; Jauk, E.; Fink, A.; Neubauer, A.C. Alpha power increases in right parietal cortex reflects focused internal attention. Neuropsychologia 2014, 56, 393–400.
64. Katahira, K.; Yamazaki, Y.; Yamaoka, C.; Ozaki, H.; Nakagawa, S.; Nagata, N. EEG correlates of the flow state: A combination of increased frontal theta and moderate frontocentral alpha rhythm in the mental arithmetic task. Front. Psychol. 2018, 9, 300.
65. Jiao, Y.; Lu, B.L. Detecting driver sleepiness from EEG alpha wave during daytime driving. In Proceedings of the 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Kansas, MO, USA, 13–16 November 2017; pp. 728–731.
66. Bauernfeind, G.; Scherer, R.; Pfurtscheller, G.; Neuper, C. Single-trial classification of antagonistic oxyhemoglobin responses during mental arithmetic. Med. Biol. Eng. Comput. 2011, 49, 979–984.
67. Harrison, A.H.; Noseworthy, M.D.; Reilly, J.P.; Guan, W.; Connolly, J.F. EEG and fMRI agree: Mental arithmetic is the easiest form of imagery to detect. Conscious Cogn. 2017, 48, 104–116.
68. Shin, J.; Lühmann, A.v.; Blankertz, B.; Kim, D.W.; Jeong, J.; Hwang, H.J.; Müller, K.R. Open access dataset for EEG + NIRS single-trial classification. IEEE Trans. Neural Syst. Rehabil. Eng. 2017, 25, 1735–1745.
69. Shin, J.; von Lühmann, A.; Kim, D.-W.; Mehnert, J.; Hwang, H.-J.; Müller, K.-R. Simultaneous acquisition of EEG and NIRS during cognitive tasks for an open access dataset. Sci. Data 2018, 5, 180003.
70. Mikkelsen, K.B.; Kidmose, P.; Hansen, L.K. On the keyhole hypothesis: High mutual information between ear and scalp EEG. Front. Hum. Neurosci. 2017, 11, 341.
71. Guay, S.; De Beaumont, L.; Drisdelle, B.L.; Lina, J.-M.; Jolicoeur, P. Electrophysiological impact of multiple concussions in asymptomatic athletes: A re-analysis based on alpha activity during a visual-spatial attention task. Neuropsychologia 2018, 108, 42–49.
72. Perelmouter, J.; Birbaumer, N. A binary spelling interface with random errors. IEEE Trans. Rehabil. Eng. 2000, 8, 227–232.
73. Park, S.A.; Hwang, H.J.; Lim, J.H.; Choi, J.H.; Jung, H.K.; Im, C.H. Evaluation of feature extraction methods for EEG-based brain-computer interfaces in terms of robustness to slight changes in electrode locations. Med. Biol. Eng. Comput. 2013, 51, 571–579.
74. So, W.K.Y.; Wong, S.W.H.; Mak, J.N.; Chan, R.H.M. An evaluation of mental workload with frontal EEG. PLoS ONE 2017, 12, e0174949.
75. Curran, E.; Sykacek, P.; Stokes, M.; Roberts, S.J.; Penny, W.; Johnsrude, I.; Owen, A.M. Cognitive tasks for driving a brain-computer interfacing system: A pilot study. IEEE Trans. Neural Syst. Rehabil. Eng. 2004, 12, 48–54.
76. Naito, M.; Michioka, Y.; Ozawa, K.; Ito, Y.; Kiguchi, M.; Kanazawa, T. A communication means for totally locked-in ALS patients based on changes in cerebral blood volume measured with near-infrared light. IEICE Trans. Inf. Syst. 2007, 7, 1028–1037.
77. Power, S.D.; Falk, T.H.; Chau, T. Classification of prefrontal activity due to mental arithmetic and music imagery using hidden Markov models and frequency domain near-infrared spectroscopy. J. Neural Eng. 2010, 7, 26002.
78. Petersen, S.E.; Fox, P.T.; Posner, M.I.; Mintun, M.; Raichle, M.E. Positron emission tomographic studies of the cortical anatomy of single-word processing. Nature 1988, 331, 585.
79. Boostani, R.; Karimzadeh, F.; Nami, M. A comparative review on sleep stage classification methods in patients and healthy individuals. Comput. Methods Programs Biomed. 2017, 140, 77–91.
80. Alimardani, F.; Cho, J.H.; Boostani, R.; Hwang, H.J. Classification of bipolar disorder and schizophrenia using steady-state visual evoked potential based features. IEEE Access 2018, 6, 40379–40388.
1Department of Medical IT Convergence Engineering, Kumoh National Institute of Technology, Gumi 39177, Korea
2Berlin Institute of Technology, Machine Learning Group, Marchstrasse 23, 10587 Berlin, Germany
3Department of Biomedical Engineering, Hanyang University, Seoul 04763, Korea
*Author to whom correspondence should be addressed.
© 2018. This work is licensed under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.