Prediction of P300 BCI Aptitude in Severe Motor Impairment

Abstract

Brain-computer interfaces (BCIs) provide a non-muscular communication channel for persons with severe motor impairments. Previous studies have shown that the aptitude with which a BCI can be controlled varies from person to person. A reliable predictor of performance could facilitate selection of a suitable BCI paradigm. Eleven severely motor impaired participants performed three sessions of a P300 BCI web browsing task. Before each session auditory oddball data were collected to predict the BCI aptitude of the participants exhibited in the current session. We found a strong relationship of early positive and negative potentials around 200 ms (elicited with the auditory oddball task) with performance. The amplitude of the P2 (r  =  −0.77) and of the N2 (r  =  −0.86) had the strongest correlations. Aptitude prediction using an auditory oddball was successful. The finding that the N2 amplitude is a stronger predictor of performance than P3 amplitude was reproduced after initially showing this effect with a healthy sample of BCI users. This will reduce strain on the end-users by minimizing the time needed to find suitable paradigms and inspire new approaches to improve performance.

About the Authors:

Sebastian Halder

* E-mail: [email protected]

Affiliations Institute of Psychology, University of Würzburg, Würzburg, Germany, Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany, Department of Computer Engineering, University of Tübingen, Tübingen, Germany

Carolin Anne Ruf

Affiliation: Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany

Adrian Furdea

Affiliation: Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany

Emanuele Pasqualotto

Affiliation: Psychological Sciences Research Institute, Université catholique de Louvain, Louvain-la-neuve, Belgium

Daniele De Massari

Affiliations Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany, Ospedale San Camillo, Istituto Di Ricovero e Cura a Carattere Scientifico Fondazione, Venezia-Lido, Italy, Graduate Training Centre of Neuroscience, International Max Planck Research School, Tübingen, Germany

Linda van der Heiden

Affiliations Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany, Department of Cognitive Psychology, University of Finance and Management, Warsaw, Poland

Martin Bogdan

Affiliations Department of Computer Engineering, University of Tübingen, Tübingen, Germany, Computer Engineering, University of Leipzig, Leipzig, Germany

Wolfgang Rosenstiel

Affiliation: Department of Computer Engineering, University of Tübingen, Tübingen, Germany

Niels Birbaumer

Affiliations Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany, Ospedale San Camillo, Istituto Di Ricovero e Cura a Carattere Scientifico Fondazione, Venezia-Lido, Italy

Andrea Kübler

Affiliation: Institute of Psychology, University of Würzburg, Würzburg, Germany

Tamara Matuz

Affiliation: Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany

Introduction

One of the earliest discussions of a communication channel that is independent of muscular control can be found in Vidal et al. [1]. These communication systems, termed brain-computer interfaces (BCIs), use components extracted from the electroencephalogram (EEG) as a control signal. Control signals include slow cortical potentials (SCPs), the P300 event-related potential (ERP) component, visually evoked potentials (VEPs) and the sensorimotor-rhythm (SMR) [2], [3], [4], [5]. Brain activity used for BCI control can also be measured with functional magnetic resonance imaging (fMRI) [6], [7], [8], electrocorticography (ECoG) [9], [10], magnetoencephalography (MEG) [11] and functional near infared red spectroscopy (fNIRS) [12]. Recent studies have also shown that the EEG response to yes and no questions can be conditioned and may be used to control a binary communication system [13], [14], [15], [16]. Hybrid-BCIs combine different control signals [17], as extensively reviewed by van Gerven et al. [18].

The P300 BCI paradigm was first introduced by Farwell and Donchin [3]. The user shifts his or her attention to a single letter in a matrix. The rows and columns of this matrix then flash in a random pattern. When the row or column containing the letter the user is focusing on flashes this elicits a P300. The P300 following the target stimulations is larger than the P300 following non-target stimulations which enables the classification of the intended row and column. Classification requires repetition of flashes. Averaging trials decreases the amplitude of the spontaneous EEG and conserves the amplitude of the stimulus-locked P300. This increases the signal-to-noise ratio (SNR) enabling successful classification. Implementations of this BCI paradigm commonly use the linear discriminant analysis LDA algorithm (and derivations thereof) for classification [3], [19], [20], [21]. For a review of classification algorithms for BCIs see Lotte et al. [22].

Persons with severe motor impairments are differentiated into the LIS and the complete locked-in state CLIS [23]. Persons in the LIS retain residual muscle control for basic communication. Persons in the CLIS have no muscle control and cannot communicate. Before entering the CLIS BCI performance is not influenced by the severity of the disease [24]. To date attempts to achieve this goal have failed [23], [25].Therefore, it is one of the primary goals of BCI research to restore communication for persons in the CLIS. In amyotrophic lateral sclerosis (ALS) the transition from LIS to CLIS is gradual and the precise time point is hard to define. In a single case study, a person with ALS lost control of facial muscles and the external anal sphincter before losing control of eye movements [25]. Besides ALS, other disorders can lead to LIS or even CLIS. These include brainstem stroke, cerebral palsy, muscular dystrophies and spinal cord injury (SCI). As communication is a basic need, BCI could potentially contribute to maintain or regain or even improve quality of life [26]. Recently, it has been proposed to use BCIs as an alternative method of cognitive assessment in persons with ALS [27]. Reliable cognitive assessment will improve the way BCIs can be tailored to the needs of the individual.

Healthy users achieved average accuracies above 90% using visual P300 BCI spellers [28]. In a sample of N  =  81 healthy participants eleven percent did not reach accuracies above 80% [29]. In samples of motor impaired persons the accuracies achieved with BCIs often decrease dramatically. In a study with four motor impaired participants, two did not achieve accuracies above 70%, in the second phase one of four [30]. A study by Kübler and Birbaumer showed data from eleven persons with motor impairments (in LIS) that achieved an average accuracy of 66% [23]. Of these eleven, four achieved a level of control considered sufficient for independent use of BCIs (which was not defined as a quantitative but a qualitative measure evaluating the person’s ability to use a BCI for tasks such as communication, web surfing or environmental control). Two more reached accuracies above 70% correct symbol selection, the criterion threshold at which the use of communication aids becomes feasible [31], [32]. Thus in this sample, 45% of the participants were unable to freely communicate with a visual P300 BCI.

Several studies have described physiological and psychological predictors of BCI performance. Hammer et al. showed that the ability to concentrate and visuo-motor coordination predict SMR BCI performance [33]. More recently motor imagery questionnaires have been shown to be a strong predictor of BCI performance [34]. From a physiological perspective, Blankertz et al. have shown that the amplitude of the resting SMR peak correlates strongly with subsequent SMR BCI performance [35]. Using fMRI, Halde et al. were able to demonstrate that SMR BCI users with high aptitude and low aptitude have identical neural activity elicited by motor execution but differ during motor imagery and particularly motor observation [36]. During motor observation the number of activated voxels correlated significantly with BCI-performance (r  =  0.53). Specifically, the number of activated voxels in the right middle frontal gyrus was correlated with BCI-performance (r  =  0.72). This underlines the importance of task monitoring and working memory throughout the BCI session. Structural differences between high and low aptitude users were also shown [37]. A volumetric analysis showed no differences between the groups but structural integrity and myelination quality of deep white matter structures were strong predictors of BCI-performance. Studies with users of SCP BCIs have shown the predictive power of performance in early sessions for later performance [38]. More specifically, the number of sessions needed to achieve significant cursor control correlated moderately with the number of sessions required to achieve criterion level control (above 70%, [39]).

Concerning P300 BCIs, motivation impacts the performance achieved in a subsequent BCI session [28]. The authors described a reduced P300 amplitude for the least motivated participants as opposed to the most motivated participants. A moderate but significant correlation between spectral power in the high alpha and the low beta band during a baseline recording and subsequent P300 BCI performance has also been shown [40]. The data originated from measurements performed over the course of two years with one person with ALS. The authors analyzed a total of 197 runs in which the participant spelled 1200 letters. Using resting state data recorded in a different session than the P300 BCI test sessions, it has been shown that the frequency band of the maximum peak in the power spectrum correlates strongly with visual P300 BCI performance [41]. Additionally, spectral and temporal EEG features correlate with performance during BCI usage [42].

Physiological factors not extracted from the EEG can also influence BCI performance. In a study performed with healthy participants the relationship between heart rate variability and P300 BCI performance was demonstrated [43]. In another study it was shown that an auditory oddball recorded before the P300 BCI session can be used to predict performance in a sample of 40 healthy participants [44]. It has also been demonstrated in one person in the CLIS and two in the LIS that oddball data can be used to predict performance in a classical conditioning BCI [16]. An overview of existing predictors can be found in Table 1. The predictors are grouped according to the categories of the model suggested by Kübler [45].

[Figure omitted. See PDF.]

Table 1. Overview of existing predictors.

https://doi.org/10.1371/journal.pone.0076148.t001

In this paper we propose using an auditory oddball session performed before the P300 BCI session to predict performance. Instead of a sample of healthy participants (see [44]) we apply this method of performance prediction to a sample of severely motor impaired persons. We chose to use an auditory oddball instead of a visual oddball to have a predictor that is independent of the visual system, which becomes compromised e.g. in later stages of ALS. Due to a stronger dependence of the ERP morphology on stimulus discriminability, intensity and probability we do not assume that using the auditory instead of the visual modality will be a disadvantage [46], [47], [48], [49,]. Using a short BCI session instead of the oddball for prediction requires that all instructions are understood by the user in the first session. Any misunderstandings will introduce an unwanted bias in the prediction. Performing the auditory oddball requires little attentional resources and the instructions are easy to understand. For this reason, an auditory oddball as we used in this study has also been used to assess brain function in severly brain injured participants [50]. Thus, we conclude that the auditory oddball represents a robust and generally applicable solution for testing in a sample of severly motor impaired BCI users. We evaluated the success of our method by analyzing which components of the P300 ERP elicited by the auditory oddball task correlated with later BCI performance.

Methods

Participants

Eleven persons with motor impairments (6 male, mean age 54.36 years, standard deviation (SD) 10.89 years, range 36–71 years) participated in three sessions of the P300 BCI performance prediction study (see Table 2). We selected participants based on their willingness to participate and general suitability for EEG studies (no skin irritations, epilepsy) and recruited them through local support groups. There was no financial compensation and all participants were informed that the system could not be provided to them for personal use after the study. The main motivation of participation is to contribute to the development of BCI systems so that others may profit in the future. The level of impairment ranged from zero to 43 (mean 17.7) according to the ALS functional rating scale revised (ALS FRS-R) [51]. This instrument evaluates the ability of the participant to carry out activities of daily living such as speech and handwriting. Lower values indicate a lower level of functionality. The ALS FRS-R questionnaire evaluates twelve items with a score between zero and four. Thus the score can range from zero (maximally disabled) to 48 (not disabled). We informed each participant about the purpose of the study and each participant gave informed consent prior to the experimental session. The participants gave informed consent in the presence of their legal representative. The consent was given using residual movements that had been agreed on before. Written consent was then provided by the legal representatives (if the participants were unable to provide written consent).The Ethical Review Board of the Medical Faculty of the University of Tübingen approved the study and consent procedure.

[Figure omitted. See PDF.]

Table 2. Description of participants with motor impairment.

https://doi.org/10.1371/journal.pone.0076148.t002

Experimental Design

The participants of this study took part in two separate experiments. First, an auditory standard oddball experiment, to predict the performance in the second, a visual P300 BCI task. We provide an overview of the design in Figure 1.

[Figure omitted. See PDF.]

Figure 1. In each session every participant performed an auditory oddball and a visual P300 BCI internet browsing task.

The participants performed three sessions in total. We averaged the results of the web browsing task across the three sessions to determine the performance of each participant. The BCI system provided online feedback. We evaluated performance using the number of errors made by the participant. Features extracted from the auditory oddball session served as a performance measure and we assessed whether we can use them to predict BCI aptitude of participants with severe motor impairment in the visual P300 BCI task.

https://doi.org/10.1371/journal.pone.0076148.g001

Auditory Oddball

The auditory stimuli were comprised of a set of standard tones (duration 160 ms; chords at 517 Hz, 646 Hz and 775 Hz) and deviants (the oddball; duration 160 ms; a 517 Hz tone). The ratio of standards and deviants was 4∶1. Each sequence of stimuli consisted of five tones. One run consisted of 20 sequences. In total we performed three runs resulting in 60 deviant and 240 standard tones (one session). We instructed the participants to count the deviants. The inter stimulus interval (ISI) was 800 ms, resulting in a run length of 96 s and a total length of the experiment of 288 s (4 min 48 s).

The Visual P300 BCI

During the visual P300 BCI experiment participants attended a nested matrix designed to control a web browser [52]. We included the initial matrix as depicted in Figure 2. We will present the detailed results of the web browsing task independently of the performance prediction study and spared the participants the strain of additional conventional P300 speller sessions that would otherwise have been needed. Thus, we used the P300 BCI web browsing sessions for performance prediction. The matrix had an initial dimension of 7×7 (see Figure 2) from which the user can navigate to two different 5×6 matrices (bottom row, “J–Z*” and “0–9,@”). One sequence therefore comprised between 11 (5×6 matrix) and 14 (7×7 matrix) flashes (one for each row and column) of 62.5 ms duration followed by a 125 ms inter-flash interval. We set the number of flashes to a subject specific level based on a preceding measurement with a conventional 6×6 speller matrix. During the training measurement the participants spelled 17 letters without feedback in two separate runs. Each stimulus was presented 15 times. We used these two runs to train the classifier for online feedback with the visual P300 BCI. All participants performed a third run spelling eleven letters with feedback to verify the selected settings.

[Figure omitted. See PDF.]

Figure 2. The visual P300 BCI matrix the participants used to control the web browser.

Participants used letters to select hyperlinks and for text input. The participant selected links on websites with more than 26 links using the letters marked with a “*”. Other functions included moving forward and backward between pages or reloading. The system provided submatrices for numbers and double letters from J–Z*.

https://doi.org/10.1371/journal.pone.0076148.g002

During the web browsing task, the system paused for 8 s after symbol selection in which it performed signal classification and presented the selected letter to the participant. We chose the length of the pause to give the next website enough time to load and the user enough time to select the next action. The participants had to perform a given sequence of tasks using the web browser and make a minimum of 40 correct selections before the tasks were completed. The number of selections needed to complete the task successfully varied between users. We used the number of errors instead of more conventional measures such as accuracy or bitrate because different errors may need different numbers of selections to correct (depending on what command the participant sent to the web browser erroneously). Thus, making a mistake and then needing to perform correcting steps may actually increase accuracy in retrospect. An increase in accuracy occurs if the error needs multiple correct selections to be undone. This can e.g. occur if the user hits enter by mistake when entering a term in a search field. To correct it the user must select the “backwards” function, reselect the search field and complete, correct or possibly re-enter the search term. As a consequence these multiple (ideally) correct selections will increase accuracy even though the total time until the intended goal is completed has also increased. Thus, in our scenario performance (minimum time needed for achieving the pre-defined goal) would have decreased. Then again it is not the aptitude of the user that increases the number of corrective steps needed after a particular mistake which makes using the total time needed for that task another suboptimal measure. This is the reason for choosing the absolute number of errors made by the participant. Each of the eleven participants performed three sessions using the P300 web browser. For all comparisons between low and high aptitude users we split the group at the median. For the correlation analysis, the number of errors was averaged across sessions for each participant.

Data Acquisition

We performed stimulus presentation and data collection with the BCI2000 software [53]. We recorded the EEG using an Ag/AgCl electrode cap with 16 channels (manufactured by EASYCAP GmbH, Herrsching, Germany; F3, Fz, F4, T7, T8, C3, Cz, C4, Cp3, Cp4, P3, Pz, P4, Po7, Po8 and Oz) based on the modified 10–20 system of the American Electroencephalographic Society [54]. The reference was placed on the right and the ground on the left mastoid. The sampling rate was set to 256 Hz with a high pass filter at 0.1 Hz and a low pass filter at 60 Hz (auditory oddball: 30 Hz) using a g.tec 16-channel gUSBamp EEG amplifier (g.tec medical engineering GmbH, Austria).

Offline Processing

During the offline processing we high-pass filtered the data acquired during presentation of the auditory oddball at 0.5 Hz and then low-pass filtered at 20 Hz using a two-way least-squares FIR filtering by a function from the EEGLAB toolbox [55].

To isolate and remove ocular artifacts we employed the blind source separation (BSS) method algorithm for multiple unknown signals extraction (AMUSE) [56], [57]. AMUSE is particularly suited to remove ocular artifacts [58]. To increase external validity we performed no other artifact correction or rejection.

For offline analysis we replaced the right mastoid reference with a common average reference (CAR). The CAR re-references the potential at each electrode with mastoid reference by subtracting the average potential of all electrodes. After segmenting the data into individual epochs (0–800 ms), we baseline corrected by subtracting from every epoch the mean amplitudes in the −100 to 0 ms pre-stimulus interval.

We defined amplitude of the P300 as the local maximum between 250 and 700 ms, the N1 as the local minimum between 100 and 200 ms, the P2 as the local maximum between 200 and 250 ms and the N2 as the local minimum between 250 and 375 ms. Note that using the CAR also influences the topography of the investigated ERP components. For example components which have a frontal maximum absolute value will appear to have a dipolar topography after applying the CAR. Negative components on Fz will appear positive on Oz if the negative amplitude was high enough. Thus we inverted the sign for the analysis of the ERP peaks for electrodes posterior to Cz. This effect must also be taken into consideration when evaluating the topography of the ERPs.

Classification

We used stepwise linear discriminant analysis (SWLDA) for online and offline classification. This algorithm is commonly employed as a classification method for visual and auditory P300 BCIs [3], [30], [59], [60], [61]. The algorithm adds the most significant features to the model first (p<0.1, otherwise the model generation fails). Then the algorithm adds the remaining features to the model in order of their significance. During the backward stepwise regression step the algorithm removes features from the model that do not fulfill a significance level of p<0.15. After 60 iterations or if no further features fulfill the inclusion criterion the model generation stops and the model is applied to the data in the current state.

We smoothed the spatiotemporal features of each trial with a moving average filter, with a width of 25 samples, and then decimated them by a factor of 25 prior to feature selection and classification. For classification we used a time window from 0 to 1000 ms and the full channel set for online feedback.

During online classification we applied the model separately to the trials following row and column flashes. The BCI system displays the row and and the column trials with a maximum score as feedback. This means that the system requires no bias term for classification of the P300 responses. The classifier weights were trained on and applied to single trials for feedback. The classifier output was summed over all trials of a particular stimulus and the row and column with the maximum output selected for feedback.

Statistical Analysis

All r-values were calculated as rank correlations which were performed according to the method described in [62]. For the correlation analysis performance of the participants was normalized to the mean performance across the three sessions (to ensure that all values were independent). Significance was assumed for p<0.05 for all statistical tests (including t-tests). All results of the ERP analysis were given as mean with standard deviation.

Results

Online Performance

Due to the nature of the task (the participants had to accomplish certain goals with a P300 controlled web browser using predefined steps) we give performance in number of errors. The participants made a mean amount of 15.1±13.3 errors (range 0–51 errors) in a task that needed a minimum of 40 selections. We found no correlation between ALS FRS-R and the average error made by each user in the three measurement sessions (r  =  0.29, p  =  0.39).

Group Differences in the Auditory Standard Oddball

Target and standard trials of the auditory oddball were evaluated separately for high and low aptitude users. Figure 3 shows the target/non-target differences of the whole spatio-temporal feature matrix in (A). In Figure 3 (B) we show the differences between the ERPs of high (red) and low (blue) aptitude users on electrodes Fz, Cz, Pz and Oz. For comparison purposes, we also plotted the average response across all participants (green). Finally, the topographies of the target/non-target differences from 200 to 600 ms are shown in 100 ms steps.

[Figure omitted. See PDF.]

Figure 3. The average amplitude across all participants of the response to the auditory oddball is shown as the full spatio-temporal feature matrix (EEG channels are in the sequence given in the data acquisition section) of the target non-target difference (A).

Time course at Fz, Cz, Pz and Oz of the averaged ERP are shown for high aptitude (red), low aptitude (blue) and all users (green) for targets (continuous lines) and non-targets (dashed lines; B). Topographic distribution (the scale is the same as in A) of the target non-target difference at 200, 300, 400, 500 and 600 ms (C).

https://doi.org/10.1371/journal.pone.0076148.g003

The averaged auditory oddball P300 ERP component peaks had an amplitude of 2.08±1.3μV at 361.51±87.1 ms on electrode Cz. High aptitude users had an average amplitude of 1.52±1.3μV and low aptitude users of 2.54±1.3μV. An independent t-test showed that the difference was not significant (t₉  =  −1.3, p  =  0.23). Latencies averaged to 392.19±91.9 ms for high aptitude and to 335.94±81.7 ms for low aptitude users. A independent t-test was not significant (t₉  =  1.08, p  =  0.31).

Correlation between Auditory Oddball Response and Aptitude

Figure 3 shows that the amplitude differences of later ERP components between the aptitude groups were smaller than the amplitude differences of earlier positive and negative ERP components. This was confirmed by the correlation plot of the spatio-temporal feature matrix shown in Figure 4 (A). The highest r² values were in the latency range from 100 to 350 ms. In Figure 4 (B) the upper of the two scatter plots illustrates the positive correlation between errors in the browsing task and amplitude of the auditory oddball P300 on frontal electrodes, whereas the lower scatter plot shows the negative correlation on occipital electrodes. The correlation on PO8 reached an r of −0.93. The topographies in Figure 4 (C) further underline that frontal amplitudes have a positive correlation with the number of errors, whereas occipital electrodes have a negative correlation with the number of errors.

[Figure omitted. See PDF.]

Figure 4. Signed r² values between auditory oddball amplitudes at all time points and channels with visual P300 BCI performance (defined as the number errors during the web browsing task) in red for positive correlations and in blue for negative correlations (A).

Two elements were selected from the matrix (marked by green circles in A) for visualization using scatter plots (B) showing a correlation of r  =  0.9 (p < 0.01) on electrode C3 and a correlation of r  =  −0.9 (p < 0.01) on electrode Oz. Finally, topographic distributions of the signed r² values are shown at the bottom (C). Note that due to the use of “number of errors” as performance measure positive correlations indicate a decrease in performance with increasing amplitude, whereas negative correlations indicate an increase of performance with decreasing amplitude.

https://doi.org/10.1371/journal.pone.0076148.g004

Due to the high number of comparisons performed in the approach shown in Figure 4, we decided also to calculate the correlations of individually calculated peaks on channels Fz, Cz, Pz and Oz. The results are shown in Table 3. Performance correlated strongest with the amplitude of the N2 component at Oz (r  =  −0.86).

[Figure omitted. See PDF.]

Table 3. Mean amplitudes, latencies and correlations thereof with BCI performance are shown for N1 (100 and 200 ms),P2 (200 and 250 ms), N2 (250 and 375 ms) and P300 (250 and 700 ms).

https://doi.org/10.1371/journal.pone.0076148.t003

Discussion

Eleven persons with motor impairments participated in the study using a visual P300 BCI for control of a web browser. It was possible to use features extracted from EEG data recorded during an auditory oddball task for prediction of aptitude. A correlation analysis of the different ERP components showed a strong relationship between early positive and negative potentials around 200 ms with performance. We found this relationship between individual sample points of the whole spatio-temporal feature matrix (up to r  =  0.93) and also between individual peaks (up to r  =  −0.86).

BCI Performance

We defined performance as the number of errors needed to complete a task consisting of 40 selections in a P300 7×7 matrix with two 5×6 submatrices. On average the participants made 15.1 errors until performing all 40 selections correctly. The number of errors did not correlate with the degree of impairment. This is in concordance with other studies that did not show a relationship between the ability to use a BCI and disease progression [24]. For better comparability of the performance of the participants in this study with the performance of other samples of persons with motor impairment, the accuracy was estimated as 40 correct selections out of 55.1 total selections. This results in 73% online accuracy which is above the criterion level of 70% accuracy [31], [39]. We believe that the accuracy achieved by the participants in this study can be compared to the accuracy achieved by ALS or persons with other motor disabilities in previous P300 BCI studies [26], [30], [52], [63], [64], [65]. In fact none of the other studies demanded such a complex task from the participants as performed by the participants in this study. Therefore, these results reflect a robust estimation of the ability to control a BCI. Participants needed to make selections using a 7×7 letter matrix with two 5×6 sub-matrices. Additionally, the participants controlled a web browser on an additional screen (requiring attentional switches between the P300 matrix and the screen displaying the web browser) which changed the displayed content accordingly. This increased the demand on attentional resources of the participants. It is also the largest sample of participants to have participated in any of the aforementioned studies. Using novel stimulation techniques such as famous faces instead of flashing rows and columns information transfer rate (ITR) can be increased substantially than what we report in this study [66].

Comparison of ERPs

ERP differences between healthy participants and participants with severe motor impairment have been explored in several studies. The general conclusion is that besides motor impairment there appears to be an effect of the disease on attention and working memory. In some studies this loss of cognitive funcitons has been linked to the severity of the disease [67]. Other studies found that ALS may be, but must not be linked to frontotemporal dementia (FTD). In samples of persons with FTD the occurence of ALS is much higher than can be expected in a random sample [68]. This has been attributed to the observation that the gene defective in familial ALS is sometimes linked to the gene causing FTD [69], [70]. Thus, the two diseases occur more frequently in the same person than can be expected by chance alone. A summary of cognitive effects associated with ALS can be found in Raaphorst et al. [71]. This is reflected in electrophysiological studies. In an auditory selective attention task with eight persons with ALS, ERPs were reduced compared to age-matched healthy controls [72], [73]. It was also shown that with the progression of impairment auditory and visual P300 latency was increased [74]. Cognitive impairments and decreased P3a/P3b amplitudes as well as higher P3a latencies were also shown [75], [76]. An important factor that may contribute to ERP abnormalities in persons with ALS are periodic failures in the ventilation system that may lead to anoxia [77]. Analysis of intensive care unit patients, especially those receiving artificial ventilation, have revealed significant impairment of cognitive functions [78]. Recent data suggests that abnormalities tend to increase, e.g. in P300 latency, with the disease duration [79], [80]. Compared a sample of healthy participants [44], amplitudes were decreased and latencies increased in the sample of persons with motor impairments in this study.

We found that the minimum amplitude on Cz between 250 and 375 ms indicates fewer errors in the internet browsing task performed by the participants of this study. On Pz the polarity switches (due to the CAR) and increased amplitudes indicate fewer errors. The N2 has been described as more than a sensory component and its involvement in cognitive control processes such as response inhibition has been underlined [81]. This indicates a role of selective attention and arousal in BCI tasks when the participants ignore non-targets and respond to the target stimuli which is reflected in variations in N2 amplitude. We also found that the latency increases with the number of errors on frontal EEG channels.

Amplitude and latency of the P2 have a weaker correlation than amplitude and latency of the N2 [82]. Indeed, we did not find correlations between latency and performance. There are no P2 amplitude differences between targets and non-targets for the low aptitude users. In particular, on channel Cz there was no difference between the P2 elicited by targets and non-targets whereas the averaged target and non-target curves of the high aptitude users diverge after the N1. Our data shows a lower P2 amplitude on frontal channels and higher amplitude on occipital channels for high aptitude users. On all channels the absolute values of the P2 amplitude were higher for low aptitude users.

In fact, the P2 amplitude appears to decrease with increased attention (for a review of the P2 see Crowley and Colrain [83]). An enhancement of the P2 amplitude to unattended stimuli was found by using EEG and MEG [84], [85]. In both studies this attenuation of the P2 in the attend condition coincided with an enhancement of the N1 (which has higher absolute amplitudes in the high aptitude group in this study). The authors explained this with two different stages of selective attention. The first being inhibition, which is visible as the increased N1, and the second being filtering, which is accompanied by a decreased P2 in the EEG. Under this assumption high aptitude users exhibit increased attention and better filtering mechanisms in the target condition compared to the non-target condition and also compared to low aptitude users.

Conclusions

A short auditory oddball experiment can be used to predict performance in a sample of persons with severe motor impairment. This was shown previously in a sample of healthy participants and also with a person in CLIS [16], [44]. It is probable that a single variable that predicts performance will not be found, due to the many factors that influence performance. As proposed by Kübler [45], these factors include physiological, anatomical and psychological variables (besides technical variables which are beyond the scope of this paper). Physiological variables such as heart rate variability (HRV) [43], the amplitude of the SMR-peak [35], amplitudes of the ERPs used to control the BCI [42] and the volume of the brain-areas recruited during motor imagery in particular in pre- and supplementary motor areas [36] influence performance. Additionally the anatomy of a BCI user may be affected by lesions or neurodegeneration [72] which influences the EEG and consequently BCI performance. Even in healthy users differences in brain-connectivity can predict variations in BCI performance [37]. Finally, psychological factors such as mood and motivation Kleih:2010fk and visuo-motor coordination [33] have an added impact on the ability to control a BCI. The factors which we found in this study are clearly physiological of origin but are probably a measure of a psychological variable: increased attention and better filtering mechanisms (which seem to be persistent throughout a BCI session). Eventually, all predictors should be combined and redundancies removed. Clearly some predictors cannot be applied in a feasible manner with every BCI user, e.g. the predictors requiring magnetic resonance imaging (MRI) measurements. Nonetheless, they provide valuable input to the overall model and point to negative influences on BCI performance that may be removed without the need for continuous MRI measurements.

In combination with future investigations of BCI performance these methods can be used to select the optimal paradigm for persons with motor impairments [86]. Additional improvements will be possible using EEG features to monitor BCI performance during usage. Findings as in this paper may be used as a control variable to specifically train the variables in question (such as attention). This will increase the chance that early BCI sessions are successful and not frustrating experiences.

Author Contributions

Conceived and designed the experiments: SH CR MB WR NB TM. Performed the experiments: SH EP DM LH. Analyzed the data: SH AF DM. Contributed reagents/materials/analysis tools: SH AF DM. Wrote the paper: SH CR NB AK TM.

Citation: Halder S, Ruf CA, Furdea A, Pasqualotto E, De Massari D, van der Heiden L, et al. (2013) Prediction of P300 BCI Aptitude in Severe Motor Impairment. PLoS ONE 8(10): e76148. https://doi.org/10.1371/journal.pone.0076148

References

1. Vidal JJ (1973) Toward direct brain-computer communication. Annu Rev Biophys Bioeng 2: 157–80.

2. Birbaumer N, Ghanayim N, Hinterberger T, Iversen I, Kotchoubey B, et al. (1999) A spelling device for the paralysed. Nature 398: 297–8.

3. Farwell L, Donchin E (1988) Talking off the top of your head: toward a mental prosthesis utilizing even-related brain potentials. Electroencephalogr Clin Neurophysiol 70: 510–523.

4. Spüler M, Rosenstiel W, Bogdan M (2012) Online adaptation of a c-VEP brain-computer interface (BCI) based on error-related potentials and unsupervised learning. PLoS One 7: e51077.

5. Pfurtscheller G, Lopes da Silva FH (1999) Event-related EEG/MEG synchronization and desynchronization: basic principles. Clin Neurophysiol 110: 1842–57.

6. Sitaram R, Caria A, Veit R, Gaber T, Rota G, et al.. (2007) FMRI brain-computer interface: a tool for neuroscientific research and treatment. Comput Intell Neurosci : 25487.

7. Sorger B, Dahmen B, Reithler J, Gosseries O, Maudoux A, et al. (2009) Another kind of ‘BOLD response’: answering multiple-choice questions via online decoded single-trial brain signals. Prog Brain Res 177: 275–92.

8. Monti MM, Vanhaudenhuyse A, Coleman MR, Boly M, Pickard JD, et al. (2010) Willful modulation of brain activity in disorders of consciousness. N Engl J Med 362: 579–89.

9. Hochberg LR, Serruya MD, Friehs GM, Mukand JA, Saleh M, et al. (2006) Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature 442: 164–71.

10. Pistohl T, Ball T, Schulze-Bonhage A, Aertsen A, Mehring C (2008) Prediction of arm movement trajectories from ECoG-recordings in humans. J Neurosci Methods 167: 105–14.

11. Mellinger J, Schalk G, Braun C, Preissl H, Rosenstiel W, et al. (2007) An MEG-based brain-computer interface (BCI). Neuroimage 36: 581–93.

12. Wriessnegger SC, Kurzmann J, Neuper C (2008) Spatio-temporal differences in brain oxygenation between movement execution and imagery: a multichannel near-infrared spectroscopy study. Int J Psychophysiol 67: 54–63.

13. Furdea A, Ruf CA, Halder S, De Massari D, Bogdan M, et al. (2012) A new (semantic) reflexive brain-computer interface: in search for a suitable classifier. J Neurosci Methods 203: 233–40.

14. Ruf CA, De Massari D, Furdea A, Matuz T, Fioravanti C, et al.. (2013) Semantic classical conditioning and brain-computer interface (BCI) control: Encoding of affirmative and negative thinking. Frontiers in Neuroscience 7.

15. De Massari D, Matuz T, Furdea A, Ruf CA, Halder S, et al. (2013) Brain-computer interface and semantic classical conditioning of communication in paralysis. Biol Psychol 92: 267–74.

16. De Massari D, Ruf CA, Furdea A, Matuz T, van der Heiden L, et al.. (2013) Brain communication in the locked-in state. Brain.

17. Pfurtscheller G, Allison BZ, Brunner C, Bauernfeind G, Solis-Escalante T, et al. (2010) The hybrid BCI. Front Neurosci 4: 30.

18. van Gerven M, Farquhar J, Schaefer R, Vlek R, Geuze J, et al. (2009) The brain-computer interface cycle. J Neural Eng 6: 041001.

19. Blankertz B, Lemm S, Treder M, Haufe S, Müller KR (2011) Single-trial analysis and classification of ERP components–a tutorial. Neuroimage 56: 814–25.

20. Brouwer AM, van Erp JBF (2010) A tactile P300 brain-computer interface. Front Neurosci 4: 19.

21. Townsend G, LaPallo BK, Boulay CB, Krusienski DJ, Frye GE, et al. (2010) A novel P300-based brain-computer interface stimulus presentation paradigm: moving beyond rows and columns. Clin Neurophysiol 121: 1109–20.

22. Lotte F, Congedo M, Lécuyer A, Lamarche F, Arnaldi B (2007) A review of classification algorithms for EEG-based brain-computer interfaces. J Neural Eng 4: R1–R13.

23. Kübler A, Birbaumer N (2008) Brain-computer interfaces and communication in paralysis: extinction of goal directed thinking in completely paralysed patients? Clin Neurophysiol 119: 2658–2666.

24. Silvoni S, Cavinato M, Volpato C, Ruf CA, Birbaumer N, et al.. (2013) Amyotrophic lateral sclerosis progression and stability of brain-computer interface communication. Amyotroph Lateral Scler Frontotemporal Degener.

25. Murguialday AR, Hill J, Bensch M, Martens S, Halder S, et al. (2011) Transition from the locked in to the completely locked-in state: a physiological analysis. Clin Neurophysiol 122: 925–33.

26. Zickler C, Riccio A, Leotta F, Hillian-Tress S, Halder S, et al. (2011) A brain-computer interface as input channel for a standard assistive technology software. Clin EEG Neurosci 42: 236–44.

27. Cipresso P, Carelli L, Solca F, Meazzi D, Meriggi P, et al. (2012) The use of P300-based BCIs in amyotrophic lateral sclerosis: from augmentative and alternative communication to cognitive assessment. Brain Behav 2: 479–98.

28. Kleih SC, Nijboer F, Halder S, Kübler A (2010) Motivation modulates the P300 amplitude during brain-computer interface use. Clin Neurophysiol 121: 1023–31.

29. Guger C, Daban S, Sellers E, Holzner C, Krausz G, et al. (2009) How many people are able to control a P300-based brain-computer interface (BCI)? Neurosci Lett 462: 94–8.

30. Nijboer F, Sellers EW, Mellinger J, Jordan MA, Matuz T, et al. (2008) A P300-based brain-computer interface for people with amyotrophic lateral sclerosis. Clin Neurophysiol 119: 1909–16.

31. Kübler A, Neumann N, Kaiser J, Kotchoubey B, Hinterberger T, et al. (2001) Brain-computer communication: self-regulation of slow cortical potentials for verbal communication. Arch Phys Med Rehabil 82: 1533–9.

32. Choularton S, Dale R (2004) User responses to speech recognition errors: consistency of behaviour across domains. In: Proceedings of SST. volume 4.

33. Hammer EM, Halder S, Blankertz B, Sannelli C, Dickhaus T, et al. (2012) Psychological predictors of SMR-BCI performance. Biol Psychol 89: 80–6.

34. Vuckovic A, Osuagwu BA (2013) Using a motor imagery questionnaire to estimate the performance of a brain-computer interface based on object oriented motor imagery. Clin Neurophysiol 124: 1586–95.

35. Blankertz B, Sannelli C, Halder S, Hammer EM, Kübler A, et al. (2010) Neurophysiological predictor of SMR-based BCI performance. Neuroimage 51: 1303–9.

36. Halder S, Agorastos D, Veit R, Hammer EM, Lee S, et al. (2011) Neural mechanisms of braincomputer interface control. Neuroimage 55: 1779–90.

37. Halder S, Varkuti B, Bogdan M, Kübler A, Rosenstiel W, et al. (2013) Prediction of brain-computer interface aptitude from individual brain structure. Front Hum Neurosci 7: 105.

38. Neumann N, Birbaumer N (2003) Predictors of successful self control during brain-computer communication. J Neurol Neurosurg Psychiatry 74: 1117–21.

39. Kübler A, Neumann N, Wilhelm B, Hinterberger T, Birbaumer N (2004) Brain-computer predictability of brain-computer communication. J Psychophysiol 18: 121–129.

40. Halder S, Hill J, Nijboer F, Furdea A, Kübler A (2008) Session-to-session P300 BCI performance correlation with baseline frequency spectra for a user with ALS. Proceedings of the 4th International Brain-Computer Interface Workshop and Training Course : 8–13.

41. Halder S, Spühler M, Hammer E, Kleih S, Bogdan M, et al.. (2011) Prediction of visual P300 BCI aptitude using spectral features. Proceedings of the 5th International Brain-Computer InterfaceWorkshop and Training Course : 144–145.

42. Mak JN, McFarland DJ, Vaughan TM, McCane LM, Tsui PZ, et al. (2012) EEG correlates of P300-based brain-computer interface (BCI) performance in people with amyotrophic lateral sclerosis. J Neural Eng 9: 026014.

43. Kaufmann T, Vögele C, Sütterlin S, Lukito S, Kübler A (2012) Effects of resting heart rate variability on performance in the P300 brain-computer interface. Int J Psychophysiol 83: 336–41.

44. Halder S, Hammer EM, Kleih SC, Bogdan M, Rosenstiel W, et al. (2013) Prediction of auditory and visual P300 brain-computer interface aptitude. PLoS ONE 8: e53513.

45. Kübler A, Blankertz B, Müller KR, Neuper C (2011) A model of BCI control. In: Müller-Putz GR, Scherer R, Billinger M, Kreilinger A, Kaiser V, et al., editors, Proceedings of the 5th International Brain-Computer Interface Conference. 100–103.

46. Squires NK, Donchin E, Squires KC, Grossberg S (1977) Bisensory stimulation: inferring decision-related processes from P300 component. J Exp Psychol Hum Percept Perform 3: 299–315.

47. Polich J, Ellerson PC, Cohen J (1996) P300, stimulus intensity, modality, and probability. Int J Psychophysiol 23: 55–62.

48. Comerchero MD, Polich J (1999) P3a and P3b from typical auditory and visual stimuli. Clin Neurophysiol 110: 24–30.

49. Katayama J, Polich J (1999) Auditory and visual P300 topography from a 3 stimulus paradigm. Clin Neurophysiol 110: 463–8.

50. Kotchoubey B, Lang S, Baales R, Herb E, Maurer P, et al. (2001) Brain potentials in human patients with extremely severe diffuse brain damage. Neurosci Lett 301: 37–40.

51. Cedarbaum JM, Stambler N, Malta E, Fuller C, Hilt D, et al. (1999) The ALSFRS-R: a revised ALS functional rating scale that incorporates assessments of respiratory function. BDNF ALS study group (phase III). J Neurol Sci 169: 13–21.

52. Mugler EM, Ruf CA, Halder S, Bensch M, Kübler A (2010) Design and implementation of a P300-based brain-computer interface for controlling an internet browser. IEEE Trans Neural Syst Rehabil Eng 18: 599–609.

53. Schalk G, McFarland D, Hinterberger T, Birbaumer N, Wolpaw J (2004) BCI2000: a general-purpose brain-computer interface (BCI) system. IEEE Trans Biomed Eng 51: 1034–1043.

54. Sharbrough F (1991) American electroencephalographic society guidelines for standard electrode position nomenclature. J Clin Neurophysiol 8: 200–202.

55. Delorme A, Makeig S (2004) EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods 134: 9–21.

56. Tong L, Liu RW, Soon V, Huang YF (1991) Indeterminacy and identifiability of blind identification. IEEE Transactions on circuits and systems 38: 499–509.

57. Cichocki A, Shishkin S, Musha T, Leonowicz Z, Asada T, et al. (2005) EEG filtering based on blind source separation (BSS) for early detection of Alzheimer’s disease. Clin Neurophysiol 116: 729–737.

58. Halder S, Bensch M, Mellinger J, Bogdan M, Kübler A, et al.. (2007) Online artifact removal for brain-computer interfaces using support vector machines and blind source separation. Comput Intell Neurosci : 82069.

59. Krusienski D, Sellers E, Cabestaing F, Bayoudh S, McFarland D, et al. (2006) A comparison of classification techniques for the P300 speller. J Neural Eng 3: 299–305.

60. Krusienski DJ, Sellers EW, McFarland DJ, Vaughan TM, Wolpaw JR (2008) Toward enhanced P300 speller performance. J Neurosci Methods 167: 15–21.

61. Furdea A, Halder S, Krusienski DJ, Bross D, Nijboer F, et al. (2009) An auditory oddball (P300) spelling system for brain-computer interfaces. Psychophysiology 46: 617–25.

62. Spearman C (1987) The proof and measurement of association between two things. By C. Spearman, 1904. Am J Psychol 100: 441–71.

63. Sellers E, Donchin E (2006) A P300-based brain-computer interface: initial tests by ALS patients. Clin Neurophysiol 117: 538–548.

64. Piccione F, Giorgi F, Tonin P, Priftis K, Giove S, et al. (2006) P300-based brain computer interface: reliability and performance in healthy and paralysed participants. Clin Neurophysiol 117: 531–537.

65. Hoffmann U, Vesin JM, Ebrahimi T, Diserens K (2008) An efficient P300-based brain-computer interface for disabled subjects. J Neurosci Methods 167: 115–125.

66. Kaufmann T, Schulz SM, Köblitz A, Renner G, Wessig C, et al. (2013) Face stimuli effectively prevent brain-computer interface inefficiency in patients with neurodegenerative disease. Clin Neurophysiol 124: 893–900.

67. Abe K, Fujimura H, Toyooka K, Sakoda S, Yorifuji S, et al. (1997) Cognitive function in amyotrophic lateral sclerosis. J Neurol Sci 148: 95–100.

68. Lomen-Hoerth C, Anderson T, Miller B (2002) The overlap of amyotrophic lateral sclerosis and frontotemporal dementia. Neurology 59: 1077–9.

69. Hosler BA, Siddique T, Sapp PC, Sailor W, Huang MC, et al. (2000) Linkage of familial amyotrophic lateral sclerosis with frontotemporal dementia to chromosome 9q21-q22. JAMA 284: 1664–9.

70. Morita M, Al-Chalabi A, Andersen PM, Hosler B, Sapp P, et al. (2006) A locus on chromosome 9p confers susceptibility to als and frontotemporal dementia. Neurology 66: 839–44.

71. Raaphorst J, de Visser M, Linssen WHJP, de Haan RJ, Schmand B (2010) The cognitive profile of amyotrophic lateral sclerosis: A meta-analysis. Amyotroph Lateral Scler 11: 27–37.

72. Vieregge P, Wauschkuhn B, Heberlein I, Hagenah J, Verleger R (1999) Selective attention is impaired in amyotrophic lateral sclerosis–a study of event-related EEG potentials. Brain Res Cogn Brain Res 8: 27–35.

73. Pinkhardt EH, Jürgens R, Becker W, Mölle M, Born J, et al. (2008) Signs of impaired selective attention in patients with amyotrophic lateral sclerosis. J Neurol 255: 532–8.

74. Paulus KS, Magnano I, Piras MR, Solinas MA, Solinas G, et al. (2002) Visual and auditory eventrelated potentials in sporadic amyotrophic lateral sclerosis. Clin Neurophysiol 113: 853–61.

75. Hanagasi HA, Gurvit IH, Ermutlu N, Kaptanoglu G, Karamursel S, et al. (2002) Cognitive impairment in amyotrophic lateral sclerosis: evidence from neuropsychological investigation and event-related potentials. Brain Res Cogn Brain Res 14: 234–44.

76. Ogawa T, Tanaka H, Hirata K (2009) Cognitive deficits in amyotrophic lateral sclerosis evaluated by event-related potentials. Clin Neurophysiol 120: 659–64.

77. Kotchoubey B, Lang S, Winter S, Birbaumer N (2003) Cognitive processing in completely paralyzed patients with amyotrophic lateral sclerosis. Eur J Neurol 10: 551–8.

78. Hopkins RO, Brett S (2005) Chronic neurocognitive effects of critical illness. Curr Opin Crit Care 11: 369–75.

79. Raggi A, Consonni M, Iannaccone S, Perani D, Zamboni M, et al. (2008) Auditory event-related potentials in non-demented patients with sporadic amyotrophic lateral sclerosis. Clin Neurophysiol 119: 342–50.

80. Volpato C, Piccione F, Silvoni S, Cavinato M, Palmieri A, et al. (2010) Working memory in amyotrophic lateral sclerosis: auditory event-related potentials and neuropsychological evidence. J Clin Neurophysiol 27: 198–206.

81. Folstein JR, Van Petten C (2008) Influence of cognitive control and mismatch on the n2 component of the erp: a review. Psychophysiology 45: 152–70.

82. Polich J, Alecander J, Bauer L, Kuperman S, Rohrbaugh J, et al. (1997) P300 topography of amplitude/latency correlations. Brain Topogr 9: 275–282.

83. Crowley KE, Colrain IM (2004) A review of the evidence for p2 being an independent component process: age, sleep and modality. Clin Neurophysiol 115: 732–44.

84. Rif J, Hari R, Hämäläinen MS, Sams M (1991) Auditory attention affects two different areas in the human supratemporal cortex. Electroencephalogr Clin Neurophysiol 79: 464–72.

85. Michie PT, Solowij N, Crawford JM, Glue LC (1993) The effects of between-source discriminability on attended and unattended auditory erps. Psychophysiology 30: 205–20.

86. Kaufmann T, Holz EM, Kübler A (2013) Comparison of tactile, auditory and visual modality for brain-computer interface use: A case study with a patient in the locked-in state. Frontiers in Neuroscience.

87. Burde W, Blankertz B (2006) Is the locus of control of reinforcement a predictor of brain-computer interface performance? In: Proceedings of the 3rd International Brain-Computer Interface Workshop and Training Course. volume 2006, 108–109.