Parallel convolutional neural network and empirical mode decomposition for high accuracy in motor imagery EEG signal classification

Abstract

In recent years, the utilization of motor imagery (MI) signals derived from electroencephalography (EEG) has shown promising applications in controlling various devices such as wheelchairs, assistive technologies, and driverless vehicles. However, decoding EEG signals poses significant challenges due to their complexity, dynamic nature, and low signal-to-noise ratio (SNR). Traditional EEG pattern recognition algorithms typically involve two key steps: feature extraction and feature classification, both crucial for accurate operation. In this work, we propose a novel method that addresses these challenges by employing empirical mode decomposition (EMD) for feature extraction and a parallel convolutional neural network (PCNN) for feature classification. This approach aims to mitigate non-stationary issues, improve performance speed, and enhance classification accuracy. We validate the effectiveness of our proposed method using datasets from the BCI competition IV, specifically datasets 2a and 2b, which contain motor imagery EEG signals. Our method focuses on identifying two- and four-class motor imagery EEG signal classifications. Additionally, we introduce a transfer learning technique to fine-tune the model for individual subjects, leveraging important features extracted from a group dataset. Our results demonstrate that the proposed EMD-PCNN method outperforms existing approaches in terms of classification accuracy. We conduct both qualitative and quantitative analyses to evaluate our method. Qualitatively, we employ confusion matrices and various performance metrics such as specificity, sensitivity, precision, accuracy, recall, and f1-score. Quantitatively, we compare the classification accuracies of our method with those of existing approaches. Our findings highlight the superiority of the proposed EMD-PCNN method in accurately classifying motor imagery EEG signals. The enhanced performance and robustness of our method underscore its potential for broader applicability in real-world scenarios.

1. Introduction

The bioelectric information contained in electroencephalography (EEG) signals reflects the physiological processes taking place in the human brain [1–4]. They are frequently used with brain-computer interfaces (BCIs), which turn recorded brain activity into useful information to connect with outside surroundings. The brain-computer interface (BCI) is an innovative non-muscular communication system that connects directly from human intent to external equipment like computers, robots, unmanned aerial vehicles, and so on. BCI can assist individuals in directly controlling BCI-based robots to do a variety of activities through thought and has a very broad range of potential applications in daily life [5–7].

Numerous research projects use multiple electroencephalography (EEG) signal types, such as event-related potential (ERP), steady-state visually evoked potential (SSVEP), and motor imagery (MI). The event-related potential (ERP), known as P300 evoked potentials, demonstrates how sensitively a subject respond to stimuli [8–10]. By locating the SSVEP frequencies in the EEG data, steady state visually evoked potential (SSVEP), which has been employed in various applications including controlling the grasping of the robotic hand and evaluating adult visual acuity, can recognize a set of characters when visual stimulation is present. The feasibility of hybrid SSVEP + P300 visual BCI systems for quad-copter flight control, aiming to overcome limitations in existing BCI-based controls. The system integrates SSVEP and P300 potentials for enhanced control commands, facilitated by a GUI allowing users to control flight direction by gazing at visual stimuli buttons. Comparative analysis with conventional BCI and keyboard flight control systems demonstrates superior performance, though slightly lower than commercial keyboard controls. Importantly, the proposed system shows promise for aiding individuals with severe motor disabilities [11]. Additionally, motor imagery (MI) is another EEG signal that is frequently used. By converting MI frequencies into distinct commands, MI signals can control robot motion. EEG signals of two or more different types may merge to form a full signal.

One sort of EEG signal from the sensorimotor cortex is called the MI when a researcher imagines moving a body component without actually moving it. The physical parts of fictitious tasks could be the left and right hand, tongue, foot, fingers, shoulder, elbows, and so forth [12]. Thus, a crucial and essential issue in BCI research is the classification of various MI signal types. Brain activity, known as motor imagery (EEG), is observed when a participant intends to move limbs like their hands or feet. In response to these picturing or thinking tasks, the MI EEG signal is generated in the sensorimotor cortex portion of the brain. Researchers have used these MI signals to distinguish between various oscillatory brain activations for various tasks [13]. Utilizing several machine learning and deep learning methods, automated MI categorization has been carried out.

The motor-imagery brain state can be classified using the widely used modality of EEG, which uses a signal recording approach. The following are the challenges encountered during the classification of the motor imagery EEG signal. Although there are a lot of motor imagery tasks, training different binary classifiers takes a lot of time. Deep models are shown to require extensive training data and neural network attribute adjustments. During classification, the data loss is very high. For better performance, finding robust features is a challenging task, and, in most cases, there will be a higher failure rate. These difficulties with the traditional methods serve as motivation for developing a new way of categorizing the EEG data. This paper aims to develop a novel method to improve the classification of motor imagery EEG signals, and its contribution has been discussed below. The classification of MI EEG signals aims to improve the performance and accuracy of BCIs by effectively distinguishing between different motor imagery tasks. This would enable more reliable and efficient communication and control of external devices through brain activity. The classification of MI EEG signals contributes to advancements in EEG data processing and analysis techniques. Developing robust and efficient classification algorithms enhances the capabilities of neuroimaging technologies, paving the way for more sophisticated and accessible tools in clinical, research, and BCI applications.

The weakness and the limitations of the proposed work is the 1. Limited Robustness to Noise and Variability—Despite achieving high classification accuracy on the BCI competition IV datasets, the method’s robustness to noise and variability in real-world EEG recordings may be limited. EEG signals are inherently noisy, and variations in electrode placement, signal quality, and participant characteristics can affect classification performance. Future research could explore techniques to enhance the model’s robustness to such variability, such as data augmentation, denoising methods, or adaptive learning approaches -. 2. Transfer Learning Dependency The reliance on transfer learning to adapt the model to individual-specific datasets may introduce biases or limitations. Transfer learning assumes that features extracted from a group dataset are transferable to individual datasets, which may not always hold true due to inter-subject variability. Investigating alternative approaches or strategies to mitigate the need for transfer learning could improve the model’s generalizability and applicability. 3. Computational Complexity—The computational complexity associated with empirical mode decomposition and parallel convolutional neural networks may limit the model’s scalability and real-time applicability, especially in resource-constrained environments or applications requiring low-latency responses. Optimizing the model architecture, exploring light weight alternatives, or leveraging hardware acceleration techniques could help address computational challenges and improve efficiency.

The rest of this paper is organized as follows: In Section 2, the related study of feature extraction and classification techniques used in motor imagery EEG signal analysis is discussed. In Section 3, the dataset description used in the proposed work, the details of the features of the EEG signal, and the basics of the empirical mode decomposition method and convolutional neural network algorithm for the EEG signal representation are described. The description of EMD-PCNN is proposed in Section 4. In Section 5, the classification results and the evaluation parameters are presented. Finally, we draw a conclusion in Section 6. The overall block diagram of the proposed work is illustrated in Fig 1.

[Figure omitted. See PDF.]

2. Related study

The categorization of EEG signals generally involves four steps. The first phase is signal recording, where the EEG signals are recorded from willing individuals using the 10–20 international system, which is used as the accepted benchmark for scalp electrode location for EEG. The 10–20 international system of EEG electrode placement is illustrated in Fig 2.

[Figure omitted. See PDF.]

Second, when the needed experimental sessions are completed by the subjects, the multichannel raw EEG data have a very low signal-to-noise ratio (SNR), requiring signal pre-processing to reduce the noise. Following pre-processing, the next stage is feature extraction, which is the process of extracting the usable features from the signals. Finally, classification is performed using these retrieved features, which is the last step. The methods available for feature extraction and classification are discussed in sections 2.1 and 2.2.

2.1 Feature extraction

EEG signals exhibit time-varying sensitivity, non-stationarity, and non-linearity. Numerous studies have identified mu and beta frequency bands as characteristics of motor imagery EEG signals because both real motions and imagined movements can cause changes in mu and beta rhythm. For example, the extraction of EEG-MI features makes extensive use of frequency decomposition techniques like the transform techniques DFT and FFT.

Feature extraction strategies were split into basic and advanced categories by Swati et al. [14]. The two fundamental methods were time domain and frequency domain, with frequency domain elements being more crucial for the detection of motor imagery. The two most sophisticated methods were space-time-frequency domain technology and time-frequency domain technology. Combining the features of the time and frequency domains, or even adding spatial properties, might enhance recognition ability. In addition to that, various wavelet techniques are also available for motor imagery EEG signals, such as discrete wavelet transform, wavelet packet transform, dual tree complex wavelet transform, etc. A well-known technique for extracting characteristics for motor imagery recognition is the common spatial pattern (CSP). The CSP approach was used to translate the multi-channel brain signals for particular activities converted into low-dimensional space for EEG categorization; such activities are the movements of the right and left hands [15].

2.2 Classification

There are various traditional algorithms available for the classification of motor imagery EEG signals. Among them, linear discriminant analysis and support vector machines are the most popular [16]. Additionally, the EEG-MI research employs the k-nearest neighbour, sparse Bayesian classifier, and fisher linear discriminant [17]. Furthermore, the neural network can adaptively extract data from the EEG and may integrate extracted features and pattern recognition into an entire model. Support vector machine (SVM) [18–20] and linear discriminant analysis (LDA) [21–25] have been the two most used classifiers in the field of EEG recognition during the last ten years. Nevertheless, one of the primary issues with traditional research was the low classification rate, which was caused by the signals’ low signal-to-noise ratio (SNR). Neural networks [26–28] and deep neural networks [29–33] are the classification algorithms that have been applied to the motor imaging brain computer interface.

Previous studies showed that academics from all sectors have embraced deep learning, particularly convoluted neural networks, which have been widely deployed in artificial intelligence, speech recognition, and other fields. However, research was still proceeding on CNN’s potential applications in the discipline of brain-computer interface. A deep convolution neural network model known as EEGNET was developed by Nayak et al. [34] and is capable of reliably differentiating EEG data from various BCI tasks. This result demonstrates that the EEGNET not only exhibited good cross-paradigm generalization capacity and strong robustness but also produced the same results as the most effective conventional EEG classification method, FBCSP. In order to train CNN for classification, Zapata et al. retrieved the original signal’s power spectral density (PSD) features [35].

Although the derived features only contain the EEG frequency domain characteristics, ignoring the time domain elements of EEG signals, the recognition rate was still greater than with the old method. The short-time Fourier transform (STFT) was used to turn the raw signals into time-frequency graphs, and the CNN was trained using the combined images from the C3 and C4 channels [36]. Spiking neural networks (SNNs) were introduced to the EEG classification challenge by Virgilio et al. [37], who were able to reach an accuracy of 0.83 for the binary classification of MI tasks. Third-generation artificial neural networks, or SNNs, are seen to have a significant chance of replacing second-generation ANNs like CNN [38].

The comparison of traditional ML algorithms with Deep Learning methods, focusing on Multi-Layered Perceptron (MLP) for Motor Imagery classification in BCIs. While Support Vector Machine (SVM) exhibits fast training and prediction, MLP achieves ≈90% accuracy in subject-independent classification with half the time, indicating its potential as a foundational framework for advanced BCI optimization. Further refinement could lead to highly accurate and robust subject-independent BCIs [39]. A multi-scale convolutional neural network (MS-CNN) for EEG-based motor imagery (MI) classification in brain-machine interfaces (BMIs), addressing challenges like inter-subject variability and low signal-to-noise ratio (SNR). By integrating discriminant user-specific features and employing data augmentation techniques, the model achieves superior classification accuracy and Cohen’s kappa-coefficient on the BCI competition IV2b dataset, outperforming existing EEG-based MI classification models. This framework holds promise for enhancing real-time human-robot interaction through efficient and robust EEG signal decoding [40].

A transfer learning-based multi-scale feature fused CNN (MSFFCNN) to address challenges in EEG-based motor imagery (MI) classification for brain-computer interfaces (BCIs). By offering four model variants including subject-independent and subject-adaptive configurations, the approach achieves high classification accuracy (94.06%) and kappa value (0.88) surpassing existing MI classification models, even with fewer training samples. The proposed framework presents an efficient solution for building robust MI-BCI systems, demonstrating the effectiveness of adaptive transfer learning in enhancing classification performance [41].

3. Materials and methods

3.1 Dataset description

We use two distinct motor imagery EEG datasets to assess our proposed algorithm in this research work to accurately measure its effectiveness. The size, number of channels, kind of motor imagery, and subject composition of these two datasets are very different from one another. The BCI Competition IV dataset 2a and the BCI Competition IV dataset 2b are publicly available datasets that contain motor imagery and EEG signal data and have been utilized by numerous researchers. We can thus easily compare our model with the findings of several other similar articles by using this dataset.

3.1.1 BCI competition IV dataset 2a.

A 22-channel motor imaging dataset was gathered for the BCI Competition IV and is called BCI Competition IV dataset 2a. Nine healthy individuals provided the four types of left hand, right hand, foot, and tongue imagery motions that make up this dataset. Consequently, the task is a four-class motor imagery EEG decoding challenge. This competition’s organizers offered a dataset that was band-pass filtered between 0.5 and 100 Hz and sampled at 250 Hz. A third-order Butterworth filter was used to band-pass filter the EEG signal between 4 and 38 Hz. For uniformity, an exponentially shifted panning window with a decay factor of 0.999 was applied to each electrode signal. The signals were then recorded, per previous study results, between 0.5 and 2.5 seconds after the initial occurrence of the motor imagery. The subjects completed the motor imagery task until the fixation cross disappeared from the screen at t = 6 s. No feedback was given. The timing diagram of the BCI competition IV dataset 2a has been depicted in Fig 3. The CNN input, or individual trial, in this instance has the form of (1, 22, 500). For the training and test sets, it appears that each individual ultimately produced 288 sets of EEG trials. In order to execute an early end during the first phase of the optimization process, the training set had been further separated into a validation set.

[Figure omitted. See PDF.]

3.1.2 BCI competition IV dataset 2b.

BCI Competition IV dataset 2b, a dataset is made up of nine EEG participants. There were five sessions available for every subject. The final set of recordings had feedback, but the first two contained training data without any. For each subject in this paper, only data without feedback has been chosen as the dataset. A fixed cross and an extra-brief auditory warning tone (1 kHz, 70 ms) were played at the beginning of each trial. A short while later, a visual indication emerged for 1.25 seconds, displaying an arrow pointing to the left or right, depending on the category that was requested. Within four seconds, the subjects had to picture the matching hand gestures. Every trial was separated by a minimum of 1.5 seconds of rest. To further prevent adaptation, the rest interval was extended by up to one second of randomization. The timing diagram of the BCI competition IV dataset 2b has been illustrated in Fig 4.

[Figure omitted. See PDF.]

Three bipolar recordings (C3, Cz, and C4) were made at a sampling frequency of 250 Hz for the dataset. They had a 50 Hz trap filter turned on and were band-pass filtered between 0.5 Hz and 100 Hz. For every participant, the three bipolar recordings were located in slightly different areas. Experiments on the motor imagery task for both left- and right-handed motions were included in the data collection. There were 120 trials per session.

The power spectral density (PSD) of C3, Cz, and C4 electrode is shown in Fig 5(A)–5(C). In motor imagery EEG signal classification, the power spectral density (PSD) of electrodes C3, C4, and CZ plays a crucial role in capturing frequency domain information associated with motor imagery tasks. C3, C4, and Cz electrodes are commonly selected for motor imagery EEG signal analysis due to their proximity to motor-related areas of the brain (central cortex). The PSD is typically across specific frequency bands relevant to motor imagery tasks, such as delta (1-4Hz), theta (4-8Hz), alpha (8-12Hz), beta (12-30Hz), and gamma (greater than 30Hz). These bands capture different neural oscillations associated with motor planning, execution, and inhibition. Overall, the PSD of electrodes C3, C4, and CZ provides valuable frequency domain information for discriminating between different motor imagery tasks, facilitating accurate classification in EEG-based brain-computer interface (BCI) systems. The topographical maps of four classes (Right hand, Left hand, Foot, and Tongue) is illustrated in Fig 6.

[Figure omitted. See PDF.]

Diagram for Power Spectral Density of the electrodes (a) C3 electrode (b) Cz electrode (c) C4 electrode.

[Figure omitted. See PDF.]

Diagram for the topographical maps of four classes (Right hand, Left hand, Foot, and Tongue).

3.2 EMD

For nonlinear and nonstationary signals, empirical mode decomposition (EMD) is a fully data-driven technique for estimating time-frequency with highly localised information [42], by which the signal at hand is divided into a limited number of intrinsic mode functions (IMFs) and the intrinsic mode functions are defined as the components of AM/FM.

Two requirements must be met in order for a signal to qualify as an IMF: (i) The difference between the number of zero crossings and extrema is no more than one. (ii) The mean of the envelopes linking the local maxima and minima, respectively, is close to zero. Every IMF can therefore be thought of as a narrow-band signal reflecting a variable temporal scale inherent to the data, which is a crucial characteristic that sets EMD apart from Fourier approaches. The Fig 7 represents the flowchart of an EMD algorithm.

[Figure omitted. See PDF.]

The standard EMD can be calculated in several steps. In this, the first and foremost one is computing the local mean and the local mean can be calculated by taking the average of both the upper and lower envelopes [43]. The standard EMD algorithm is shown in Table 1.

[Figure omitted. See PDF.]

3.3 Features of EEG signals

For a given sequence g(n) = {g(1), g(2),…,g(N)}, where N is the length of the signal.

3.3.1 Energy.

When determining left-right MI EEG, energy is an important metric and can be calculated using the following formula.

(3.1)

3.3.2 Autoregressive (AR) coefficients.

AR models have been demonstrated to be a beneficial feature in MI recognition tasks and have been widely used in BCI research. The time-varying properties of signals are represented by the AR coefficients. In addition, AR is very good at modelling the EEG as filtered white noise with appropriate energy bands, which makes it perfect for EEG signal analysis. The following is the equation for the AR coefficient:(3.2)

Where e(t) is the added white noise. The correct AR model order is set to seven.

3.3.3 Fuzzy approximate entropy.

The fuzzy approximation entropy (FAP) can be obtained using the procedures listed below.

A) From a given signal f(i), the structure vector(3.3)where,(3.4)

B) Using the tolerance factor r, a fuzzy membership function has been established in order to calculate the maximal distance .

(3.5)

C) The fuzzy approximate entropy is defined as follows:(3.6)

Where,(3.7)

Where,(3.8)

3.3.4 Peak-to-peak amplitude.

The difference between the signal’s maximum and minimum values is the peak-to-peak amplitude.

(3.9)

3.3.5 Skewness.

The asymmetry of a real-valued random variable’s probability distribution is measured by its skewness. Usually, the sample skewness is computed as follows:(3.10)where sample mean and standard deviation is denoted as and s.

3.3.6 Kurtosis.

The probability distribution of a real-valued random variable’s is measured by kurtosis. One way to calculate sample kurtosis is to:(3.11)

These features are calculated for each epoch (segments of EEG data corresponding to an event of interest) and are used to represent the characteristics of the EEG signal. The features are then concatenated into a feature vector for each epoch.

3.4 Convolutional neural network

Machine learning is a rapidly developing field, and deep learning is a crucial component of it. Deep learning has steadily emerged as a pioneer in several domains, drawing the interest of many researchers. CNNs are a popular neural network model in deep learning with several applications in different fields. Additionally, appropriate research on BCI systems is progressing in the intervening period. The Fig 8 illustrates the basic block diagram of a convolutional neural network.

[Figure omitted. See PDF.]

In the middle of the network construction, several convolutional and pooling layers can be joined, followed by fully connected layers. The convolutional layer, which is at the core of CNN, is primarily responsible for extracting features from the input signal. It is essential that the convolutional layer be able to apply the necessary convolutional operations to the input signal. Convolutional kernels, sometimes referred to as filters, can be found in several instances inside a single convolutional layer. It is possible to modify the convolution kernels’ weight parameters and bias while the neural network is being trained. Convolutional operations can produce feature mappings from the input to the output by applying the concepts of matrix multiplication. It is assumed that the neural element is located at (m, n) in the feature map that the kth convolution kernel produces. Eq (4) shows the output.(3.12)where f is the nerve element’s activation function, w(i)k is the kth convolution kernel of the i_th layer, b is the bias, and I(m, n) is the input data. Its frequent form includes tanch, rectified linear unit (ReLU) and sigmoid.

(3.13)(3.14)(3.15)

On the other hand, more specific characteristics can be extracted through additional connections to tighter layers. In order to extract more complex abstract features from the signal as much as possible, it is possible to combine and superimpose many convolutional layers to constantly extract features from high-dimensional input data. The convolution layer processes actions like padding and stride in addition to the convolution operation, and the corresponding computational process is more complicated. The technique of lowering the input data’s length and breadth dimensions is called pooling, sometimes referred to as the sampling layer. The pooling layer can extract local information by performing a down sampling operation after the convolutional layer. The pooling layer makes it possible to reduce the number of network parameters, which also reduces the model’s complexity and computational load. In addition, this can help the network model’s overfitting issue and be incredibly resilient to slight data inaccuracies. Two main pooling operations are present in the pooling layer. By using maximum pooling, the biggest element value of the convolution kernel in the target region determines the feature value of the region. The convolution layer’s tiny flaws are optimized, and more local features are highlighted by the maximum pooling procedure. Similarly, by averaging pooling, the average value of its elements determines the feature value of the target region. The average pooling method not only preserves more adjacency information but also reduces the region error brought on by the convolution kernel’s size. In Eqs 3.16 and 3.17, the maximum and average pooling expressions are displayed. We suppose that the pooling kernel has a size of (N * N).

Max Pooling:(3.16)

Average Pooling:(3.17)

In most cases, the fully linked layer is regarded as the last element in the CNN structure. When the feature space transformation travels through the fully linked layer, which integrates the previously extracted features for use in matrix multiplication, it transforms the feature maps created by the intermediate layers into vector format. The fully connected layer is in charge of converting the spatial high-dimensional information that the CNN has already retrieved and using non-linear mapping to wrap up the entire learning process.

The employment of design concepts including weight sharing, non-linear mapping of fully connected layers, sampling of pooling layers, sparse connectivity of convolutional layers, and sampling of pooling layers is primarily responsible for the CNN network model’s superior robustness and excellent generalization performance. For efficient feature extraction and classification recognition, a significant number of samples must be processed and examined in classical recognition research. Convolutional operations and other related operations are used to automatically train CNN models based on the inherent properties of the signal. CNN’s network model structure offers good interpretability in addition to excellent feature extraction effects.

4. Proposed EMD-PCNN method

The initial data is reduced by the first IMF,(4.1)and the process is repeated on the residue r(t) until it stabilises or has stopped oscillating; this process, known as sifting, is guided by a properly defined stopping criterion [28]. The signal v(t) is then calculated by,(4.2)

Where c_i(t), i = 1,…, M are the IMF’s and r(t) is the remaining residue. By using the EMD, the above features are extracted such as energy, AR coefficients, fuzzy approximate entropy, peak-to-peak, skewness, kurtosis.

For performance improvement, several CNN models and approaches have been used in various sectors. As the convolution process proceeds layer by layer, CNN is able to automatically extract rich information. While the end layers are focused on extracting global characteristics, initial layers extract local and spatial features. Simple shapes like borders and boundaries might be considered lower-level features, while complicated shapes and entire objects are represented by high-level features. Convolutional features can accomplish different abstractions of object information at each level, as demonstrated by the numerous researchers who have extracted convolutional features from various layers and fused them to increase performance. The layered architecture allows for the loss of some important elements, which can be used to enrich the final feature set. Performance is increased even if the method uses more resources and has extra features. Researchers have developed many methods for extracting and combining multilayer convolutional features for various domains.

A parallel convolutional neural network (CNN) is a type of neural network architecture designed to process input data in parallel across multiple computational units or devices. In traditional CNNs, convolutional layers are applied sequentially, meaning each layer waits for the previous layer to finish processing before it can begin its own computations. However, in parallel CNNs, multiple convolutional layers or operations can be executed simultaneously, leading to potentially faster processing times and improved efficiency. The key components and characteristics of a parallel CNN are the following: The Convolutional layer is one of the fundamental building blocks of a CNN. Each layer applies a set of learnable filters (kernels) to the input data to extract features. In a parallel CNN, multiple convolutional layers can operate concurrently, distributing the workload across different computational units.

Parallelism in a CNN can be achieved at various levels, including data parallelism and model parallelism. Data parallelism involves splitting the input data across multiple processing units and applying the same operations simultaneously. Model parallelism, on the other hand, partitions the neural network model itself across different devices, with each device responsible for computing a portion of the model’s operations. Parallel CNNs are often deployed on multi-GPU or multi-core systems to exploit parallel processing capabilities. Each GPU or core can independently process a subset of the data or model parameters, allowing for faster training and inference. Efficient communication and synchronization mechanisms are crucial in parallel CNNs to ensure that different processing units can exchange information and coordinate their computations effectively. This is particularly important in scenarios where data or model parameters need to be synchronized across different devices. Parallel CNNs offer scalability benefits, allowing them to handle larger datasets and more complex models by leveraging additional computational resources. This scalability is especially valuable in tasks such as image recognition, object detection, and natural language processing, where large amounts of data and computational power are often required.

Overall, parallel convolutional neural networks enable faster training and inference times, improved scalability, and efficient utilization of computational resources, making them well-suited for tasks that demand high computational performance and scalability.

The one-dimensional convolutional neural network (1D-CNN), which is the foundation of the MI-EEG BCI system discussed here, is distinguished by the CNN kernel’s ability to only slide over the elements of the input pattern’s single dimension in this case, time during convolution. The 1D-CNN specifically accepts as input a matrix of dimensions M × N, where M is the time window length that is taken into consideration and N is the number of EEG channels. Specifically, a matrix with dimensions M × N where M is the length of the time frame under consideration and N is the number of EEG channels is fed into the 1D-CNN. In this case, M = 640, or 160 × 4 steps, and N = 2, or two symmetrical channels have as input. A variable dimension Q × N kernel is used in each 1D convolutional layer; Q represents the temporal window that the filter covers, and N = 2 because the kernel does not slide over the channels.

The 1D convolutional layer is represented mathematically as follows:(4.3)where y_r is the filter feature map of size R (R = M) output of unit r. The two-dimensional input component that overlaps the filter in the scenario when stride = 1 and padding is employed is denoted by x, the bias term is b, the activation function of the filter is f, and w is the convolutional filter’s connection weight. Following the convolution operation (R), we can use the following formula to get the dimension of the filter feature map:

The first convolutional layer (L1) employs padding to maintain the precise size of the input and output, and 32 filters of size 20 with a stride of 1. Batch normalization (BN) is used after the first convolutional layer. In order to improve network generalization, this entails normalizing the input to the following layer, which typically results in a significant boost in learning speed and noticeable regularization benefits. BN operates in a different way during testing and training. BN uses the whole batch (the collection of instances needed to calculate the loss and the gradient the learning algorithm uses) to normalize and zero-centre the input during training. The Fig 9 illustrates the block diagram of Parallel Convolutional Neural Network.

[Figure omitted. See PDF.]

4.1 Network training and transfer learning

The categorical cross-entropy loss function served as the foundation for the neural network parameter optimization. The PCNN parameters were updated, and the category cross-entropy loss was minimized using the Adam optimization approach. In the deep learning literature, selecting the optimization algorithm is still up for discussion. We could have tested various optimization algorithms, but instead we have opted to stick with current one. Adam optimization is a stochastic gradient descent (SGD) approach that adjusts according to the gradient’s first- and second-order moments. It is based on a distinct learning rate for each parameter. Training employed a random order and a non-fixed number of epochs, where an epoch is a learning loop that uses all the examples once in a random order. To prevent overfitting, the validation early-stopping strategy was specifically applied. Through the use of transfer learning, a neural network can be trained to find general patterns and features in a large dataset (such as MI-EGG data from numerous individuals) and use those features to predict data on a new, similar dataset (such as data pertaining to a new individual) with the least amount of additional training. Using our method, we applied transfer learning to all subjects individually. A training set (70% of the data) and a test set (30% of the data) were separated from the dataset.

5. Results and analysis

Our proposed model EMD-PCNN is evaluated based on the two ways such as Qualitative and Quantitative.

5.1 Qualitative evaluation

The proposed EMD-PCNN method is analysed in a qualitative manner. The various evaluation parameters are used to evaluate the quality of the proposed technique. Such evaluation parameters are Specificity, Sensitivity, precision, and Accuracy. Specificity is used to identify the true negative of the proposed model, whereas sensitivity is used to identify the positive instances of the proposed method. Precision is used to trace or predict the true value of the proposed model. Accuracy is used to determine whether the classification model is correct overall and how often it acquires the correct value. Precision and accuracy are independent of each other. These evaluation parameters are analysed using the confusion matrix, which is described briefly in the following section.

5.1.1 Confusion matrix.

A confusion matrix or an error matrix is used in various fields, especially in machine learning, to identify or solve the problem of the classification phase statistically. The algorithm performance is visualised in both the supervised and unsupervised, it is called the matching matrix in simple terms. In the confusion matrix, there are two classes available: one is the actual class, and the other is the predicted class. The predicted class is represented by each row of the matrix, and the actual class is represented by each column of the matrix, or vice versa. The Fig 10 illustrates the confusion matrix for the proposed EMD-PCNN method.

[Figure omitted. See PDF.]

5.1.2 Evaluation parameters.

From the confusion matrix, we can calculate different evaluation metrics such as Specificity, Sensitivity, Precision Accuracy, Recall and F1-score for the validation of the model. The evaluation parameter calculation is done based on the following Eqs 5.1, 5.2, 5.3, 5.4, 5.5, and 5.6. Such evaluation metrics are used to assess the classification efficiency of the proposed EMD-PCNN technique. The Table 2 illustrates the hyper parameter setting of our proposed method EMD-PCNN. Table 3 illustrates the Subject-specific classification results for each subject for the BCI IV dataset (in %) and Table 4 Performance results of the Proposed method (in %). The T-SNE visualization of the proposed method EMD-PCNN is illustrated in Fig 11.

[Figure omitted. See PDF.]

[Figure omitted. See PDF.]

[Figure omitted. See PDF.]

[Figure omitted. See PDF.]

5.1.2.1 Specificity.

(5.1)

5.1.2.2 Sensitivity.

(5.2)

5.1.2.3 Precision.

(5.3)

5.1.2.4 Accuracy.

(5.4)

5.1.2.5 Recall.

(5.5)

5.1.2.5 F1- score.

(5.6)

5.2 Quantitative evaluation

The quantitative evaluation of the proposed method is discussed in Table 5 which describes the description of the proposed method and the existing one with the classification accuracy. Ang et al. (2012) employed a filter bank common spatial pattern (CSP) method achieving a 68% accuracy, while Tabar et al. (2016) used a 1D CNN with a stacked autoencoder (SAE) reaching 70%. Lawhern et al. (2018) implemented a CNN with depth and separable convolutions, yielding a 60% accuracy. Schirrmeister et al. (2017) introduced cropped training in a CNN architecture, improving accuracy to 72%. Sakhavi et al. (2018) combined temporal features with filter bank CSP and CNN, resulting in a 74.4% accuracy. Kevric and Subasi (2017) achieved a notable 92.8% accuracy with a KNN model using wavelet packet decomposition. In 2021, Mandeep Kaur Ghumman and Satvir Singh applied a support vector machine with CSP, achieving a 66.4% accuracy, while Rahul Sharma et al. (2022) used a multi-layer perceptron model that attained a high 92% accuracy. Arunabha M. Roy’s 2022 studies introduced a multi-scale CNN and a transfer learning-based multi-scale feature fused CNN (MSFFCNN), reaching accuracies of 93.74% and 94.06%, respectively. In 2023, Yu Xie et al. applied a CNN with continuous wavelet transform, achieving an impressive 97.61% accuracy. The highest accuracy, 99.39%, is attributed to the proposed method combining a PCNN with Empirical Mode Decomposition (EMD), showcasing the significant advancements and potential of hybrid deep learning approaches in EEG classification. In Fig 12, the subject classification results obtained by using BCI competition IV dataset for the proposed method has been illustrated.

[Figure omitted. See PDF.]

[Figure omitted. See PDF.]

6. Conclusion

Motor imagery EEG signals hold significant promise for brain-computer interface (BCI) applications, enabling users to control external devices through cognitive processes. However, accurate classification of these signals remains a challenge due to their non-stationary nature and complex spectral characteristics. In this paper, we propose a novel approach that combines empirical mode decomposition (EMD) and parallel convolutional neural networks (PCNN) to address these challenges. By decomposing EEG signals into intrinsic mode functions (IMFs) using EMD and employing PCNN for feature classification, we aim to enhance classification accuracy and robustness. Our methodology consists of two main stages: feature extraction and classification. In the feature extraction stage, EEG signals are decomposed into IMFs using EMD, which effectively captures their non-stationary characteristics. Subsequently, these IMFs are used as input features for the PCNN classifier. The PCNN architecture is designed to operate in parallel, enabling efficient processing of multi-dimensional input data and extraction of discriminative features. We evaluate the proposed EMD-PCNN approach using the BCI Competition IV datasets 2a and 2b, which contain motor imagery EEG recordings from multiple subjects performing different tasks. We compare the classification performance of our method against existing approaches, including traditional machine learning algorithms and deep learning models. Our experimental results demonstrate that the EMD-PCNN method outperforms existing techniques in terms of classification accuracy. Specifically, we achieve a classification accuracy of 99.1% for four classes and 99.39% for two classes, surpassing the performance of state-of-the-art methods. Additionally, qualitative analysis using confusion matrices provides insights into the effectiveness of our approach in accurately distinguishing between different motor imagery tasks. Furthermore, quantitative evaluation metrics such as specificity, sensitivity, precision, accuracy, recall, and F1-score further validate the superiority of our method. In conclusion, we have presented a novel approach for the classification of motor imagery EEG signals using empirical mode decomposition and parallel convolutional neural networks. Our method demonstrates significant improvements in classification accuracy compared to existing techniques, effectively addressing the non-stationary nature of EEG signals. The proposed EMD-PCNN framework holds promise for enhancing the performance of BCI systems, paving the way for more reliable and efficient brain-computer interfaces. Future research directions may involve exploring additional feature extraction techniques and optimizing the PCNN architecture for real-time applications.

6.1 Future scope

Future studies will concentrate on enhancing deep learning algorithms to improve the classification of motor imagery EEG data and produce a more dependable brain-computer interface. We believe that the results of this study will encourage other researchers to use neural networks to improve the ability to classify brain signals and in future, by using the proposed result an external application or devices will develop for severely disabled people.

Supporting information

S1 Dataset.

https://doi.org/10.1371/journal.pone.0311942.s001

(GDF)

S2 Dataset.

https://doi.org/10.1371/journal.pone.0311942.s002

(GDF)

S3 Dataset.

https://doi.org/10.1371/journal.pone.0311942.s003

(GDF)

S4 Dataset.

https://doi.org/10.1371/journal.pone.0311942.s004

(GDF)

References

1. 1. Curran Eleanor A., and Stokes Maria J. "Learning to control brain activity: A review of the production and control of EEG components for driving brain–computer interface (BCI) systems." Brain and cognition 51, no. 3 (2003): 326–336.

* View Article

* Google Scholar

2. 2. Zimmermann-Schlatter Andrea, Schuster Corina, Puhan Milo A., Siekierka Ewa, and Steurer Johann. "Efficacy of motor imagery in post-stroke rehabilitation: a systematic review." Journal of neuroengineering and rehabilitation 5, no. 1 (2008): 1–10. pmid:18341687

* View Article

* PubMed/NCBI

* Google Scholar

3. 3. Tong Yanna, John T. Pendy Jr, William A. Li, Huishan Du, Tong Zhang, Xiaokun Geng, et al. "Motor imagery-based rehabilitation: potential neural correlates and clinical application for functional recovery of motor deficits after stroke." Aging and disease 8, no. 3 (2017): 364. pmid:28580191

* View Article

* PubMed/NCBI

* Google Scholar

4. 4. Kumar ,Vijaya K. M. Chakrapani , and Rakshith Kedambadi. "Motor imagery training on muscle strength and gait performance in ambulant stroke subjects-a randomized clinical trial." Journal of clinical and diagnostic research: JCDR 10, no. 3 (2016): YC01. pmid:27134985

* View Article

* PubMed/NCBI

* Google Scholar

5. 5. Pawuś Dawid, and Szczepan Paszkiel . "The application of integration of EEG signals for authorial classification algorithms in implementation for a mobile robot control using movement imagery—Pilot study." Applied Sciences 12, no. 4 (2022): 2161.

* View Article

* Google Scholar

6. 6. Yang Huijuan, Sakhavi Siavash, Kai Keng Ang, and Cuntai Guan. "On the use of convolutional neural networks and augmented CSP features for multi-class motor imagery of EEG signals classification." In 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 2620–2623. IEEE, 2015. pmid:26736829

* View Article

* PubMed/NCBI

* Google Scholar

7. 7. Nath Malaya Kumar, Kanhe Aniruddha, and Mishra Madhusudhan. "A novel deep learning approach for classification of COVID-19 images." In 2020 IEEE 5th international conference on computing communication and automation (ICCCA), pp. 752–757. IEEE, 2020.

* View Article

* Google Scholar

8. 8. Jaipriya D., and Sriharipriya K. C. "A comparative analysis of masking empirical mode decomposition and a neural network with feed-forward and back propagation along with masking empirical mode decomposition to improve the classification performance for a reliable brain-computer interface." Frontiers in Computational Neuroscience 16 (2022): 1010770. pmid:36405787

* View Article

* PubMed/NCBI

* Google Scholar

9. 9. Zhang Kai, Xu Guanghua, Han Zezhen, Ma Kaiquan, Zheng Xiaowei, Chen Longting, et al. "Data augmentation for motor imagery signal classification based on a hybrid neural network." Sensors 20, no. 16 (2020): 4485. pmid:32796607

* View Article

* PubMed/NCBI

* Google Scholar

10. 10. Yang Banghua, Duan Kaiwen, Fan Chengcheng, Hu Chenxiao, and Wang Jinlong. "Automatic ocular artifacts removal in EEG using deep learning." Biomedical Signal Processing and Control 43 (2018): 148–158.

* View Article

* Google Scholar

11. 11. Kapgate Deepak. "Efficient quadcopter flight control using hybrid SSVEP+ P300 visual brain computer interface." International Journal of Human–Computer Interaction 38, no. 1 (2022): 42–52.

* View Article

* Google Scholar

12. 12. Zhang Zhiwen, Duan Feng, Jordi Sole-Casals Josep Dinares-Ferran, Cichocki Andrzej, Yang Zhenglu, et al. "A novel deep learning approach with data augmentation to classify motor imagery signals." IEEE Access 7 (2019): 15945–15954.

* View Article

* Google Scholar

13. 13. Pei Yu, Luo Zhiguo, Yan Ye, Yan Huijiong, Jiang Jing, Li Weiguo, et al. "Data augmentation: using channel-level recombination to improve classification performance for motor imagery EEG." Frontiers in Human Neuroscience 15 (2021): 645952. pmid:33776673

* View Article

* PubMed/NCBI

* Google Scholar

14. 14. Aggarwal Swati, and Chugh Nupur. "Signal processing techniques for motor imagery brain computer interface: A review." Array 1 (2019): 100003.

* View Article

* Google Scholar

15. 15. Lu Haiping, Eng How-Lung, Guan Cuntai, Plataniotis Konstantinos N., and Venetsanopoulos Anastasios N. "Regularized common spatial pattern with aggregation for EEG classification in small-sample setting." IEEE transactions on Biomedical Engineering 57, no. 12 (2010): 2936–2946. pmid:20889425

* View Article

* PubMed/NCBI

* Google Scholar

16. 16. Subasi Abdulhamit, and Ismail Gursoy M. "EEG signal classification using PCA, ICA, LDA and support vector machines." Expert systems with applications 37, no. 12 (2010): 8659–8666.

* View Article

* Google Scholar

17. 17. Zhang Yu, Zhou Guoxu, Jin Jing, Zhao Qibin, Wang Xingyu, and Cichocki Andrzej. "Sparse Bayesian classification of EEG for brain–computer interface." IEEE transactions on neural networks and learning systems 27, no. 11 (2015): 2256–2267. pmid:26415189

* View Article

* PubMed/NCBI

* Google Scholar

18. 18. Kirar Jyoti Singh, and Agrawal R. K. "Relevant feature selection from a combination of spectral-temporal and spatial features for classification of motor imagery EEG." Journal of medical systems 42, no. 5 (2018): 1–15. pmid:29546648

* View Article

* PubMed/NCBI

* Google Scholar

19. 19. Gaur Pramod, Ram Bilas Pachori, Hui Wang, and Girijesh Prasad . "An empirical mode decomposition-based filtering method for classification of motor-imagery EEG signals for enhancing brain-computer interface." In 2015 International Joint Conference on Neural Networks (IJCNN), pp. 1–7. IEEE, 2015.

* View Article

* Google Scholar

20. 20. Cano-Izquierdo Jose-Manuel, Ibarrola Julio, and Almonacid Miguel. "Improving motor imagery classification with a new BCI design using neuro-fuzzy S-dFasArt." IEEE Transactions on Neural Systems and Rehabilitation Engineering 20, no. 1 (2011): 2–7. pmid:21997321

* View Article

* PubMed/NCBI

* Google Scholar

21. 21. Sagee G. S., and Hema S. "EEG feature extraction and classification in multiclass multiuser motor imagery brain computer interface u sing Bayesian Network and ANN." In 2017 International Conference on Intelligent Computing, Instrumentation and Control Technologies (ICICICT), pp. 938–943. IEEE, 2017.

* View Article

* Google Scholar

22. 22. Hamedi Mahyar, Salleh Sh-Hussain, Alias Mohd Noor, and Iman Mohammad-Rezazadeh. "Neural network-based three-class motor imagery classification using time-domain features for BCI applications." In 2014 IEEE Region 10 Symposium, pp. 204–207. IEEE, 2014.

* View Article

* Google Scholar

23. 23. Agarwal Saurabh Kumar, Shah Saatvik, and Kumar Rajesh. "Classification of mental tasks from EEG data using backtracking search optimization based neural classifier." Neurocomputing 166 (2015): 397–403.

* View Article

* Google Scholar

24. 24. Li Ming-Ai, Han Jian-Fu, and Duan Li-Juan. "A novel MI-EEG imaging with the location information of electrodes." IEEE Access 8 (2019): 3197–3211.

* View Article

* Google Scholar

25. 25. Zhang Jin, Yan Chungang, and Gong Xiaoliang. "Deep convolutional neural network for decoding motor imagery-based brain computer interface." In 2017 IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC), pp. 1–5. IEEE, 2017.

* View Article

* Google Scholar

26. 26. Sakhavi Siavash, Guan Cuntai, and Yan Shuicheng. "Parallel convolutional-linear neural snetwork for motor imagery classification." In 2015 23rd European Signal Processing Conference (EUSIPCO), pp. 2736–2740. IEEE, 2015.

* View Article

* Google Scholar

27. 27. Ko Wonjun, Yoon Jeeseok, Kang Eunsong, Jun Eunji, Choi Jun-Sik, and Suk Heung-Il. "Deep recurrent spatio-temporal neural network for motor imagery based BCI." In 2018 6th International Conference on Brain-Computer Interface (BCI), pp. 1–3. IEEE, 2018.

* View Article

* Google Scholar

28. 28. Donoghue John P. "Connecting cortex to machines: recent advances in brain interfaces." Nature neuroscience 5, no. 11 (2002): 1085–1088.

* View Article

* Google Scholar

29. 29. Donoghue J. P., Mijail D., Serruya G. N., Hatsopoulos L. P., and Matthew F. R. "Brain-machine interface: Instant neural control of a movement signal." Nature 416 (2002): 14.

* View Article

* Google Scholar

30. 30. Aler Ricardo, Inés M. Galván , and José M. Valls . "Applying evolution strategies to preprocessing EEG signals for brain–computer interfaces." Information Sciences 215 (2012): 53–66.

* View Article

* Google Scholar

31. 31. Birbaumer Niels. "Slow cortical potentials: plasticity, operant control, and behavioral effects." The Neuroscientist 5, no. 2 (1999): 74–78.

* View Article

* Google Scholar

32. 32. Suleiman , Abdul-Bary Raouf , and Toka Abdul-Hameed Fatehi. "Features extraction techniques of EEG signal for BCI applications." Faculty of Computer and Information Engineering Department College of Electronics Engineering, University of Mosul, Iraq (2007).

* View Article

* Google Scholar

33. 33. Im Chang-Hwan. Computational EEG Analysis. Springer Singapore, 2018.

* View Article

* Google Scholar

34. 34. Nayak , Pinki S. Meenakshi , and Nageswara Rao Medikondu. "Predicting the tasks of disabled persons using deep learning-based motor imagery model in BCI applications." Research on Biomedical Engineering 39, no. 4 (2023): 977–989.

* View Article

* Google Scholar

35. 35. Pérez-Zapata A. F., Andrés Felipe Cardona-Escobar, Jorge Alberto Jaramillo-Garzón, and Díaz Gloria M. "Deep convolutional neural networks and power spectral density features for motor imagery classification of EEG signals." In Augmented Cognition: Intelligent Technologies: 12th International Conference, AC 2018, Held as Part of HCI International 2018, Las Vegas, NV, USA, July 15–20, 2018, Proceedings, Part I, pp. 158–169. Springer International Publishing, 2018.

* View Article

* Google Scholar

36. 36. Xu Baoguo, Zhang Linlin, Song Aiguo, Wu Changcheng, Li Wenlong, Zhang Dalin, et al. "Wavelet transform time-frequency image and convolutional network-based motor imagery EEG classification." Ieee Access 7 (2018): 6084–6093.

* View Article

* Google Scholar

37. 37. Virgilio G.CD.A. Sossa JH, Javier M. Antelis , and Luis E. Falcón . "Spiking Neural Networks applied to the classification of motor tasks in EEG signals." Neural Networks: the Official Journal of the International Neural Network Society 122 (2019): 130–143. pmid:31677441

* View Article

* PubMed/NCBI

* Google Scholar

38. 38. Jaipriya D., and Sriharipriya K. C. "Brain Computer Interface-Based Signal Processing Techniques for Feature Extraction and Classification of Motor Imagery Using EEG: A Literature Review." Biomedical Materials & Devices (2023): 1–13.

* View Article

* Google Scholar

39. 39. Qi Lin, Zhang Haoran, Cao Xuehao, Lyu Xuyang, Xu Lisheng, Yang Benqiang, et al. "Multi-scale feature fusion convolutional neural network for concurrent segmentation of left ventricle and myocardium in cardiac mr images." Journal of Medical Imaging and Health Informatics 10, no. 5 (2020): 1023–1032.

* View Article

* Google Scholar

40. 40. Roy Arunabha M. "A multi-scale fusion CNN model based on adaptive transfer learning for multi-class MI-classification in BCI system." BioRxiv (2022): 2022–03.

* View Article

* Google Scholar

41. 41. Shao Xiaorui, and Kim Chang-Soo. "Unsupervised domain adaptive 1D-CNN for fault diagnosis of bearing." Sensors 22, no. 11 (2022): 4156. pmid:35684777

* View Article

* PubMed/NCBI

* Google Scholar

42. 42. Park Cheolsoo, Looney David, Kidmose Preben, Ungstrup Michael, and Mandic Danilo P. "Time-frequency analysis of EEG asymmetry using bivariate empirical mode decomposition." IEEE Transactions on Neural Systems and Rehabilitation Engineering 19, no. 4 (2011): 366–373. pmid:21342855

* View Article

* PubMed/NCBI

* Google Scholar

43. 43. Bashar Syed Khairul, and Mohammed Imamul Hassan Bhuiyan. "Classification of motor imagery movements using multivariate empirical mode decomposition and short time Fourier transform based hybrid method." Engineering science and technology, an international journal 19, no. 3 (2016): 1457–1464.

* View Article

* Google Scholar

44. 44. Tang Xianlun, Li Wei, Li Xingchen, Ma Weichang, and Dang Xiaoyuan. "Motor imagery EEG recognition based on conditional optimization empirical mode decomposition and multi-scale convolutional neural network." Expert Systems with Applications 149 (2020): 113285.

* View Article

* Google Scholar

45. 45. Fang Yonghui, Chen Minyou, and Zheng Xufei. "Extracting features from phase space of EEG signals in brain–computer interfaces." Neurocomputing 151 (2015): 1477–1485.

* View Article

* Google Scholar

46. 46. Amin Syed Umar, Alsulaiman Mansour, Muhammad Ghulam, Mohamed Amine Mekhtiche, and Shamim Hossain M. "Deep Learning for EEG motor imagery classification based on multi-layer CNNs feature fusion." Future Generation computer systems 101 (2019): 542–554.

* View Article

* Google Scholar

47. 47. Dai Guanghai, Zhou Jun, Huang Jiahui, and Wang Ning. "HS-CNN: a CNN with hybrid convolution scale for EEG motor imagery classification." Journal of neural engineering 17, no. 1 (2020): 016025. pmid:31476743

* View Article

* PubMed/NCBI

* Google Scholar

48. 48. Ma Xunguang, Wang Dashuai, Liu Danhua, and Yang Jimin. "DWT and CNN based multi-class motor imagery electroencephalographic signal recognition." Journal of neural engineering 17, no. 1 (2020): 016073. pmid:31972552

* View Article

* PubMed/NCBI

* Google Scholar

49. 49. Zhang Ruilong, Zong Qun, Dou Liqian, and Zhao Xinyi. "A novel hybrid deep learning scheme for four-class motor imagery classification." Journal of neural engineering 16, no. 6 (2019): 066004. pmid:31341093

* View Article

* PubMed/NCBI

* Google Scholar

50. 50. Pfurtscheller Gert, Neuper Christa, Schlogl Alois, and Lugger Klaus. "Separability of EEG signals recorded during right and left motor imagery using adaptive autoregressive parameters." IEEE transactions on Rehabilitation Engineering 6, no. 3 (1998): 316–325. pmid:9749909

* View Article

* PubMed/NCBI

* Google Scholar

51. 51. Ang Kai Keng, Zheng Yang Chin Chuanchu Wang, Guan Cuntai, and Zhang Haihong. "Filter bank common spatial pattern algorithm on BCI competition IV datasets 2a and 2b." Frontiers in neuroscience 6 (2012): 39. pmid:22479236

* View Article

* PubMed/NCBI

* Google Scholar

52. 52. Tabar Yousef Rezaei, and Halici Ugur. "A novel deep learning approach for classification of EEG motor imagery signals." Journal of neural engineering 14, no. 1 (2016): 016003. pmid:27900952

* View Article

* PubMed/NCBI

* Google Scholar

53. 53. Lawhern Vernon J., Solon Amelia J., Waytowich Nicholas R., Gordon Stephen M., Hung Chou P., and Lance Brent J. "EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces." Journal of neural engineering 15, no. 5 (2018): 056013. pmid:29932424

* View Article

* PubMed/NCBI

* Google Scholar

54. 54. Schirrmeister Robin Tibor, Jost Tobias Springenberg Lukas Dominique Josef Fiederer, Glasstetter Martin, Eggensperger Katharina, Tangermann Michael, et al. "Deep learning with convolutional neural networks for EEG decoding and visualization." Human brain mapping 38, no. 11 (2017): 5391–5420 pmid:28782865

* View Article

* PubMed/NCBI

* Google Scholar

55. 55. Sakhavi Siavash, Guan Cuntai, and Yan Shuicheng. "Learning temporal information for brain-computer interface using convolutional neural networks." IEEE transactions on neural networks and learning systems 29, no. 11 (2018): 5619–5629. pmid:29994075

* View Article

* PubMed/NCBI

* Google Scholar

56. 56. Kevric Jasmin, and Subasi Abdulhamit. "Comparison of signal decomposition methods in classification of EEG signals for motor-imagery BCI system." Biomedical Signal Processing and Control 31 (2017): 398–406.

* View Article

* Google Scholar

57. 57. Ghumman Mandeep Kaur, Singh Satvir, Singh Navtej, and Jindal Balkrishan. "Optimization of parameters for improving the performance of EEG-based BCI system." Journal of Reliable Intelligent Environments 7 (2021): 145–156.

* View Article

* Google Scholar

58. 58. Sharma Rahul, Kim Minju, and Gupta Akansha. "Motor imagery classification in brain-machine interface with machine learning algorithms: Classical approach to multi-layer perceptron model." Biomedical Signal Processing and Control 71 (2022): 103101.

* View Article

* Google Scholar

59. 59. Roy Arunabha M. "An efficient multi-scale CNN model with intrinsic feature integration for motor imagery EEG subject classification in brain-machine interfaces." Biomedical Signal Processing and Control 74 (2022): 103496.

* View Article

* Google Scholar

60. 60. Roy Arunabha M. "Adaptive transfer learning-based multiscale feature fused deep convolutional neural network for EEG MI multiclassification in brain–computer interface." Engineering Applications of Artificial Intelligence 116 (2022): 105347.

* View Article

* Google Scholar

61. 61. Xie Yu, and Oniga Stefan. "Classification of Motor Imagery EEG Signals Based on Data Augmentation and Convolutional Neural Networks." Sensors 23, no. 4 (2023): 1932. pmid:36850530

* View Article

* PubMed/NCBI

* Google Scholar

Citation: D. J, K. C. S (2025) Parallel convolutional neural network and empirical mode decomposition for high accuracy in motor imagery EEG signal classification. PLoS ONE 20(1): e0311942. https://doi.org/10.1371/journal.pone.0311942

About the Authors:

Jaipriya D.

Roles: Methodology, Writing – original draft

Affiliation: School of Electronics Engineering (SENSE), Vellore Institute of Technology, Vellore, Tamil Nadu, India

Sriharipriya K. C.

Roles: Writing – review & editing

E-mail: [email protected]

Affiliation: School of Electronics Engineering (SENSE), Vellore Institute of Technology, Vellore, Tamil Nadu, India

ORICD: https://orcid.org/0000-0002-6755-6631

References

1. Curran Eleanor A., and Stokes Maria J. "Learning to control brain activity: A review of the production and control of EEG components for driving brain–computer interface (BCI) systems." Brain and cognition 51, no. 3 (2003): 326–336.

2. Zimmermann-Schlatter Andrea, Schuster Corina, Puhan Milo A., Siekierka Ewa, and Steurer Johann. "Efficacy of motor imagery in post-stroke rehabilitation: a systematic review." Journal of neuroengineering and rehabilitation 5, no. 1 (2008): 1–10. pmid:18341687

3. Tong Yanna, John T. Pendy Jr, William A. Li, Huishan Du, Tong Zhang, Xiaokun Geng, et al. "Motor imagery-based rehabilitation: potential neural correlates and clinical application for functional recovery of motor deficits after stroke." Aging and disease 8, no. 3 (2017): 364. pmid:28580191

4. Kumar ,Vijaya K. M. Chakrapani , and Rakshith Kedambadi. "Motor imagery training on muscle strength and gait performance in ambulant stroke subjects-a randomized clinical trial." Journal of clinical and diagnostic research: JCDR 10, no. 3 (2016): YC01. pmid:27134985

5. Pawuś Dawid, and Szczepan Paszkiel . "The application of integration of EEG signals for authorial classification algorithms in implementation for a mobile robot control using movement imagery—Pilot study." Applied Sciences 12, no. 4 (2022): 2161.

6. Yang Huijuan, Sakhavi Siavash, Kai Keng Ang, and Cuntai Guan. "On the use of convolutional neural networks and augmented CSP features for multi-class motor imagery of EEG signals classification." In 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 2620–2623. IEEE, 2015. pmid:26736829

7. Nath Malaya Kumar, Kanhe Aniruddha, and Mishra Madhusudhan. "A novel deep learning approach for classification of COVID-19 images." In 2020 IEEE 5th international conference on computing communication and automation (ICCCA), pp. 752–757. IEEE, 2020.

8. Jaipriya D., and Sriharipriya K. C. "A comparative analysis of masking empirical mode decomposition and a neural network with feed-forward and back propagation along with masking empirical mode decomposition to improve the classification performance for a reliable brain-computer interface." Frontiers in Computational Neuroscience 16 (2022): 1010770. pmid:36405787

9. Zhang Kai, Xu Guanghua, Han Zezhen, Ma Kaiquan, Zheng Xiaowei, Chen Longting, et al. "Data augmentation for motor imagery signal classification based on a hybrid neural network." Sensors 20, no. 16 (2020): 4485. pmid:32796607

10. Yang Banghua, Duan Kaiwen, Fan Chengcheng, Hu Chenxiao, and Wang Jinlong. "Automatic ocular artifacts removal in EEG using deep learning." Biomedical Signal Processing and Control 43 (2018): 148–158.

11. Kapgate Deepak. "Efficient quadcopter flight control using hybrid SSVEP+ P300 visual brain computer interface." International Journal of Human–Computer Interaction 38, no. 1 (2022): 42–52.

12. Zhang Zhiwen, Duan Feng, Jordi Sole-Casals Josep Dinares-Ferran, Cichocki Andrzej, Yang Zhenglu, et al. "A novel deep learning approach with data augmentation to classify motor imagery signals." IEEE Access 7 (2019): 15945–15954.

13. Pei Yu, Luo Zhiguo, Yan Ye, Yan Huijiong, Jiang Jing, Li Weiguo, et al. "Data augmentation: using channel-level recombination to improve classification performance for motor imagery EEG." Frontiers in Human Neuroscience 15 (2021): 645952. pmid:33776673

14. Aggarwal Swati, and Chugh Nupur. "Signal processing techniques for motor imagery brain computer interface: A review." Array 1 (2019): 100003.

15. Lu Haiping, Eng How-Lung, Guan Cuntai, Plataniotis Konstantinos N., and Venetsanopoulos Anastasios N. "Regularized common spatial pattern with aggregation for EEG classification in small-sample setting." IEEE transactions on Biomedical Engineering 57, no. 12 (2010): 2936–2946. pmid:20889425

16. Subasi Abdulhamit, and Ismail Gursoy M. "EEG signal classification using PCA, ICA, LDA and support vector machines." Expert systems with applications 37, no. 12 (2010): 8659–8666.

17. Zhang Yu, Zhou Guoxu, Jin Jing, Zhao Qibin, Wang Xingyu, and Cichocki Andrzej. "Sparse Bayesian classification of EEG for brain–computer interface." IEEE transactions on neural networks and learning systems 27, no. 11 (2015): 2256–2267. pmid:26415189

18. Kirar Jyoti Singh, and Agrawal R. K. "Relevant feature selection from a combination of spectral-temporal and spatial features for classification of motor imagery EEG." Journal of medical systems 42, no. 5 (2018): 1–15. pmid:29546648

19. Gaur Pramod, Ram Bilas Pachori, Hui Wang, and Girijesh Prasad . "An empirical mode decomposition-based filtering method for classification of motor-imagery EEG signals for enhancing brain-computer interface." In 2015 International Joint Conference on Neural Networks (IJCNN), pp. 1–7. IEEE, 2015.

20. Cano-Izquierdo Jose-Manuel, Ibarrola Julio, and Almonacid Miguel. "Improving motor imagery classification with a new BCI design using neuro-fuzzy S-dFasArt." IEEE Transactions on Neural Systems and Rehabilitation Engineering 20, no. 1 (2011): 2–7. pmid:21997321

21. Sagee G. S., and Hema S. "EEG feature extraction and classification in multiclass multiuser motor imagery brain computer interface u sing Bayesian Network and ANN." In 2017 International Conference on Intelligent Computing, Instrumentation and Control Technologies (ICICICT), pp. 938–943. IEEE, 2017.

22. Hamedi Mahyar, Salleh Sh-Hussain, Alias Mohd Noor, and Iman Mohammad-Rezazadeh. "Neural network-based three-class motor imagery classification using time-domain features for BCI applications." In 2014 IEEE Region 10 Symposium, pp. 204–207. IEEE, 2014.

23. Agarwal Saurabh Kumar, Shah Saatvik, and Kumar Rajesh. "Classification of mental tasks from EEG data using backtracking search optimization based neural classifier." Neurocomputing 166 (2015): 397–403.

24. Li Ming-Ai, Han Jian-Fu, and Duan Li-Juan. "A novel MI-EEG imaging with the location information of electrodes." IEEE Access 8 (2019): 3197–3211.

25. Zhang Jin, Yan Chungang, and Gong Xiaoliang. "Deep convolutional neural network for decoding motor imagery-based brain computer interface." In 2017 IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC), pp. 1–5. IEEE, 2017.

26. Sakhavi Siavash, Guan Cuntai, and Yan Shuicheng. "Parallel convolutional-linear neural snetwork for motor imagery classification." In 2015 23rd European Signal Processing Conference (EUSIPCO), pp. 2736–2740. IEEE, 2015.

27. Ko Wonjun, Yoon Jeeseok, Kang Eunsong, Jun Eunji, Choi Jun-Sik, and Suk Heung-Il. "Deep recurrent spatio-temporal neural network for motor imagery based BCI." In 2018 6th International Conference on Brain-Computer Interface (BCI), pp. 1–3. IEEE, 2018.

28. Donoghue John P. "Connecting cortex to machines: recent advances in brain interfaces." Nature neuroscience 5, no. 11 (2002): 1085–1088.

29. Donoghue J. P., Mijail D., Serruya G. N., Hatsopoulos L. P., and Matthew F. R. "Brain-machine interface: Instant neural control of a movement signal." Nature 416 (2002): 14.

30. Aler Ricardo, Inés M. Galván , and José M. Valls . "Applying evolution strategies to preprocessing EEG signals for brain–computer interfaces." Information Sciences 215 (2012): 53–66.

31. Birbaumer Niels. "Slow cortical potentials: plasticity, operant control, and behavioral effects." The Neuroscientist 5, no. 2 (1999): 74–78.

32. Suleiman , Abdul-Bary Raouf , and Toka Abdul-Hameed Fatehi. "Features extraction techniques of EEG signal for BCI applications." Faculty of Computer and Information Engineering Department College of Electronics Engineering, University of Mosul, Iraq (2007).

33. Im Chang-Hwan. Computational EEG Analysis. Springer Singapore, 2018.

34. Nayak , Pinki S. Meenakshi , and Nageswara Rao Medikondu. "Predicting the tasks of disabled persons using deep learning-based motor imagery model in BCI applications." Research on Biomedical Engineering 39, no. 4 (2023): 977–989.

35. Pérez-Zapata A. F., Andrés Felipe Cardona-Escobar, Jorge Alberto Jaramillo-Garzón, and Díaz Gloria M. "Deep convolutional neural networks and power spectral density features for motor imagery classification of EEG signals." In Augmented Cognition: Intelligent Technologies: 12th International Conference, AC 2018, Held as Part of HCI International 2018, Las Vegas, NV, USA, July 15–20, 2018, Proceedings, Part I, pp. 158–169. Springer International Publishing, 2018.

36. Xu Baoguo, Zhang Linlin, Song Aiguo, Wu Changcheng, Li Wenlong, Zhang Dalin, et al. "Wavelet transform time-frequency image and convolutional network-based motor imagery EEG classification." Ieee Access 7 (2018): 6084–6093.

37. Virgilio G.CD.A. Sossa JH, Javier M. Antelis , and Luis E. Falcón . "Spiking Neural Networks applied to the classification of motor tasks in EEG signals." Neural Networks: the Official Journal of the International Neural Network Society 122 (2019): 130–143. pmid:31677441

38. Jaipriya D., and Sriharipriya K. C. "Brain Computer Interface-Based Signal Processing Techniques for Feature Extraction and Classification of Motor Imagery Using EEG: A Literature Review." Biomedical Materials & Devices (2023): 1–13.

39. Qi Lin, Zhang Haoran, Cao Xuehao, Lyu Xuyang, Xu Lisheng, Yang Benqiang, et al. "Multi-scale feature fusion convolutional neural network for concurrent segmentation of left ventricle and myocardium in cardiac mr images." Journal of Medical Imaging and Health Informatics 10, no. 5 (2020): 1023–1032.

40. Roy Arunabha M. "A multi-scale fusion CNN model based on adaptive transfer learning for multi-class MI-classification in BCI system." BioRxiv (2022): 2022–03.

41. Shao Xiaorui, and Kim Chang-Soo. "Unsupervised domain adaptive 1D-CNN for fault diagnosis of bearing." Sensors 22, no. 11 (2022): 4156. pmid:35684777

42. Park Cheolsoo, Looney David, Kidmose Preben, Ungstrup Michael, and Mandic Danilo P. "Time-frequency analysis of EEG asymmetry using bivariate empirical mode decomposition." IEEE Transactions on Neural Systems and Rehabilitation Engineering 19, no. 4 (2011): 366–373. pmid:21342855

43. Bashar Syed Khairul, and Mohammed Imamul Hassan Bhuiyan. "Classification of motor imagery movements using multivariate empirical mode decomposition and short time Fourier transform based hybrid method." Engineering science and technology, an international journal 19, no. 3 (2016): 1457–1464.

44. Tang Xianlun, Li Wei, Li Xingchen, Ma Weichang, and Dang Xiaoyuan. "Motor imagery EEG recognition based on conditional optimization empirical mode decomposition and multi-scale convolutional neural network." Expert Systems with Applications 149 (2020): 113285.

45. Fang Yonghui, Chen Minyou, and Zheng Xufei. "Extracting features from phase space of EEG signals in brain–computer interfaces." Neurocomputing 151 (2015): 1477–1485.

46. Amin Syed Umar, Alsulaiman Mansour, Muhammad Ghulam, Mohamed Amine Mekhtiche, and Shamim Hossain M. "Deep Learning for EEG motor imagery classification based on multi-layer CNNs feature fusion." Future Generation computer systems 101 (2019): 542–554.

47. Dai Guanghai, Zhou Jun, Huang Jiahui, and Wang Ning. "HS-CNN: a CNN with hybrid convolution scale for EEG motor imagery classification." Journal of neural engineering 17, no. 1 (2020): 016025. pmid:31476743

48. Ma Xunguang, Wang Dashuai, Liu Danhua, and Yang Jimin. "DWT and CNN based multi-class motor imagery electroencephalographic signal recognition." Journal of neural engineering 17, no. 1 (2020): 016073. pmid:31972552

49. Zhang Ruilong, Zong Qun, Dou Liqian, and Zhao Xinyi. "A novel hybrid deep learning scheme for four-class motor imagery classification." Journal of neural engineering 16, no. 6 (2019): 066004. pmid:31341093

50. Pfurtscheller Gert, Neuper Christa, Schlogl Alois, and Lugger Klaus. "Separability of EEG signals recorded during right and left motor imagery using adaptive autoregressive parameters." IEEE transactions on Rehabilitation Engineering 6, no. 3 (1998): 316–325. pmid:9749909

51. Ang Kai Keng, Zheng Yang Chin Chuanchu Wang, Guan Cuntai, and Zhang Haihong. "Filter bank common spatial pattern algorithm on BCI competition IV datasets 2a and 2b." Frontiers in neuroscience 6 (2012): 39. pmid:22479236

52. Tabar Yousef Rezaei, and Halici Ugur. "A novel deep learning approach for classification of EEG motor imagery signals." Journal of neural engineering 14, no. 1 (2016): 016003. pmid:27900952

53. Lawhern Vernon J., Solon Amelia J., Waytowich Nicholas R., Gordon Stephen M., Hung Chou P., and Lance Brent J. "EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces." Journal of neural engineering 15, no. 5 (2018): 056013. pmid:29932424

54. Schirrmeister Robin Tibor, Jost Tobias Springenberg Lukas Dominique Josef Fiederer, Glasstetter Martin, Eggensperger Katharina, Tangermann Michael, et al. "Deep learning with convolutional neural networks for EEG decoding and visualization." Human brain mapping 38, no. 11 (2017): 5391–5420 pmid:28782865

55. Sakhavi Siavash, Guan Cuntai, and Yan Shuicheng. "Learning temporal information for brain-computer interface using convolutional neural networks." IEEE transactions on neural networks and learning systems 29, no. 11 (2018): 5619–5629. pmid:29994075

56. Kevric Jasmin, and Subasi Abdulhamit. "Comparison of signal decomposition methods in classification of EEG signals for motor-imagery BCI system." Biomedical Signal Processing and Control 31 (2017): 398–406.

57. Ghumman Mandeep Kaur, Singh Satvir, Singh Navtej, and Jindal Balkrishan. "Optimization of parameters for improving the performance of EEG-based BCI system." Journal of Reliable Intelligent Environments 7 (2021): 145–156.

58. Sharma Rahul, Kim Minju, and Gupta Akansha. "Motor imagery classification in brain-machine interface with machine learning algorithms: Classical approach to multi-layer perceptron model." Biomedical Signal Processing and Control 71 (2022): 103101.

59. Roy Arunabha M. "An efficient multi-scale CNN model with intrinsic feature integration for motor imagery EEG subject classification in brain-machine interfaces." Biomedical Signal Processing and Control 74 (2022): 103496.

60. Roy Arunabha M. "Adaptive transfer learning-based multiscale feature fused deep convolutional neural network for EEG MI multiclassification in brain–computer interface." Engineering Applications of Artificial Intelligence 116 (2022): 105347.

61. Xie Yu, and Oniga Stefan. "Classification of Motor Imagery EEG Signals Based on Data Augmentation and Convolutional Neural Networks." Sensors 23, no. 4 (2023): 1932. pmid:36850530