/Deep-Learning-for-BCI

Resources for Book: Deep Learning for EEG-based Brain-Computer Interface: Representations, Algorithms and Applications

Primary LanguageJupyter NotebookMIT LicenseMIT

Deep Learning for Brain-Computer Interface (BCI)

Update

The whole well-processed and DL-ready data of 109 subjects from EEGMMIDB are uploaded!

Overview

This tutorial contains implementable python and jupyter notebook codes and benchmark datasets to learn how to recognize brain signals based on deep learning models. This tutorial associates our survey on DL-based noninvasive brain signals and book on DL-based BCI: Representations, Algorithms and Applications.

  • We present a new taxonomy of BCI signal paradigms according to the acquisition methods. ECOG: Electrocorticography, EEG: Electroencephalography, fNIRS: functional near-infrared spectroscopy, fMRI: functional magnetic resonance imaging, EOG: Electrooculography, MEG: Magnetoencephalography.

  • We systemically introduce the fundamental knowledge of deep learning models. MLP: Multi-Layer Perceptron, RNN: Recurrent Neural Networks, CNN: Convolutional Neural Networks, LSTM: Long Short-Term Memory, GRU: Gated Recurrent Units, AE: Autoencoder, RBM: Restricted Boltzmann Machine, DBN: Deep Belief Networks, VAE: Variational Autoencoder, GAN: Generative Adversarial Networks. D-AE denotes Stacked-Autoencoder which refers to the autoencoder with multiple hidden layers. Deep Belief Network can be composed of AE or RBM, therefore, we divided DBN into DBN-AE (stacked AE) and DBN-RBM (stacked RBM).

  • Moreover, the guidelines for the investigation and design of BCI systems are provided from the aspects of signal category, deep learning models, and applications. The following figures show the distribution on signals and DL models in frointer DL-based BCI studies.
Distribution on signals Distribution on DL models
Distribution on signals Distribution on DL models



  • Special attention has been given to the state-of-the-art studies on deep learning for EEG-based BCI research in terms of algorithms. Specically, we introduces a number of advanced deep learning algorithms and frameworks aimed at several major issues in BCI including robust brain signal representation learning, cross-scenario classification, and semi-supervised classification.

  • Furthermore, several novel prototypes of deep learning-based BCI systems are proposed which shed light on real-world applications such as authentication, visual reconstruction, language interpretation, and neurological disorder diagnosis. Such applications can dramatically benefit both healthy individuals and those with disabilities in real life.

BCI Dataset

Collection of brain signals is both financially and temporally costly. We extensively explore the benchmark data sets applicable to rain signal research and provides 31 public data sets with download links that cover most brain signal types.

Brain Signals  Dataset #-Subject #-Classes Sampling Rate (Hz) #-Channels Download Link
FM EcoG BCI-C IV, Data set IV 3 5 1000 48 -- 64 Link
MI EcoG BCI-C III 
Data set I
1 2 1000 64 Link
Sleeping EEG Sleep-EDF Telemetry 22 6 100 2 EEG, 1 EOG, 1 EMG Link
Sleeping EEG Sleep-EDF: Cassette 78 6 100, 1 2 EEG (100 Hz), 1 EOG (100 Hz),
1 EMG (1 Hz)
Link
Sleeping EEG MASS-1 53 5 256 17/19 EEG, 2 EOG, 5 EMG Link
Sleeping EEG MASS-2 19 6 256 19 EEG, 4 EOG, 1EMG Link
Sleeping EEG MASS-3 62 5 256 20 EEG, 2 EOG, 3 EMG Link
Sleeping EEG MASS-4 40 6 256 4 EEG, 4 EOG, 1 EMG Link
Sleeping EEG MASS-5 26 6 256 20 EEG, 2 EOG, 3 EMG Link
Sleeping EEG SHHS 5804 N/A 125, 50 2 EEG (125 Hz), 1EOG (50 Hz),
1 EMG (125 Hz)
Link
Seizure EEG CHB-MIT 22 2 256 18 Link
Seizure EEG TUH 315 2 200 19 Link
MI EEG EEGMMI 109 4 160 64 Link
MI EEG BCI-C II, Data set III 1 2 128 3 Link
MI EEG BCI-C III, Data set III a 3 4 250 60 Link
MI EEG BCI-C III, Data set III b 3 2 125 2 Link
MI EEG BCI-C III, Data set IV a 5 2 1000 118 Link
MI EEG BCI-C III, Data set IV b 1 2 1001 119 Link
MI EEG BCI-C III, Data set IV c 1 2 1002 120 Link
MI EEG BCI-C IV, Data set I 7 2 1000 64 Link
MI EEG BCI-C IV, Data set II a 9 4 250 22 EEG, 3 EOG Link
MI EEG BCI-C IV, Data set II b 9 2 250 3 EEG, 3 EOG Link
Emotional EEG AMIGOS 40 4 128 14 Link
Emotional EEG SEED 15 3 200 62 Link
Emotional EEG DEAP 32 4 512 32 Link
Others EEG Open MIIR 10 12 512 64 Link
VEP BCI-C II, Data set II b 1 36 240 64 Link
VEP BCI-C III, Data set II 2 26 240 64 Link
fMRI ADNI 202 3 N/A N/A Link
fMRI BRATS 65 4 N/A N/A Link
MEG BCI-C IV, Data set III 2 4 400 10 Link

In order to let the readers have a quick access of the dataset and can play around it, we provide the well-processed and ready-ro-use dataset of EEG Motor Movement/Imagery Database (EEGMMIDB). This dataset contains 109 subjects while the EEG signals are recorded in 64 channels with 160 Hz sampling rate. After our clearning and sorting, each npy file represents a subject: the data shape of each npy file is [N, 65], the first 64 columns correspond to 64 channel features, the last column denotes the class label. The N varies for different subjects, but N should be either 259520 or 255680. This is the inherent difference in the original dataset.

Running the code

In our tutorial files, you will learn the pipline and workflow of BCI system including data acquiction, pre-processing, feature extraction (optional), classification, and evaluation. We present necessary references and actionable codes of the most typical deep learning models (GRU, LSTM, CNN, GNN) while taking advantage of temporal, spatial, and topographical depencencies. We also provide python codes that are very handy. For example, to check the EEG classification performance of CNN, run the following code:

python 4-2_CNN.py 

For PyTorch beginners, we highly recommond Morvan Zhou's PyTorch Tutorials.

Chapter resources

For the algorithms and applications introduced in the book, we provide the necessary implementary codes (TensorFlow version):

Citing

If you find our research is useful for your research, please consider citing our survey or book:

@article{zhang2020survey,
  title={A survey on deep learning-based non-invasive brain signals: recent advances and new frontiers},
  author={Zhang, Xiang and Yao, Lina and Wang, Xianzhi and Monaghan, Jessica JM and Mcalpine, David and Zhang, Yu},
  journal={Journal of Neural Engineering},
  year={2020},
  publisher={IOP Publishing}
}

@book{zhang2021deep,
  title={Deep Learning for EEG-based Brain-Computer Interface: Representations, Algorithms and Applications},
  author={Zhang, Xiang and Yao, Lina},
  year={2021},
  publisher={World Scientific Publishing}
}

Requirements

The tutorial codes are tested to work under Python 3.7.

Recent versions of Pytorch, torch-geometric, numpy, and scipy are required. All the required basic packages can be installed using the following command: ''' pip install -r requirements.txt ''' Note: For toch-geometric and the related dependices (e.g., cluster, scatter, sparse), the higher version may work but haven't been tested yet.

Miscellaneous

Please send any questions you might have about the code and/or the algorithm to xiang_zhang@hms.harvard.edu.

License

This tutorial is licensed under the MIT License.