NeuroSimulator

This repository allows for the connection of a cart game on Unity with EEG signals acquired through a headset compatible with the OpenVIBE Acquisition Server. A basic architecture of the working of the system is shown below.

Requirements

The requirements for using this project are:

Python 3.9
Tensorflow 2.8
Unity 2022
OpenVIBE 3.2.0

Specifications for the Incoming Signal

The EEG signal this system is trained for is of:

Sampling Frequency of 160Hz
16 electrode channels

Training of Model

You can use the pre-trained model for this, or you can train your own with either your data or the one we used which is publically available at: https://drive.google.com/drive/folders/11tCrbFUudiq6_ADMQRNwThn3n-eYqREv?usp=sharing

How to use

Follow the following steps

Add "Streaming_and_Classification.py", "Interface_With_Agent.cs" and "Pre-Trained Model" (or your own saved model if you have trained it yourself) in the scripts directory of your agent (prefrably a carting agent).
Turn on the OpenVIBE Aquisition Server and adjust the parameters to your choosing and select the used driver after connecting the EEG headset.
Run "LSL_Exporting.xml"
Run the game
Run "Streaming_And_Classification.py"
Enjoy!

Results

We ran this on a wheelchair simulator game, available at https://github.com/zeerakt/EEGCart We had the following results on the Physics Simulation.

Moving Forward

- Turning Left

- Turning Right

Credits

To https://github.com/hauke-d for the CNN Model
To https://github.com/CanYouCatchMe01 for the Socket Communication
To https://github.com/sccn for the Lab Streaming Layer