Granolar : Granular synth based on machine learning (VAE)
Granolar is a machine learning project that reconstructs grains of sound, it is executable with Python3.
See the requirements folder for the compulsory libraries
pip3 install -r requirements.txtRun
python3 vae_main.pyRun one of the following scripts:
- recover_model.py
- recover_model_batch.py
You need to have pyo installed, and also wxpython to have the pyo gui. Just run:
pip3 install pyo
pip3 install wxpythonThe file simple_OSC_control.pd is a Pure Data patch for sending simple OSC messages to control the synth. To control recover_model.py you need to send one element tuples of floats in the range (0,1). To control recover_model_batch.py you need to send tuples of 16 floats in the range of (0,1).
Attention: Depending on the operating system and how the scripts are run, pyo could not work at all or crash. Running pyo on Windows doesn't work most of the time, depending on the computer. When running from an IDE you are advised to turn off the pyo GUI, as it can crash when you close it.
this archive contains:
- database: folder with the database that will be use for the training
- mnist_vae: folder with the VAE apply to the mnist database
- references: folder with the documents of reference
- saved_model: folder containing the trained model of VAE
- trials_scripts: folder containing all the failed scripts
- loader.py: loader of the database
- loss_train_test.py: compute the loss, train, test and save the model
- vae_main.py: main project script
- vae_model.py: architecture of the model
- comparison.py: plot inputs and outputs, concatenate the output and convert it in .wav
- recover_model.py , recover_model_batch.py: granular synthesis of the VAE decoder's output, controlled by OSC
- simple_OSC_control.pd: Pure Data OSC controller for the synth