Official implementation of RAVE: A variational autoencoder for fast and high-quality neural audio synthesis (article link) by Antoine Caillon and Philippe Esling.
If you use RAVE as a part of a music performance or installation, be sure to cite either this repository or the article !
We propose a Google Colab handling the training of a RAVE model on a custom dataset !
RAVE needs python 3.9. Install the dependencies using
pip install -r requirements.txtDetailed instructions to setup a training station for this project are available here.
RAVE comes with two command line utilities, resample and duration. resample allows to pre-process (silence removal, loudness normalization) and augment (compression) an entire directory of audio files (.mp3, .aiff, .opus, .wav, .aac). duration prints out the total duration of a .wav folder.
Both RAVE and the prior model are available in this repo. For most users we recommand to use the cli_helper.py script, since it will generate a set of instructions allowing the training and export of both RAVE and the prior model on a specific dataset.
python cli_helper.pyHowever, if you want to customize even more your training, you can use the provided train_{rave, prior}.py and export_{rave, prior}.py scripts manually.
Once trained, you can reconstruct an entire folder containing wav files using
python reconstruct.py --ckpt /path/to/checkpoint --wav-folder /path/to/wav/folderYou can also export RAVE to a torchscript file using export_rave.py and use the encode and decode methods on tensors.
UPDATE
If you want to use the realtime mode, you should update your dependencies !
pip install -r requirements.txtRAVE and the prior model can be used in realtime on live audio streams, allowing creative interactions with both models.
RAVE is compatible with the nn~ max/msp and PureData external.
An audio example of the prior sampling patch is available in the docs/ folder.
You can also use RAVE as a VST audio plugin using the RAVE vst !
If you have questions, want to share your experience with RAVE or share musical pieces done with the model, you can use the Discussion tab !
Demonstration of what you can do with RAVE and the nn~ external for maxmsp !
Using nn~ for puredata, RAVE can be used in realtime on embedded platforms !
