Muskit: Open-source music processing toolkits

Muskit is an open-source music processing toolkit. Currently we mostly focus on benchmarking the end-to-end singing voice synthesis and expect to extend more tasks in the future. Muskit employs pytorch as a deep learning engine and also follows ESPnet and Kaldi style data processing, and recipes to provide a complete setup for various music processing experiments. The main structure and base codes are adapted from ESPnet (we expect to merge the Muskit into ESPnet in later stages)

News

The project is current merging to ESPnet! If you have any comments and suggestions, please feel free to discuss either in this repo or espnet. See espnet/espnet#4437 for details.

We just start to merge the first version to ESPnet! With some improvements in the fundational data structure, we even reach better performances than the original Muskits. Please stay tuned for our later news!

Key Features

ESPnet style complete recipe

Support numbers of SVS recipes in several databases (e.g., Kiritan, Oniku_db, Ofuton_db, Natsume database, CSD database)
On the fly feature extraction and text processing

SVS: Singing Voice Synthesis

Reproducible results in serveral SVS public domain copora
Various network architecutres for end-to-end SVS
- RNN-based non-autoregressive model
- Xiaoice
- Sequence-to-sequence Transformer (with GLU-based encoder)
- MLP singer
- Tacotron-singing (in progress)
- DiffSinger (to be published)
Multi-speaker & Multilingual extention
- Speaker ID embedding
- Language ID embedding
- Global sytle token (GST) embedding
Various language support
- Jp / En / Kr / Zh
Integration with neural vocoders
- the style matches the PWG repo with supports of various of vocoders

Installation

The full installation guide is available at https://github.com/SJTMusicTeam/Muskits/wiki/Installation-Instructions

Demonstration

Real-time SVS demo with Muskits

pppku/Muskits