Welcome to the Hebrew Clone Voice project, featuring "Robo-shaul"! This project uses advanced speech synthesis technology to create a lifelike Hebrew voice that closely mimics the unique vocal qualities of Shaul Amsterdamski. Using a combination of Tacotron2 and HifiGAN, I've created a state-of-the-art voice synthesis system that delivers unparalleled realism and clarity.
I would like to express my deepest gratitude to the talented individuals who contributed to the development of this project. Special thanks go to Nir Geller @Nir Geller, who provided invaluable expertise in coding and implementing the automation with the transliteration. I also extend my thanks to Moshe Moyal @1moshe135, who generously provided his skills in creating the 3D model of the Shaul Amsterdamski, and to Daniel Ophir, who contributed his expertise in animation. Without their hard work and dedication, this project would not have been possible. I am truly grateful for their contributions and proud to have worked with such talented individuals.
Want to know how to do the magic? Skip to the "How To Do" part.
default.mov
"גבי אל תכניס לי מילים לפה"
I used Tacotron2 and HifiGAN for the TTS model.
Tacotron2 is a neural network-based text-to-speech system that generates speech from written text. The system is based on a sequence-to-sequence architecture, which takes as input a sequence of text characters and outputs a sequence of audio samples that correspond to the text. Tacotron2 is trained using a large corpus of audio recordings and their corresponding transcripts, which allows the system to learn the mapping between text and speech.
HifiGAN is a generative adversarial network (GAN) that is used to enhance the quality of audio generated by Tacotron2. The output of Tacotron2 can sometimes sound robotic or artificial, but HifiGAN can be used to add natural-sounding audio details such as background noise, reverb, and pitch variations. HifiGAN is trained on a large dataset of high-quality audio recordings, which allows it to learn how to generate realistic audio samples that closely match the input from Tacotron2.
In the Hebrew Clone Voice project, I've combined Tacotron2 and HifiGAN to create a system that generates high-quality, lifelike Hebrew speech that sounds just like the Shaul character. my system is trained on a large corpus of Hebrew audio recordings, which allows it to accurately capture the unique phonetic features of the Hebrew language. With my system, you can now create your own Hebrew Clone Voice for any text you input.
If you want to use my training model skip this part.
- Download the dataset
- Download the jupyter notebook and put in one folder the notebook and "metadata.xlsx"
- run the "Training Notebook" while the zip wavs in your drive, and paste the list.txt in the notebook
-
Download all the "interface" folder on your pc and run "interface.py".
-
Run the "Synthesis Notbook" and put your trained model id (The default is the trained model I made)
The full 3D model will be in the day of the compotition