/vakyansh-models

Open source speech to text models for Indic Languages

MIT LicenseMIT

Vakyansh Open Source Models

TTS Models (Repo)

Below models are trained using Glow TTS and hifi GAN combination.

Language Gender glow ckpt hifi-gan ckpt
Hindi Female voice_0_glow voice_0_hifi
Hindi Male voice_1_glow voice_1_hifi

Pretrained ASR Models

Pretrained Model Description Architecture Pretrained Hours
CLSRIL-23 Cross Lingual Speech Representations for Indic Languages, Contains 10,000 hours of training data from 23 Indic Languages.
Citation: https://arxiv.org/abs/2107.07402
Base 10,000
hindi_pretrained_4kh Trained on 4200 hours of Hindi Data Base 4200
kannada_pretrained_1400h Trained on 1400 hours of Kannada data XLSR 1400

Finetuned ASR Models (works on v2-hydra branch*)

Language Pretrained Model Finetuned Model Dictionary Single Model for Inference Finetuned Hours TS model
Hindi CLSRIL-23 him_4200 dict hindi_infer 4200 h hindi_ts
Indian English CLSRIL-23 enm_700 dict english_infer 700 h english_ts
Kannada CLSRIL-23 knm_560 dict kannada_infer 560 h kannada_ts
Tamil CLSRIL-23 tam_250 dict tamil_infer 250 h tamil_ts
Bengali CLSRIL-23 bnm_200 dict bengali_infer 200 h bengali_ts
Nepali CLSRIL-23 nem_130 dict nepali_infer 130 h nepali_ts
Telugu CLSRIL-23 tem_100 dict telugu_infer 100 h telugu_ts
Gujarati CLSRIL-23 gum_100 dict gujarati_infer 100 h gujarati_ts
Marathi CLSRIL-23 mrm_100 dict marathi_infer 100 h
Odia CLSRIL-23 orm_100 dict odia_infer 100 h
Sanskrit CLSRIL-23 sam_60 dict sanskrit_infer 60 h
Maithili CLSRIL-23 maim_50 dict maithili_infer 50 h
Urdu CLSRIL-23 urm_60h dict urdu_infer 60h
Punjabi CLSRIL-23 pam_10h dict punjabi_infer 10 h
Dogri CLSRIL-23 doi_55h dict dogri_infer 55 h
Malayalam CLSRIL-23 mlm_8h dict malayalam_infer 8 h
Bhojpuri CLSRIL-23 bhom_60h dict bhojpuri_infer 60 h
Rajasthani CLSRIL-23 raj_45h dict rajasthani_infer 45 h
Assamese CLSRIL-23 asm_8h dict assamese_infer 8 h

Language Models (Works with Finetuned ASR Models)

Data is taken from AI For Bharat Corpus but we do post processing by tokenizing and removing duplicates.

Language Type Lexicon LM
Hindi kenlm 5-gram hindi_lexicon hindi_lm
Indian English kenlm 5-gram english_lexicon english_lm
Kannada kenlm 5-gram kannada_lexicon kannada_lm
Tamil kenlm 5-gram tamil_lexicon tamil_lm
Bengali kenlm 5-gram bengali_lexicon bengali_lm
Nepali kenlm 5-gram nepali_lexicon nepali_lm
Telugu kenlm 5-gram telugu_lexicon telugu_lm
Gujarati kenlm 5-gram gujarati_lexicon gujarati_lm
Marathi kenlm 5-gram marathi_lexicon marathi_lm
Odia kenlm 5-gram odia_lexicon odia_lm
Sanskrit kenlm 5-gram sanskrit_lexicon sanskrit_lm
Maithili kenlm 5-gram maithili_lexicon maithili_lm
Urdu kenlm 5-gram urdu_lexicon urdu_lm
Punjabi kenlm 5-gram punjabi_lexicon punjabi_lm
Dogri kenlm 5-gram dogri_lexicon dogri_lm
Malayalam kenlm 5-gram malayalam_lexicon malayalam_lm
Bhojpuri kenlm 5-gram bhojpuri_lexicon bhojpuri_lm
Rajasthani kenlm 5-gram rajasthani_lexicon rajasthani_lm
Assamese kenlm 5-gram assamese_lexicon assamese_lm

Domain Specific Language Models

Language Type Domain Lexicon LM
English kenlm 5-gram Biomedical bio_lexicon bio_lm


Citation

If you use any of our resources, please cite the following article.

@misc{gupta2021clsril23,
      title={CLSRIL-23: Cross Lingual Speech Representations for Indic Languages}, 
      author={Anirudh Gupta and Harveen Singh Chadha and Priyanshi Shah and Neeraj Chimmwal and Ankur Dhuriya and Rishabh Gaur and Vivek Raghavan},
      year={2021},
      eprint={2107.07402},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}