Choosing the LID model
vienneraphael opened this issue · 0 comments
vienneraphael commented
Available models
- avoid lid.218 (original nllb LID) -> it's CC-BY-NC
- https://github.com/laurieburchell/open-lid-dataset -> GPL License, 201 lang
- https://fasttext.cc/docs/en/language-identification.html -> lid.176, CC-BY-SA license, 176 lang
Additional informations
- https://huggingface.co/hac541309/fasttext_langID_models -> models and hashes