seqtolang
is a python library for multi-langauge documents identification.
See this post for implementation details.
Install from source:
$ git clone https://github.com/hiredscorelabs/seqtolang
$ cd seqtolang
$ python setup.py install
from seqtolang import Detector
detector = Detector()
text = "In Chinese, the French phrase 'Je rentre chez moi Je rentre chez moi' will be '我正在回家'"
languages = detector.detect(text)
print(languages)
>>> [('fr', 0.499), ('en', 0.437), ('zh', 0.062)]
tokens = detector.detect(text, aggregated=False)
print(tokens)
>>> ['en', 'en', 'en', 'en', 'en', 'fr', 'fr', 'fr', 'fr', 'fr', 'fr', 'fr', 'fr', 'en', 'en', 'zh']
seqtolang
support 36 languages:
['afr', 'eus', 'bel', 'ben', 'bul', 'cat', 'zho', 'ces', 'dan', 'nld', 'eng', 'est', 'fin', 'fra',
'glg', 'deu', 'ell', 'hin', 'hun', 'isl', 'ind', 'gle', 'ita', 'jpn', 'kor', 'lat', 'lit', 'pol',
'por', 'ron', 'rus', 'slk', 'spa', 'swe', 'ukr', 'vie']
You can ask questions and join the development discussion on Github Issues
Apache License 2.0