subword-segmentation

There are 11 repositories under subword-segmentation topic.

aalto-speech/morfessor
Morfessor is a tool for unsupervised and semi-supervised morphological segmentation
Language:Python185 23 1829
aalto-speech/flatcat
Morfessor FlatCat
Language:Python12 11 46
Waino/morfessor-emprune
Morfessor EM+Prune
Language:Python10 4 02
majeek/vml-hd
Parsing and subword segmentation code for the VML-HD Dataset
Language:Python6 1 00
BassaniRiccardo/ICEBERT
ICEBERT: Interlingual-Clusters Enhanced BERT. A BERT-like model trained on clusters of monolingual subwords.
Language:Python3 0 00
Waino/morfessor-cognates
Cognate-aware morphological segmentation
Language:Python2 1 01
aalto-speech/morfessor-emprune
Morfessor EM+Prune
Language:Python1 2 0
dlsucelt/Cellar
Central repository with pretrained models for transfer learning, BPE subword-tokenization, mono/multilingual embeddings, and everything in between.
Language:Python0 1 01
TiMauzi/dawg
The concept of DAWGs is based on: Blumer, A. et al. (1985). The smallest automation recognizing the subwords of a text. Theoretical Computer Science, 40, 31–55.
Language:Java0 1 00
aalto-speech/morfessor-demo
Morfessor demonstration
Language:Python6 1
JoyeBright/FT-IWSLT2014-BPEVocab
Repository for the experiments in my paper: "A Systematic Analysis of Vocabulary and BPE Settings for Optimal Fine-tuning of NMT: A Case Study of In-domain Translation "
2 0