/weighted_bpe

Python wrapper and c++ code for weighted version of BPE tokenization

Primary LanguageC++

Watchers