Pure python implementation of byte pair encoding (subword tokenization)
Primary LanguagePythonMIT LicenseMIT