Simple tool for generating tokens with open source transformers and/or calculate per-token surprisal.
Primary LanguagePython