ArthurZucker/minbpe
Minimal, clean, educational code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.
PythonMIT
No issues in this repository yet.
Minimal, clean, educational code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.
PythonMIT
No issues in this repository yet.