ashwath007/minbpe
Minimal, clean, educational code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.
PythonMIT
Minimal, clean, educational code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.
PythonMIT