minbpe.c is a Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization in pure C. The project is inspired by minbpe by @kapathy
minbpe.c is a Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization in pure C. The project is inspired by minbpe by @kapathy