A small project visually showing how different tokenizers split word. Inspired by OpenAI's tool.
Primary LanguageHTML