With this application, you can better understand how large language models tokenize your text.
Enter your text, select a model, and see how it is tokenized!
text-embedding
text-multilingual-embedding
textembedding-gecko
textembedding-gecko-multilingual
text-bison
text-unicorn
chat-bison
code-gecko
code-bison
codechat-bison
You can try this application online, and select one of the available PaLM-based models, and see how your text is tokenized.