karpathy/build-nanogpt

Avoid tiktoken.decode panic on unknown tokens.

IggShaman opened this issue · 0 comments

In the early training stages, the example generation may produce out-of-bound tokens. The tiktoken.decode panics on those, so make sure it doesn't see them.

PR: #80