Issues
- 0
Can you offer the `sentencepiece_model.proto`?
#13 opened by workingloong - 2
- 0
Please add tags to this repo corresponding to versions you had published to official pip website
#11 opened by xuanhua - 2
protobuf version is too low
#10 opened by danyang-rainbow - 2
- 0
- 0
what‘s the meaning of token 20005?
#7 opened by xu-song - 6
How did the tokenizer learned?
#4 opened by silverriver - 1
Proper way to truncate long prompts
#3 opened by teetone - 5
Retrieve the value of the end-of-text-token
#1 opened by teetone - 1