[Bug Request]: Update "tokenizers" dependency
etsien opened this issue · 2 comments
etsien commented
What happened?
The tokenizers dependency is too stringent, causing package issues when installing.
Is there a way to bump the dependency?
cohere 5.5.0 depends on tokenizers<0.20 and >=0.19
fastembed 0.2.7 depends on tokenizers<0.16 and >=0.15
What Python version are you on? e.g. python --version
Python 3.11
using conda, pip install -r requirements.txt
Version
0.2.7 (Latest)
What os are you seeing the problem on?
Windows
Relevant stack traces and/or logs
No response
joein commented
Hi, we have already fixed it in main branch, we’re going to release it soon