Could we update Tokenizers version dependency to ^0.19 from ^0.15?
Closed this issue · 2 comments
rishabgit commented
What happened?
Not a bug but rather a request -
Currently, fastembed requires an old version (>=0.15,<0.16) which also conflicts with other libraries such as cohere's python SDK
Latest tokenizers version is 0.19.1
What Python version are you on? e.g. python --version
Python 3.10.14, Poetry
Version
0.2.7 (Latest)
What os are you seeing the problem on?
Linux
Relevant stack traces and/or logs
No response
joein commented
Hey @rishabgit , yeah, sure, #244