tensorflow/text

Recommendation: GemmaTokenizer requires tensorflow-text package (Python 3.12)

Opened this issue · 2 comments

josnin commented

I would like to recommend adding support for Python 3.12 to the TensorFlow Text library. As Python 3.12 is a relatively new release, it would be beneficial to ensure compatibility with this version.

I am encountering an error when using the GemmaTokenizer. The error message states:

 Exception encountered: GemmaTokenizer requires the tensorflow-text package. Please install with pip install tensorflow-text.
davepickup commented

I would also like to see tensorflow-text support Python 3.12. Tensorflow>=2.16 already supports 3.12 so it seems it would make sense to have the same compatibility in the tensorflow-text package.

  • 👍1
nicechester commented

Well, I managed to compile tensorflow-text with python 3.12 on aarch64. You should be able to to that if you followed the instruction on your OS.
This is the resulting binary. https://github.com/nicechester/tensorflow-docker/blob/main/wheels/tensorflow_text-2.16.1-cp312-cp312-linux_aarch64.whl