Temporary repo for building 3.13 wheel for sentencepiece until the new version comes out. Will be deleted when sentencepiece finally provides 3.13 wheels. (Please!)
ARCHIVED: sentencepience finally released 0.2.1!!!
I do not know a lick about sentencepiece inner workings or code. This is for my usage. Use wheels at your own risk!
Patches are applied on top of the sentencepiece repo that is checked out on the v0.2.0 tag.
- Add missing in sentencepiece_processor.h
- Bump CMake minimum required version to avoid warnings
- Fix crash in unigram model training
sdist compiled using Github Actions on this repo still has an error when trying to install. Looking at the source code, when installing the python-sentencepiece build script, it will fetch a fresh copy of the main sentencepiece repo. That means the repo is clear without the above patches.
Download the wheel for your platform from the Release or install it using my python-index.
pip install sentencepiece --extra-index-url https://mddanish00.github.io/python-index/simple
uv add sentencepiece --index https://mddanish00.github.io/python-index/simple
uv pip install sentencepiece --index https://mddanish00.github.io/python-index/simple
google/sentencepiece is licensed under Apache License 2.0.