Scaling Zero Shot learning from Hugging Face to production with Pachyderm.
Hugging Face came out with this, and I wanted to scale it with Pachyderm.
docker run -v `pwd`/data/:/data/ --entrypoint=python3 jimmywhitaker/zero-shot:v0.1 zs_predict.py --sequences /data/input/test_input.txt --labels /data/labels/test_labels.txt --output /data/output/
Start a Pachyderm cluster with Pachyderm Hub.
Run:
make zero-shot-base
$ python zs_predict.py --help
usage: zs_predict.py [-h] [--sequences DIR] [--labels DIR] [--output DIR]
Zero Shot Predictor
optional arguments:
-h, --help show this help message and exit
--sequences DIR input sequences to be classified
--labels DIR labels to be applied to sequences
--output DIR output directory for predictions