Setup.
$ conda env create -f environment.yaml
$ conda activate pyflink-lightgbm
Train. This will produce a trained model and save it as model.txt.
$ python train.py
Infer with vanilla Python.
$ python vanilla_infer.py
Infer with PyFlink.
$ # Spin up Flink locally
$ ~/softwares/flink-1.13.0/bin/start-cluster.sh
$ # Prepare resources needed for a PyFlink run
$ (cd /usr/local/anaconda3/envs/pyflink-lightgbm && zip -r - .) > pyenv.zip
$ zip archive.zip model.txt
$ # Submit to Flink cluster. Open http://localhost:8081/ to check
$ ~/softwares/flink-1.13.0/bin/flink run -d -pyexec pyflink-lightgbm/bin/python -pyarch archive.zip,pyenv.zip#pyflink-lightgbm -py pyflink_infer.py