In this repo you can find the code for Demo on how to use Kubeflow pipelines with Katib and MlFlow in Charmed Kubeflow. The demo was executed on AWS EC2 instance.
You need to have a working instance on kubeflow. If you need help here is my YouTube tutorial on how to deploy Charmed Kubeflow on AWS EC2 instance (eta 30 mins).
This tutorial comes with example on how to use MlFLow as part of Kubeflow pipeline as a model registry. You can install the Charmed MlFlow along Kubeflow with following steps:
juju deploy mlflow-server
juju deploy charmed-osm-mariadb-k8s mlflow-db
juju relate minio mlflow-server
juju relate istio-pilot mlflow-server
juju relate mlflow-db mlflow-server
juju relate mlflow-server admission-webhook
You can find more info about Charmed MlFlow here
If you want you can install all the requirements from requirements.txt
.
OPTIONAL (you can use my image if you want): The hearth of the solution is the kfp_katib_mlflow_demo.ipynb
before running the notebook make sure to create the docker image with docker build -t <your_acc>/kubeflow-training:latest .
and push it to registry docker push <your_acc>/kubeflow-training:latest
after that replace my image (misohu/kubeflow-training:latest
) with yours.