/kubeflow-katib-mlflow-demo

Kubeflow Pipelines Katib MlFlow

Primary LanguageJupyter Notebook

Kubeflow Pipeline demo

In this repo you can find the code for Demo on how to use Kubeflow pipelines with Katib and MlFlow in Charmed Kubeflow. The demo was executed on AWS EC2 instance.

Prerequisites

You need to have a working instance on kubeflow. If you need help here is my YouTube tutorial on how to deploy Charmed Kubeflow on AWS EC2 instance (eta 30 mins).

This tutorial comes with example on how to use MlFLow as part of Kubeflow pipeline as a model registry. You can install the Charmed MlFlow along Kubeflow with following steps:

juju deploy mlflow-server
juju deploy charmed-osm-mariadb-k8s mlflow-db
juju relate minio mlflow-server
juju relate istio-pilot mlflow-server
juju relate mlflow-db mlflow-server
juju relate mlflow-server admission-webhook

You can find more info about Charmed MlFlow here

If you want you can install all the requirements from requirements.txt.

Explanation

OPTIONAL (you can use my image if you want): The hearth of the solution is the kfp_katib_mlflow_demo.ipynb before running the notebook make sure to create the docker image with docker build -t <your_acc>/kubeflow-training:latest . and push it to registry docker push <your_acc>/kubeflow-training:latest after that replace my image (misohu/kubeflow-training:latest) with yours.