/mlops-demo-test

Test demo for MLOps pipeline

Primary LanguagePython

MLOps CI/CD Demo

This repository demoes CI/CD automation for ML. The project uses dvc to build and test a model in a reproducible way.

Setup

  1. Fork and clone this repository.
  2. Install dvc on your machine. Check you have Python installed.
  3. Create a virtualenv: 
    $ python -m venv venv
$ source venv/bin/activate
  4. Install dependencies: 
    $ pip install -r requirements.txt

Training the model

To download the training dataset and train the model run:

$ dvc repro

This will create the model file in the models directory

You can try the model's inference with:

python scripts/predict.py

The metrics should now contain benchmarks of the model in plot/png and CSV formats.

ML Stages

The DVC pipeline contains the following stages:

  1. download: download traininig dataset
  2. validate: validate dataset
  3. prepare: prepare dataset
  4. train: train the model
  5. evaluate: evaluate model
  6. metrics: calculate model error

API Server

The project includes an Flask API server. You can start it with

$ cd api
$ python server.py

Docker image

You can build a Docker image with the model and API server with:

$ docker build -t california-housing .

Try running the build image with:

$ docker run -it -p 8080:8080 california-housing

Deploy to Fly.io

To create the app for the first time:

  1. Create a free Fly.io account.
  2. Install flyctl
  3. Run fly launch

Once you have launched the application once, you can setup Semaphore to do continuous delivery:

  1. Open your Fly.io account and create an access token.
  2. Create the secret fly-deploy with the env variable FLY_TOKEN
  3. Edit the CI/CD Pipeline. Go to the last pipeline and add your secret to the block.
  4. Save the pipeline.

LICENSE