/ml-kub-project

Primary LanguagePython

CircleCI

Project Overview

In this project, is a pre-trained, sklearn model that has been trained to predict housing prices in Boston according to several features, such as average rooms in a home and data about highway access, teacher-to-pupil ratios, and so on. You can read more about the data, which was initially taken from Kaggle, on the data source site. This project operationalize a Machine Learning Microservice API.

Setup the Environment

  • Create a virtualenv with Python 3.7 and activate it. Refer to this link for help on specifying the Python version in the virtualenv.
python3 -m pip install --user virtualenv
# You should have Python 3.7 available in your host. 
# Check the Python path using `which python3`
# Use a command similar to this one:
python3 -m virtualenv --python=<path-to-Python3.7> .devops
source .devops/bin/activate

Installing

  • Run make install to install the necessary dependencies Note: If using Ubuntu, you may run into an issue with installing the dependencies. If so, run
sudo apt-get update
sudo apt install libblas3 liblapack3 liblapack-dev libblas-dev -y
sudo apt install gfortran -y

Then run make install again.

Linting

  • Install hadolint from here and add addd the necessary permisssions with chmod +x /usr/bin/hadolint
  • Run make lint to lint the project

Testing

  • Run make test to test the project

Running the app

Locally

  • Run python app.py to run the app locally
  • You can then access the app at http://localhost:80

Docker

  • Build and run the docker image: ./run_docker.sh
  • To make a prediction, run ./make_prediction.sh
  • To upload the image to Docker Hub, run ./upload_docker.sh

Kubernetes

  • Setup and Configure Docker locally
  • Setup and and install minikube locally, refer to this link
  • To set up a Kubernetes cluster, run minikube start
  • To deploy the app, run ./run_kubernetes.sh
  • To make a prediction, run ./make_prediction.sh

Files

  • app.py - The main application file
  • Dockerfile - The Dockerfile to build the image
  • Makefile - The Makefile to install dependencies and lint the project
  • make_prediction.sh - The script to make a prediction
  • run_docker.sh - The script to build and run the docker image
  • run_kubernetes.sh - The script to deploy the app to Kubernetes
  • upload_docker.sh - The script to upload the image to Docker Hub
  • requirements.txt - The requirements file to install the dependencies
  • output_txt_files/docker_out.txt - The sample output of the docker container
  • output_txt_files/kubernetes_out.txt - The sample output of the kubernetes pod
  • output_txt_files/prediction.json - The sample prediction
  • model_data/boston_housing_prediction.joblib - The model file
  • model_data/housing.csv - The data file
  • .circleci/config.yml - The CircleCI configuration file