XHEC MLOPS CRASH COURSE

The purpose of this course is to introduce the main concepts of Machine Learning Operations, or MLOps for short, and illustrate their interest and usage through a simple use case.

This repository aims to simplify as much as possible the setup of the infrastructure required for this course. It will create two docker containers, one to host the Jupyter lab that you are going to use for experiments, and the other to host the mlflow server you will log your experiments into.

Prerequisites

For the best experience, we recommend that you have Docker, bash and Make installed on your machine. Follow the instructions to install them if you don't have them already.

How to set up

Step 1: Installing prerequisites

Let's first make sure you have access to the Docker. If you are not sure, follow the instructions below.

Docker

Check that you have Docker desktop installed in your machine by running:

docker -v

If that is not the case, just follow the official instructions:

For those of you working on Windows, you might need to update Windows Subsystem for Linux. To do so, simply open PowerShell and run:

wsl --update

Once docker is installed, make sure that it is running correctly by running:

docker run -p 80:80 docker/getting-started

If you check the Docker App, you should see a getting started container running. Once you've checked that this works correctly, remove the container via the UI.

Optional

You can also perform these operations directly from the command line, by running docker ps to check the running containers and docker rm -f [CONTAINER-ID] to remove it.

Bash

Check that you have bash installed in your machine by opening a terminal and running:

bash --version

This should be the case for all Mac and Linux users.

If you are using Windows, you can use the Git Bash terminal that comes with Git. You can download it here.

Make

Check that you have make installed in your machine by opening a terminal and running:

make -v

This should be the case for all Mac and Linux users.

If you are using Windows, you can then install make using Chocolatey. You can download Chocolatey here.

Then, restart Git Bash and run:

choco install make

Restart Git Bash again and check that make is installed by running:

make -v

If you did not manage to install make, you can still follow the course by following the instructions in the next section under the label "Without make and bash installed on your terminal".

Step 2: Prepare the course infrastructure

Now, we will guide you to build the required Docker Images. With a terminal, go to the root of this folder.

First check that you have make installed by running make -v

If you have `make` and `bash` installed on your terminal

Then simply run:

make prepare-mlops-crashcourse

Without `make` and `bash` installed on your terminal

Please run:

docker build -t mlops_notebooks ./lesson/
docker build -t mlops_mlflow ./mlflow_server/

Step 3: Launch the course

Follow these steps to mount the two containers and open the user interface to start the course.

If you have `make` and `bash` installed on your terminal

You can directly bundle all this section's commands by typing:

make launch-mlops-crashcourse

Without `make` and `bash` installed on your terminal

First, create a network:

docker network create --driver bridge mlops-crashcourse

Then:

docker run -it --rm --user root -p 10000:8888 -p 8000:8000 -p 4200:4200 -v ${PWD}/mlflow_server/local:/mlflow -e JUPYTER_ENABLE_LAB=yes -e JUPYTER_TOKEN=docker -e MLFLOW_TRACKING_URI=http://mlflow:5001 --network mlops-crashcourse --name jupyter -d mlops_notebooks

And:

docker run -it -d --rm -p 5001:5000 -v ${PWD}/mlflow_server/local:/mlflow --network mlops-crashcourse --name mlflow mlops_mlflow

You can then open your favorite browser and open in two tabs the two urls we will be working with:

http://localhost:10000
http://localhost:5001

Once the Jupyter server is launched, you will need to use a token to attach your browser to it. The token you should use is MLOPS.

Step 4: Time to work

All the activities of this course can be found in the notebooks folder. If you are using docker, then you will be working on jupyter lab and the folder will be immediately visible from the root. If instead, you are running this course locally on your computer, then you can find the notebooks under lesson/notebooks.

If you encounter difficulties during, don't hesitate to call us for help

Step 5: Cleanup

Once you are done with the course, you can follow these steps to clean your workspace.

Careful! This will destroy all your work if you did not save it locally.

If you have `make` and `bash` installed on your terminal

Then simply run:
```bash
make clean-mlops-crashcourse
```

Without `make` and `bash` installed on your terminal

Please run

```bash
docker stop jupyter
docker stop mlflow
docker image rm mlops_notebooks
docker image rm mlops_mlflow
docker network rm mlops-crashcourse
```

artefactory/xhec-mlops-crashcourse