/mlops

Repository for the final project of the MLOps course (taught by R. Avouac & L. Galiana during 2nd Semester of the final year at ENSAE Paris).

Primary LanguagePythonMIT LicenseMIT

mlops Construction image Docker

Repository for the final project of the MLOps course (taught by R. Avouac & L. Galiana during 2nd Semester of the final year at ENSAE Paris).


Content

This project basically implement a NLP based recommender system and deploy it though a Streamlit app. The model built follows a very simple approach that combines an embeddings model (all-MiniLM-L6-v2) with cosine similarity to make event recommendations, based on a query. We have tried to make this engine multilingual by adding a translator in the backend.


  • The app folder contains the code and assets that are needed for the app. The code used to build the model can be found in the app/src/models/ folder. In addition, the app/src/utils folder contains a lot of functions that we used to build the app.
  • Finally, in order to deploy the app, we built a Docker image (with entrypoint being the run.shscript). We automated the image delivery thanks to some configuration stuff (k8s/deploymentand argocd folders), hence a new image is being pushed to the DockerHub at every new version of the app.

NB: The data comes from a public website, and can also be directly downloaded from this site. However, we added the data to a S3 bucket, accessible to the SSP Cloud's solution (MinIO). Hence, in our code, we directly use the data that is stored in our bucket.


To access our Streamlit app, that is, the main output of this project, one can access the latest deployed version just by clicking here. Indeed, the app is deployed on a Kubernetes cluster hosted by SSP Cloud.


On the other hand, you might want to run this app locally. This is why we are giving you the following steps to set the project up.

Setup Instructions

General instructions

Firstly, from the command line, you will have to run the following commands:

  1. Clone this repository:
git clone https://github.com/JulesBrable/mlops.git
  1. Go to the app folder:
cd mlops/app

Then choose one of the following options, depending on whether you prefer to use Docker or conda.

Option A: you have Docker installed:

  1. Build the Docker image:
docker build -t mlops .
  1. Run the Docker container:
docker run -p 5000:5000 mlops

Option B: you have conda installed:

  1. Create and activate conda env:
conda create -n mlops python=3.9.16
conda activate mlops
  1. Install the listed dependencies:
pip install -r requirements.txt

Run the app locally:

By default, we are using port 5000, so once you have run the following command, you will be able to access the app with the following link: http://localhost:5000/.

streamlit run Homepage.py --server.port=5000

Further work

To make our work more robust, we could have added a feature that automatically updates events on a weekly or monthly basis. This can be done in Kubernetes using the cronjobfunctionality. We've tried to develop this solution, but we haven't managed to implement it.

Contact