/scalable-ml-service

Primary LanguageElixir

Scalable ML Service

Quick start:

  1. Clone this repo
  2. Prepare the docker images 
    docker build -t smls-server ./api_server
docker build -t smls-model-runner ./model_runner
docker build -t smls-frontend-app ./frontend_app
  3. docker-compose up
  4. Navigate to http://localhost:3000/api/graphiql
  5. Start writing french!

The Project: The goal is to build a full architecture to gather sentence pairs in various languages and run the infrastructure to build the machine learned models and provide a translations interface.

The project is purely educational and offers me a change to work with

  • Machine learning
  • Scalable architectures
  • Application development
  • Infrastructure development
  • API development
  • Test development
  • Benchmarking a solution
  • Etc.

It will not, however, offer a training ground for general DevOps, the art of keeping the uptime near 100%, live migrations, continuous integration/deployment etc.

Related blog articles are:

  1. Scalable Machine Learning Service
  2. Productizing Machine Learning Models

Further things to do as educational tasks:

  • Auto Scaling: Currently we statically set the number of instances we run. ideally this would be adjusted depending on load.
  • Asynchronous: Currently the GraphQl interface is synchronous. one makes a query and receives an answer. though for long running tasks this is not optimal and we can push the asynchronous semantics all the way to the client.
  • Multiple models: How would we architect for a system that serves several ML models. Maybe a model per user.
  • Resilience: Currently stuff like back off, timeouts, etc has not been implemented. This would also be needed.
  • Development Setup: Currently it is hard to develop anything as there are strong dependencies.
  • Migrations: Currently the Postgres database is dead weight. However, in a real life situation we need to handle migration etc. as a part of the architecture.
  • Training: Integrate a training pipeline for models based on changes of data og changes of the model they are trained on.
  • Save Predictions: Dave the predictions made to "improve service".

Resources

Thanks to the authors behind following resources.

  1. https://www.rabbitmq.com/tutorials/tutorial-six-elixir.html
  2. https://www.rabbitmq.com/tutorials/tutorial-six-python.html
  3. https://pytorch.org/tutorials/intermediate/seq2seq_translation_tutorial.html

And thanks to the entire community making projects like these possible.