/machine-learning-lab

ML Lab enables teams to be more productive in delivering machine learning solutions for their products and datasets.

Primary LanguagePythonApache License 2.0Apache-2.0

End-to-end collaborative development platform to build and run machine learning solutions.

Getting Started • Documentation • Support • Report a Bug • FAQ • Contributing • Changelog

ML Lab is a centralized hub for development teams to seamlessly build, deploy, and operate machine learning solutions at scale. It is designed to cover the end-to-end machine learning lifecycle from data processing and experimentation to model training and deployment. It combines the libraries, languages, and tools data scientists love, with the infrastructure, services and workflows they need to deliver machine learning solutions into production.

Highlights

  • 🔐 Secure multi-user development plaform for machine learning solutions.
  • 🛠 Workspace with integrated tooling (Jupyter, VS Code, SSH, VNC, Hardware Monitoring, ...)
  • 🗃️ Upload, manage, version, and share datasets & models.
  • 🔬 Monitor and share experiments for reproducability.
  • 🎛 Deploy and operate machine learning solutions for productive usage.
  • 🐳 Deployable on a single-server via Docker or a server-cluster via Kubernetes.

Getting Started

Deploying ML Lab in a single-host deployment (via Docker) is as simple as:

docker run --rm \
           --env LAB_ACTION=install \
           --env LAB_PORT=8080 \
           --env BACKEND_SERVICE_IMAGE=docker.pkg.github.com/sap/machine-learning-lab/lab-service:0.1.0 \
           --env MODEL_SERVICE_IMAGE=docker.pkg.github.com/sap/machine-learning-lab/lab-model-service:0.1.0 \
           --env WORKSPACE_IMAGE=docker.pkg.github.com/sap/machine-learning-lab/ml-workspace-lab:0.1.0 \
           -v /var/run/docker.sock:/var/run/docker.sock \
           docker.pkg.github.com/sap/machine-learning-lab/lab-service:0.1.0

# The following two commands are needed since the GitHub package registry only works when being logged in; even for pulling.
# See note below.
docker pull docker.pkg.github.com/sap/machine-learning-lab/lab-model-service:0.1.0
docker pull docker.pkg.github.com/sap/machine-learning-lab/ml-workspace-lab:0.1.0

Note: We deployed the current version to the GitHub package repository which requires the image environment variables to be set; that might change in the future. Even to pull public images, you need to login to GitHub package registry as described here.

Voilà, that was easy! Now, Docker will pull the required Docker images to your machine. After the installation is finished, visit http://localhost:8080 and login with admin:admin (please change the admin password from the user menu).

Please refer to our documentatation for further information on deploying ML Lab for productive usage, including information about configuration options, Kubernetes deployment, adminisrative task and general usage.

Known Issues

For known issues and other questions, check out the documentation.

Support & Feedback

The ML Lab project is maintained by Benjamin Räthlein and Lukas Masuch. Please understand that we won't be able to provide individual support via email. We also believe that help is much more valuable if it's shared publicly so that more people can benefit from it.

Type Channel
🚨 Bug Reports
🎁 Feature Requests
👩‍💻 Usage Questions tbd
🗯 General Discussion tbd

Documentation

Please refer to our documentatation for information about productive deployment, configuration options, adminisrative tasks, architecture, and general usage.

Development

Requirements

  • Java >= 8, Python >= 3.6, Npm >= 6.4, Maven, Docker

Build

Execute this command in the project root folder to build this project and the respective docker container:

python build.py

This script compiles the project, assembles the various JAR artifacts (executable service, client, sources) and builds a docker container with the assembled executable jar. For additional script options:

python build.py --help

Deploy

Execute this command in the project root folder to push all docker containers to the configured docker registry:

python build.py --deploy --version={MAJOR.MINOR.PATCH-TAG}

For deployment, the version has to be provided. The version format should follow the Semantic Versioning standard (MAJOR.MINOR.PATCH). For additional script options:

python build.py --help

Configuration

For more configuration details, check out the documentation.


Contributing