MLflow is an open source platform for managing the end-to-end machine learning lifecycle. It tackles four primary functions:
- Tracking experiments to record and compare parameters and results (MLflow Tracking).
- Packaging ML code in a reusable, reproducible form in order to share with other data scientists or transfer to production (MLflow Projects).
- Managing and deploying models from a variety of ML libraries to a variety of model serving and inference platforms (MLflow Models).
- Providing a central model store to collaboratively manage the full lifecycle of an MLflow Model, including model versioning, stage transitions, and annotations (MLflow Model Registry).
- Proper monitoring of drift as well as skew as well verification/validation step(model blessing)
https://databricks.com/blog/2020/11/05/a-guide-to-mlflow-talks-at-data-ai-summit-europe-2020.html
ML Ops (https://neptune.ai/blog/best-mlops-tools)
-
Model metadata storage and management: S3 Bucket
-
Data and pipeline versioning: Zeet
-
Hyperparameter tuning: Sagemaker
-
Run orchestration and workflow pipelines: Kubeflow, Zeet
-
Model deployment and serving: Zeet
-
Production model monitoring: Zeet, Cloudwatch