Description: This project aims to design and implement a comprehensive machine learning (ML) system to manage the entire lifecycle of an ML model, including experiment tracking, model training, parameter tuning, model deployment, and monitoring. The project will utilize MLflow, an open- source platform to manage the end-to-end machine learning lifecycle. Students will select a specific domain (e.g., healthcare, finance, retail) and dataset to develop a predictive model and then use MLflow to manage the various stages of the ML project lifecycle.
Objectives:
- Experiment Tracking: Implement and demonstrate how MLflow can be used to track different experiments, including logging parameters, metrics, and outputs.
- Model Training and Tuning: Develop ML models and use MLflow to log different training sessions with varying parameters and hyperparameter tuning processes.
- Model Deployment: Package the trained model using MLflow’s model packaging tools and deploy it as a service for real-time or batch predictions.
- Performance Monitoring: Set up mechanisms to monitor the deployed model's performance over time, utilizing MLflow to track drifts in model metrics.
- Model Registry: Utilize MLflow’s Model Registry to manage model versions and lifecycle including stage transitions like staging and production.
Solution Guide :
- Select Domain and Dataset: Identify a relevant dataset for a specific industry domain. Possible datasets could include image data for a computer vision task in healthcare, financial transaction data for fraud detection, or customer data for predicting churn in telecommunications.
- Setup MLflow: Install and configure MLflow on a local machine or a cloud environment. Setup should include configuring an MLflow tracking server, a database for storing experiment metadata, and a storage space for artifacts.
- Model Development and Tracking: Develop several ML models using popular libraries like Scikit-Learn, TensorFlow, or PyTorch. Use MLflow to log all experiments including parameters, metrics, and model artifacts.
- Hyperparameter Tuning: Use MLflow with a hyperparameter tuning library (like Hyperopt) to optimize model parameters. Track all tuning sessions and results using MLflow.
- Model Deployment and Monitoring: Deploy the best-performing model using MLflow’s model serving capabilities and set up performance monitoring metrics. Document how the model's performance is tracked over time with real or simulated incoming data.
- Use MLflow Model Registry: Demonstrate the use of the MLflow Model Registry to manage and transition model versions from staging to production.
You will present this project in the last 2 weeks lectures, May 20 and May 27
Each presentation will be 5 minutes Deliverables:
- Code Repository: A well-documented codebase hosted on GitHub or similar platform, demonstrating the use of MLflow in managing the ML lifecycle.
- Project Report: A detailed report describing the methodology, tools used, model development process, results of experiments, and insights from the project.
- Presentation: A comprehensive presentation that outlines the project approach, key findings, and a demo of the MLflow setup and model deployment.
This project will give you hands-on experience with MLflow and a deep understanding of managing ML models effectively. It provides a practical, end-to-end view of the ML lifecycle management that is crucial for real-world ML deployments.