Samuellucas97/mlops

Repository for DCA0305, an undergraduate course about Machine Learning Workflows and Pipelines

Federal University of Rio Grande do Norte

Technology Center

Department of Computer Engineering and Automation

Machine Learning Based Systems Design

References

• 📚 Noah Gift, Alfredo Deza. Practical MLOps: Operationalizing Machine Learning Models [Link]
• 📚 Chip Huyen. Designing Machine Learning Systems: An Iterative Process for Production-Ready Applications. [Link]
• 📚 Hannes Hapke, Catherine Nelson. Building Machine Learning Pipelines. [Link]
• 📚 Mariano Anaya. Clean Code in Python [Link]
• 📚 Aurélien Géron. Hands on Machine Learning with Scikit-Learn, Keras and TensorFlow. [Link]
• 🤜 Dataquest Academic Program [Link]
• 😃 CS329S - ML Systems Design [Link]
• 🎯 Machine Learning Operations [Link]

Lessons

Week 01: Course Outline

• Git and Version Control
  • You'll learn how to: a) organize your code using version control, b) resolve conflicts in version control, c) employ Git and Github to collaborate with others.
  • 👊 U1T1: guided project + getting a git repository.

Week 02: CLI fundamentals

• Elements of the Command Line
  • You'll learn how to: a) employ the command line for Data Science, b) modify the behavior of commands with options, c) employ glob patterns and wildcards, d) define Important command line concepts, e) navigate he filesystem, f) manage users and permissions.
• Text Processing in the Command Line
  • You'll learn how to: a) read and explore documentation, b) perform basic text processing, c) redirect and pipe output, d) inspect files, e) define different kinds of output, f) employ streams and file descriptors.
• 🔠 U1T2: working with command line.

Week 03 - Clean Code Principles for Data Science and Machine Learning

• Outline
• Coding Best Practices
• Writing Clean Code
• Refactoring Code
• Efficient Code
• Documentation
• Python Code Quality Authority (PCQA) - pycodestyle
• PCQA - pylint
• PCQA - autopep8
• PCQA - nbQA
• ▶️ Hands on
  • 💾 Datasets [Link]
  • Writting Clean Code
  • Exercise 01
  • Exercise 02
  • Exercise 03
  • Using pycodestyle
  • Using pylint - script refactored script
  • Functions: Advanced - Best practices for writing functions

Week 04 Production Ready Code

• Outline
• Catching Errors
• Testing and Data Science
• A brief introduction about pytest
• Logging
• Case study: testing and logging
• Model Drift
• Hands on
  • Production ready code
  • Data Visualization Fundamentals
    • You will learn how to: a) how to use data visualization to explore data and b) how and when to use the most common plots.
  • Storytelling Data Visualization and Information Design
    • You will learn how to: a) Create graphs using information design principles, b) create narrative data visualizations using Matplotlib, c) create visual patterns using Gestalt principles, d) control attention using pre-attentive attributes and e) employ Matplotlib's built-in styles.

Week 05 Building a Data Pipeline

• Define functional programming
• Write a robust data pipeline with a scheduler in Python
• Define advanced Python concepts such as closures and decorators

Week 06 Building a Reproducible Model Workflow

• Outline
• Business Reflections
• Introduction to MLOps
• A brief history of MLOps and Tools
• Tools and environment installation
• Tools and environment installation cont.
• Machine Learning Pipelines
• Machine Learning Pipelines - Command Line Interface
• Versioning Data and Artifacts
• Guided Exercise - CLI + Weights and Biases
• MLflow Projects
• Introduction to YAML
• Guided Exercise - Build a MLflow component
• Linking together the components MLflow + Hydra
• Guided Exercise - Multiples MLflow components + Hydra
• Additional Material
  • Context Managers
  • Introduction to Decorators
  • Decorators: advanced

Week 07 Building a Reproducible Model Workflow Cont. - Introduction to Machine Learning

• Outline
• What is Machine Learning?
• Machine Learning Types
• Variables, Pipeline, Controlling Chaos
• Data Segregation - train, dev and test sets
• Bias vs Variance
• 🐇 Machine Learning Fundamentals

Week 08 Building a Reproducible Model Workflow Cont. - ETL, Data Checks, Data Segregation

• Outline
• Extract, Transform, Load (ETL)
  • Exploratory Data Analysis (EDA) Video
    • Part I
    • Part II
    • Part III
    • Part IV
    • Part V
    • Hands on
  • Preprocessing
• Data Segregation
• Data Checks
  • Data Validation
  • Deterministic Test
  • Non-Deterministic Test
  • Multiple Hypothesis Testing
  • Multiple Hypothesis Testing Using MLFlow
  • Multiple Hypothesis Testing Using Parameters in PyTest

Week 09 Building a Reproducible Model Workflow Cont. - Train, Validation and Experiment Tracking

• Outline
• A brief review
• Decision Trees
  • Introduction
  • Mathematical Foundations
• Evaluation Metrics
  • How to choose an evaluation metric?
  • Threshold metrics
  • Ranking metrics
• Implementing Pipelines
  • MLOps Level 0 with Pipeline incorporating train
    • Part I
    • Part II
    • Part III
    • Source-Code
  • MLOps Level 0 with Pipeline incorporating train and preprocessing
    • Part I
    • Part II
    • Part III
    • Source-Code
  • MLOps Level 1 with Pipeline incorporating train and preprocessing
    • Part I
    • Part II
    • Source-Code
  • MLOps Level 1 with Pipeline and Hyper-parameter Tuning
    • Part I
    • Part II
    • Part III
    • Part IV
    • Source-Code
  • Test evaluation
    • Part I
    • Source-Code

Week 10 Building a Reproducible Model Workflow Cont. - Final Pipeline, Release and Deploy

• Outline
• Final Pipeline
  • Big picture of the final pipeline
  • All together
    • Part I
    • Part II
    • Source-Code
• Release for reproducibility
  • Create a GitHub repository for the final pipeline
  • Semantic versioning and remote execution
• Deployment
  • Deploy with MLflow