/azureml-examples

Official community-driven Azure Machine Learning Examples, tested with GitHub Actions

Primary LanguagePythonMIT LicenseMIT

Azure Machine Learning (AML) Examples

run-workflows-badge run-notebooks-badge cleanup code style: black license: MIT

Welcome to the Azure Machine Learning (AML) examples repository!

Prerequisites

  1. An Azure subscription. If you don't have an Azure subscription, create a free account before you begin.
  2. A terminal and Python >=3.6,<3.9.

Setup

Clone this repository and install required packages:

git clone https://github.com/Azure/azureml-examples --depth 1
cd azureml-examples
pip install --upgrade -r requirements.txt

To create or setup a workspace with the assets used in these examples, run the setup script.

If you do not have an AML Workspace, run python setup-workspace.py --subscription-id $ID, where $ID is your Azure subscription id. A resource group, AML Workspace, and other necessary resources will be created in the subscription.

If you have an AML Workspace, install the AML CLI and run az ml folder attach -w $WS -g $RG, where $WS and $RG are the workspace and resource group names.

Run python setup-workspace.py -h to see other arguments.

Getting started

To get started, see the introductory tutorial which uses AML to:

  • run a "hello world" job on cloud compute, demonstrating the basics
  • run a series of PyTorch training jobs on cloud compute, demonstrating mlflow tracking & using cloud data

These concepts are sufficient to understand all examples in this repository, which are listed below.

Contents

A lightweight template repository for automating the ML lifecycle can be found here.

directory description
.cloud cloud templates
.github GitHub specific files like Actions workflow yaml definitions and issue templates
notebooks interactive jupyter notebooks for iterative ML development
tutorials self-contained directories of end-to-end tutorials
workflows self-contained directories of job to be run, organized by scenario then tool then project

Examples

Tutorials

path status notebooks description
an-introduction an-introduction 1.hello-world.ipynb
2.pytorch-model.ipynb
3.pytorch-model-cloud-data.ipynb
learn the basics of Azure Machine Learning
automl-with-pycaret automl-with-pycaret 1.classification.ipynb learn how to automate ML with PyCaret
deploy-edge deploy-edge ase-gpu.ipynb learn how to use Edge device for model deployment and scoring
deploy-triton deploy-triton 1.densenet-local.ipynb
2.bidaf-aks-v100.ipynb
learn how to efficiently deploy to GPUs using triton inference server
using-dask using-dask 1.intro-to-dask.ipynb
2.dask-cloudprovider.ipynb
learn how to read from cloud data and scale PyData tools (Numpy, Pandas, Scikit-Learn, etc.) with Dask
using-pytorch-lightning using-pytorch-lightning 1.train-single-node.ipynb
2.log-with-tensorboard.ipynb
3.log-with-mlflow.ipynb
4.train-multi-node-ddp.ipynb
learn how to train and log metrics with PyTorch Lightning
using-rapids using-rapids 1.train-and-hpo.ipynb
2.train-multi-gpu.ipynb
learn how to accelerate PyData tools (numpy, pandas, scikit-learn, etc) on NVIDIA GPUs with rapids

Notebooks

path description
notebooks/train-lightgbm-local.ipynb use AML and mlflow to track interactive experimentation in the cloud

Train

path compute environment description
workflows/train/deepspeed/cifar/job.py AML - GPU unknown train CIFAR-10 using DeepSpeed and PyTorch
workflows/train/fastai/mnist-mlproject/job.py AML - CPU mlproject train fastai resnet18 model on mnist data via mlflow mlproject
workflows/train/fastai/mnist/job.py AML - CPU conda train fastai resnet18 model on mnist data
workflows/train/fastai/pets/job.py AML - GPU docker train fastai resnet34 model on pets data
workflows/train/lightgbm/iris/job.py AML - CPU pip train a lightgbm model on iris data
workflows/train/pytorch/mnist-mlproject/job.py AML - GPU mlproject train a pytorch CNN model on mnist data via mlflow mlproject
workflows/train/pytorch/mnist/job.py AML - GPU conda train a pytorch CNN model on mnist data
workflows/train/scikit-learn/diabetes-mlproject/job.py AML - CPU mlproject train sklearn ridge model on diabetes data via mlflow mlproject
workflows/train/scikit-learn/diabetes/job.py AML - CPU conda train sklearn ridge model on diabetes data
workflows/train/tensorflow/mnist-distributed-horovod/job.py AML - GPU conda train tensorflow CNN model on mnist data distributed via horovod
workflows/train/tensorflow/mnist-distributed/job.py AML - GPU conda train tensorflow CNN model on mnist data distributed via tensorflow
workflows/train/tensorflow/mnist/job.py AML - GPU conda train tensorflow NN model on mnist data
workflows/train/xgboost/iris/job.py AML - CPU pip train xgboost model on iris data

Deploy

path compute description
workflows/deploy/pytorch/mnist/job.py unknown deploy pytorch cnn model trained on mnist data to aks
workflows/deploy/scikit-learn/diabetes/job.py unknown deploy sklearn ridge model trained on diabetes data to AKS

Contributing

We welcome contributions and suggestions! Please see the contributing guidelines for details.

Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct. Please see the code of conduct for details.

Reference