hovnatan/tutorials

Tutorials for creating and using ONNX models

ONNX Tutorials

Open Neural Network Exchange (ONNX) is an open standard format for representing machine learning models. ONNX is supported by a community of partners who have implemented it in many frameworks and tools.

These images are available for convenience to get started with ONNX and tutorials on this page

• Docker image for ONNX and Caffe2/PyTorch
• Docker image for ONNX, ONNX Runtime, and various converters

Getting ONNX models

• Pre-trained models: Many pre-trained ONNX models are provided for common scenarios in the ONNX Model Zoo.
• Services: Customized ONNX models are generated for your data by cloud based services (see below)
• Convert models from various frameworks (see below)

Services

Below is a list of services that can output ONNX models customized for your data.

• Azure Custom Vision service
• Azure Machine Learning automated ML

Converting to ONNX format

Framework / Tool	Installation	Tutorial
Caffe	apple/coremltools and onnx/onnxmltools	Example
Caffe2	part of caffe2 package	Example
Chainer	chainer/onnx-chainer	Example
Cognitive Toolkit (CNTK)	built-in	Example
CoreML (Apple)	onnx/onnxmltools	Example
Keras	onnx/keras-onnx	Example
LibSVM	onnx/onnxmltools	Example
LightGBM	onnx/onnxmltools	Example
MATLAB	Deep Learning Toolbox	Example
ML.NET	built-in	Example
MXNet (Apache)	part of mxnet package docs github	Example
PyTorch	part of pytorch package	Example1, Example2, export for Windows ML, Extending support
SciKit-Learn	onnx/sklearn-onnx	Example
SINGA (Apache) - Github (experimental)	built-in	Example
TensorFlow	onnx/tensorflow-onnx	Examples

Scoring ONNX Models

Once you have an ONNX model, it can be scored with a variety of tools.

Framework / Tool	Installation	Tutorial
Caffe2	Caffe2	Example
Cognitive Toolkit (CNTK)	built-in	Example
CoreML (Apple)	onnx/onnx-coreml	Example
MATLAB	Deep Learning Toolbox Converter	Documentation and Examples
Menoh	Github Packages or from Nuget	Example
ML.NET	Microsoft.ML Nuget Package	Example
MXNet (Apache) - Github	MXNet	API Example
ONNX Runtime	Python (Pypi) - onnxruntime and onnxruntime-gpu C/C# (Nuget) - Microsoft.ML.OnnxRuntime and Microsoft.ML.OnnxRuntime.Gpu	APIs: Python, C#, C, C++ Examples - Python, C#, C
SINGA (Apache) - Github [experimental]	built-in	Example
Tensorflow	onnx-tensorflow	Example
TensorRT	onnx-tensorrt	Example
Windows ML	Pre-installed on Windows 10	API Tutorials - C++ Desktop App, C# UWP App Examples

End-to-End Tutorials

Conversion to deployment

• Converting SuperResolution model from PyTorch to Caffe2 with ONNX and deploying on mobile device
• Transferring SqueezeNet from PyTorch to Caffe2 with ONNX and to Android app
• Converting Style Transfer model from PyTorch to CoreML with ONNX and deploying to an iPhone
• Serving PyTorch Models on AWS Lambda with Caffe2 & ONNX
• MXNet to ONNX to ML.NET with SageMaker, ECS and ECR - external link
• Convert CoreML YOLO model to ONNX, score with ONNX Runtime, and deploy in Azure
• Inference PyTorch Bert Model for High Performance in ONNX Runtime
• Inference TensorFlow Bert Model for High Performance in ONNX Runtime
• Inference Bert Model for High Performance with ONNX Runtime on AzureML
• Various Samples: Inferencing ONNX models using ONNX Runtime (Python, C#, C, Java, etc)

ONNX Quantization

• HuggingFace Bert Quantization with ONNX Runtime

Serving

• Serving ONNX models with AI-Serving
• Serving ONNX models with Cortex
• Serving ONNX models with MXNet Model Server
• Serving ONNX models with ONNX Runtime Server
• ONNX model hosting with AWS SageMaker and MXNet
• Serving ONNX models with ONNX Runtime on Azure ML
  • FER Facial Expression Recognition
  • MNIST Handwritten Digits
  • Resnet50 Image Classification

ONNX as an intermediary format

• Convert a PyTorch model to Tensorflow using ONNX

ONNX Custom Operators

• How to export Pytorch model with custom op to ONNX and run it in ONNX Runtime

Visualizing ONNX Models

• Netdrawer: Visualizing ONNX models
• Netron: Viewer for ONNX models
• Zetane: 3D visualizer for ONNX models and internal tensors

Other ONNX tools

• Verifying correctness and comparing performance
• Example of operating on ONNX protobuf
• Float16 <-> Float32 converter
• Version conversion

Contributing

We welcome improvements to the convertor tools and contributions of new ONNX bindings. Check out contributor guide to get started.

Use ONNX for something cool? Send the tutorial to this repo by submitting a PR.