/DeepLearningExamples

Deep Learning Examples

Primary LanguagePython

NVIDIA Deep Learning Examples for Tensor Cores

Introduction

This repository provides State-of-the-Art Deep Learning examples that are easy to train and deploy, achieving the best reproducible accuracy and performance with NVIDIA CUDA-X software stack running on NVIDIA Volta, Turing and Ampere GPUs.

NVIDIA GPU Cloud (NGC) Container Registry

These examples, along with our NVIDIA deep learning software stack, are provided in a monthly updated Docker container on the NGC container registry (https://ngc.nvidia.com). These containers include:

  • The latest NVIDIA examples from this repository
  • The latest NVIDIA contributions shared upstream to the respective framework
  • The latest NVIDIA Deep Learning software libraries, such as cuDNN, NCCL, cuBLAS, etc. which have all been through a rigorous monthly quality assurance process to ensure that they provide the best possible performance
  • Monthly release notes for each of the NVIDIA optimized containers

Computer Vision

Models Framework A100 AMP Multi-GPU Multi-Node TRT ONNX Triton TF-TRT NB
ResNet-50 PyTorch Yes Yes Yes - Yes - Yes - -
ResNeXt-101 PyTorch Yes Yes Yes - Yes - Yes - -
SE-ResNeXt-101 PyTorch Yes Yes Yes - Yes - Yes - -
Mask R-CNN PyTorch Yes Yes Yes - - - - - Yes
SSD PyTorch Yes Yes Yes - - - - - Yes
ResNet-50 TensorFlow Yes Yes Yes - - - - - -
ResNeXt101 TensorFlow Yes Yes Yes - - - - - -
SE-ResNeXt-101 TensorFlow Yes Yes Yes - - - - - -
Mask R-CNN TensorFlow Yes Yes Yes - - - - - -
SSD TensorFlow Yes Yes Yes - - - - - Yes
U-Net Ind TensorFlow Yes Yes Yes - - - - Yes Yes
U-Net Med TensorFlow Yes Yes Yes - - - - - -
U-Net 3D TensorFlow Yes Yes Yes - - - - - -
V-Net Med TensorFlow Yes Yes Yes - - - - - -
U-Net Med TensorFlow2 Yes Yes Yes - - - - - -
Mask R-CNN TensorFlow2 Yes Yes Yes - - - - - -
ResNet-50 MXNet - Yes Yes - - - - - -

Natural Language Processing

Models Framework A100 AMP Multi-GPU Multi-Node TRT ONNX Triton TF-TRT NB
BERT PyTorch Yes Yes Yes Yes - - Yes - -
TransformerXL PyTorch Yes Yes Yes Yes - - - - -
GNMT PyTorch Yes Yes Yes - - - - - -
Transformer PyTorch Yes Yes Yes - - - - - -
ELECTRA TensorFlow2 Yes Yes Yes Yes - - - - -
BERT TensorFlow Yes Yes Yes Yes Yes - Yes - Yes
BioBert TensorFlow Yes Yes Yes - - - - - Yes
TransformerXL TensorFlow Yes Yes Yes - - - - - -
GNMT TensorFlow Yes Yes Yes - - - - - -
Faster Transformer Tensorflow - - - - Yes - - - -

Recommender Systems

Models Framework A100 AMP Multi-GPU Multi-Node TRT ONNX Triton TF-TRT NB
DLRM PyTorch Yes Yes Yes - - Yes Yes - Yes
NCF PyTorch Yes Yes Yes - - - - - -
Wide&Deep TensorFlow Yes Yes Yes - - - - - -
NCF TensorFlow Yes Yes Yes - - - - - -
VAE-CF TensorFlow Yes Yes Yes - - - - - -

Speech to Text

Models Framework A100 AMP Multi-GPU Multi-Node TRT ONNX Triton TF-TRT NB
Jasper PyTorch Yes Yes Yes - Yes Yes Yes - Yes
Hidden Markov Model Kaldi - - Yes - - - Yes - -

Text to Speech

Models Framework A100 AMP Multi-GPU Multi-Node TRT ONNX Triton TF-TRT NB
FastPitch PyTorch Yes Yes Yes - - - - - -
FastSpeech PyTorch - Yes Yes - Yes - - - -
Tacotron 2 and WaveGlow PyTorch Yes Yes Yes - Yes Yes Yes - -

NVIDIA support

In each of the network READMEs, we indicate the level of support that will be provided. The range is from ongoing updates and improvements to a point-in-time release for thought leadership.

Feedback / Contributions

We're posting these examples on GitHub to better support the community, facilitate feedback, as well as collect and implement contributions using GitHub Issues and pull requests. We welcome all contributions!

Known issues

In each of the network READMEs, we indicate any known issues and encourage the community to provide feedback.