Deploying Deep Learning
Welcome to our training guide for inference and deep vision runtime library for NVIDIA DIGITS and Jetson Xavier/TX1/TX2.
This repo uses NVIDIA TensorRT for efficiently deploying neural networks onto the embedded platform, improving performance and power efficiency using graph optimizations, kernel fusion, and half-precision FP16 on the Jetson.
Vision primitives, such as imageNet
for image recognition, detectNet
for object localization, and segNet
for semantic segmentation, inherit from the shared tensorNet
object. Examples are provided for streaming from live camera feed and processing images from disk. See the Deep Vision API Reference Specification for accompanying documentation.
There are multiple tracks of the tutorial that you can choose to follow, including Training + Inference or Inference-Only.
> Jetson Nano Developer Kit and JetPack 4.2 is now supported in the repo.
> See our technical blog including benchmarks,Jetson Nano Brings AI Computing to Everyone
.
Hello AI World (Inference Only)
If you would like to only do the inference portion of the tutorial, which can be run on your Jetson in roughly two hours, these modules are available below:
- Setting up Jetson with JetPack
- Building the Repo from Source
- Classifying Images with ImageNet
- Locating Object Coordinates using DetectNet
Two Days to a Demo (Training + Inference)
The full tutorial includes training and inference, and can take roughly two days or more depending on system setup, downloading the datasets, and the training speed of your GPU.
- DIGITS Workflow
- DIGITS System Setup
- Setting up Jetson with JetPack
- Building the Repo from Source
- Classifying Images with ImageNet
- Using the Console Program on Jetson
- Coding Your Own Image Recognition Program
- Running the Live Camera Recognition Demo
- Re-Training the Network with DIGITS
- Downloading Image Recognition Dataset
- Customizing the Object Classes
- Importing Classification Dataset into DIGITS
- Creating Image Classification Model with DIGITS
- Testing Classification Model in DIGITS
- Downloading Model Snapshot to Jetson
- Loading Custom Models on Jetson
- Locating Object Coordinates using DetectNet
- Detection Data Formatting in DIGITS
- Downloading the Detection Dataset
- Importing the Detection Dataset into DIGITS
- Creating DetectNet Model with DIGITS
- Testing DetectNet Model Inference in DIGITS
- Downloading the Detection Model to Jetson
- DetectNet Patches for TensorRT
- Detecting Objects from the Command Line
- Multi-class Object Detection Models
- Running the Live Camera Detection Demo on Jetson
- Semantic Segmentation with SegNet
Extra Resources
In this area, links and resources for deep learning developers are listed:
- Appendix
- ros_deep_learning - TensorRT inference ROS nodes
- NVIDIA AI IoT - NVIDIA Jetson GitHub repositories
- Jetson eLinux Wiki - Jetson eLinux Wiki
Recommended System Requirements
Training GPU: Maxwell, Pascal, Volta, or Turing-based GPU (ideally with at least 6GB video memory)
optionally, AWS P2/P3 instance or Microsoft Azure N-series
Ubuntu 14.04 x86_64 or Ubuntu 16.04 x86_64.
Deployment: Jetson Xavier Developer Kit with JetPack 4.0 or newer (Ubuntu 18.04 aarch64).
Jetson TX2 Developer Kit with JetPack 3.0 or newer (Ubuntu 16.04 aarch64).
Jetson TX1 Developer Kit with JetPack 2.3 or newer (Ubuntu 16.04 aarch64).
note: this branch is verified against the following BSP versions for Jetson AGX Xavier and Jetson TX1/TX2:
> Jetson Nano - JetPack 4.2 / L4T R32.1 aarch64 (Ubuntu 18.04 LTS) inc. TensorRT 5.0
> Jetson AGX Xavier - JetPack 4.2 / L4T R32.1 aarch64 (Ubuntu 18.04 LTS) inc. TensorRT 5.0
> Jetson AGX Xavier - JetPack 4.1.1 DP / L4T R31.1 aarch64 (Ubuntu 18.04 LTS) inc. TensorRT 5.0 GA
> Jetson AGX Xavier - JetPack 4.1 DP EA / L4T R31.0.2 aarch64 (Ubuntu 18.04 LTS) inc. TensorRT 5.0 RC
> Jetson AGX Xavier - JetPack 4.0 DP EA / L4T R31.0.1 aarch64 (Ubuntu 18.04 LTS) inc. TensorRT 5.0 RC
> Jetson TX2 - JetPack 4.2 / L4T R32.1 aarch64 (Ubuntu 18.04 LTS) inc. TensorRT 5.0
> Jetson TX2 - JetPack 3.3 / L4T R28.2.1 aarch64 (Ubuntu 16.04 LTS) inc. TensorRT 4.0
> Jetson TX1 - JetPack 3.3 / L4T R28.2 aarch64 (Ubuntu 16.04 LTS) inc. TensorRT 4.0
> Jetson TX2 - JetPack 3.2 / L4T R28.2 aarch64 (Ubuntu 16.04 LTS) inc. TensorRT 3.0
> Jetson TX2 - JetPack 3.1 / L4T R28.1 aarch64 (Ubuntu 16.04 LTS) inc. TensorRT 3.0 RC
> Jetson TX1 - JetPack 3.1 / L4T R28.1 aarch64 (Ubuntu 16.04 LTS) inc. TensorRT 3.0 RC
> Jetson TX2 - JetPack 3.1 / L4T R28.1 aarch64 (Ubuntu 16.04 LTS) inc. TensorRT 2.1
> Jetson TX1 - JetPack 3.1 / L4T R28.1 aarch64 (Ubuntu 16.04 LTS) inc. TensorRT 2.1
> Jetson TX2 - JetPack 3.0 / L4T R27.1 aarch64 (Ubuntu 16.04 LTS) inc. TensorRT 1.0
> Jetson TX1 - JetPack 2.3 / L4T R24.2 aarch64 (Ubuntu 16.04 LTS) inc. TensorRT 1.0
> Jetson TX1 - JetPack 2.3.1 / L4T R24.2.1 aarch64 (Ubuntu 16.04 LTS)
Note that TensorRT samples from the repo are intended for deployment onboard Jetson, however when cuDNN and TensorRT have been installed on the host side, the TensorRT samples in the repo can be compiled for PC.
Legacy Links
Since the documentation has been re-organized, below are links mapping the previous content to the new locations.
(click on the arrow above to hide this section)DIGITS Workflow
See DIGITS Workflow
System Setup
See DIGITS Setup
Running JetPack on the Host
See JetPack Setup
Installing Ubuntu on the Host
See DIGITS Setup
Setting up host training PC with NGC container
See DIGITS Setup
Installing the NVIDIA driver
See DIGITS Setup
Installing Docker
See DIGITS Setup
NGC Sign-up
See DIGITS Setup
Setting up data and job directories
See DIGITS Setup
Starting DIGITS container
See DIGITS Setup
Natively setting up DIGITS on the Host
Installing NVIDIA Driver on the Host
Installing cuDNN on the Host
Installing NVcaffe on the Host
Installing DIGITS on the Host
Starting the DIGITS Server
Building from Source on Jetson
See Building the Repo from Source
Cloning the Repo
See Building the Repo from Source
Configuring with CMake
See Building the Repo from Source
Compiling the Project
See Building the Repo from Source
Digging Into the Code
See Building the Repo from Source
Classifying Images with ImageNet
See Classifying Images with ImageNet
Using the Console Program on Jetson
See Classifying Images with ImageNet
Running the Live Camera Recognition Demo
See Running the Live Camera Recognition Demo
Re-training the Network with DIGITS
See Re-Training the Recognition Network
Downloading Image Recognition Dataset
See Re-Training the Recognition Network
Customizing the Object Classes
See Re-Training the Recognition Network
Importing Classification Dataset into DIGITS
See Re-Training the Recognition Network
Creating Image Classification Model with DIGITS
See Re-Training the Recognition Network
Testing Classification Model in DIGITS
See Re-Training the Recognition Network
Downloading Model Snapshot to Jetson
See Downloading Model Snapshots to Jetson
Loading Custom Models on Jetson
See Loading Custom Models on Jetson
Locating Object Coordinates using DetectNet
See Locating Object Coordinates using DetectNet
Detection Data Formatting in DIGITS
See Locating Object Coordinates using DetectNet
Downloading the Detection Dataset
See Locating Object Coordinates using DetectNet
Importing the Detection Dataset into DIGITS
See Locating Object Coordinates using DetectNet
Creating DetectNet Model with DIGITS
See Locating Object Coordinates using DetectNet
Selecting DetectNet Batch Size
See Locating Object Coordinates using DetectNet
Specifying the DetectNet Prototxt
See Locating Object Coordinates using DetectNet
Training the Model with Pretrained Googlenet
See Locating Object Coordinates using DetectNet
Testing DetectNet Model Inference in DIGITS
See Locating Object Coordinates using DetectNet
Downloading the Model Snapshot to Jetson
See Downloading the Detection Model to Jetson
DetectNet Patches for TensorRT
See Downloading the Detection Model to Jetson
Processing Images from the Command Line on Jetson
See Detecting Objects from the Command Line
Launching With a Pretrained Model
See Detecting Objects from the Command Line
Pretrained DetectNet Models Available
See Detecting Objects from the Command Line
Running Other MS-COCO Models on Jetson
See Detecting Objects from the Command Line
Running Pedestrian Models on Jetson
See Detecting Objects from the Command Line
Multi-class Object Detection Models
See Detecting Objects from the Command Line
Running the Live Camera Detection Demo on Jetson
See Running the Live Camera Detection Demo
Image Segmentation with SegNet
See Semantic Segmentation with SegNet
Downloading Aerial Drone Dataset
See Semantic Segmentation with SegNet
Importing the Aerial Dataset into DIGITS
See Semantic Segmentation with SegNet
Generating Pretrained FCN-Alexnet
See Generating Pretrained FCN-Alexnet
Training FCN-Alexnet with DIGITS
See Training FCN-Alexnet with DIGITS
Testing Inference Model in DIGITS
See Training FCN-Alexnet with DIGITS
FCN-Alexnet Patches for TensorRT
See FCN-Alexnet Patches for TensorRT
Running Segmentation Models on Jetson
© 2016-2019 NVIDIA | Table of Contents