Welcome to our instructional guide for inference and realtime DNN vision library for NVIDIA Jetson Nano/TX1/TX2/Xavier.
This repo uses NVIDIA TensorRT for efficiently deploying neural networks onto the embedded Jetson platform, improving performance and power efficiency using graph optimizations, kernel fusion, and FP16/INT8 precision.
Vision primitives, such as imageNet
for image recognition, detectNet
for object localization, and segNet
for semantic segmentation, inherit from the shared tensorNet
object. Examples are provided for streaming from live camera feed and processing images. See the API Reference section for detailed reference documentation of the C++ and Python libraries.
There are multiple tracks of the tutorial that you can choose to follow, including Hello AI World for running inference and transfer learning onboard your Jetson, or the full Two Days to a Demo tutorial for training on a PC or server with DIGITS.
It's recommended to walk through the Hello AI World module first to familiarize yourself with machine learning and inference with TensorRT, before proceeding to training in the cloud with DIGITS.
> Jetson Xavier NX and JetPack 4.4 Developer Preview is now supported in the repo.
> See our latest technical blog on theNVIDIA Jetson Xavier NX Developer Kit
.
Hello AI World can be run completely onboard your Jetson, including inferencing with TensorRT and transfer learning with PyTorch. The inference portion of Hello AI World - which includes coding your own image classification application for C++ or Python, object detection, and live camera demos - can be run on your Jetson in roughly two hours or less, while transfer learning is best left to leave running overnight.
The full tutorial includes training in the cloud or PC, and inference on the Jetson with TensorRT, and can take roughly two days or more depending on system setup, downloading the datasets, and the training speed of your GPU.
Below are links to reference documentation for the C++ and Python libraries from the repo:
C++ | Python | |
---|---|---|
Image Recognition | imageNet |
imageNet |
Object Detection | detectNet |
detectNet |
Segmentation | segNet |
segNet |
These libraries are able to be used in external projects by linking to libjetson-inference
and libjetson-utils
.
Introductory code walkthroughs of using the library are covered during these steps of the Hello AI World tutorial:
Additional C++ and Python samples for running the networks on static images and live camera streams can be found here:
Images | Camera | |
---|---|---|
C++ (examples ) |
||
Image Recognition | imagenet-console |
imagenet-camera |
Object Detection | detectnet-console |
detectnet-camera |
Segmentation | segnet-console |
segnet-camera |
Python (python/examples ) |
||
Image Recognition | imagenet-console.py |
imagenet-camera.py |
Object Detection | detectnet-console.py |
detectnet-camera.py |
Segmentation | segnet-console.py |
segnet-camera.py |
note: for working with numpy arrays, see
cuda-from-numpy.py
andcuda-to-numpy.py
These examples will automatically be compiled while Building the Project from Source, and are able to run the pre-trained models listed below in addition to custom models provided by the user. Launch each example with --help
for usage info.
The project comes with a number of pre-trained models that are available through the Model Downloader tool:
Network | CLI argument | NetworkType enum |
---|---|---|
AlexNet | alexnet |
ALEXNET |
GoogleNet | googlenet |
GOOGLENET |
GoogleNet-12 | googlenet-12 |
GOOGLENET_12 |
ResNet-18 | resnet-18 |
RESNET_18 |
ResNet-50 | resnet-50 |
RESNET_50 |
ResNet-101 | resnet-101 |
RESNET_101 |
ResNet-152 | resnet-152 |
RESNET_152 |
VGG-16 | vgg-16 |
VGG-16 |
VGG-19 | vgg-19 |
VGG-19 |
Inception-v4 | inception-v4 |
INCEPTION_V4 |
Network | CLI argument | NetworkType enum | Object classes |
---|---|---|---|
SSD-Mobilenet-v1 | ssd-mobilenet-v1 |
SSD_MOBILENET_V1 |
91 (COCO classes) |
SSD-Mobilenet-v2 | ssd-mobilenet-v2 |
SSD_MOBILENET_V2 |
91 (COCO classes) |
SSD-Inception-v2 | ssd-inception-v2 |
SSD_INCEPTION_V2 |
91 (COCO classes) |
DetectNet-COCO-Dog | coco-dog |
COCO_DOG |
dogs |
DetectNet-COCO-Bottle | coco-bottle |
COCO_BOTTLE |
bottles |
DetectNet-COCO-Chair | coco-chair |
COCO_CHAIR |
chairs |
DetectNet-COCO-Airplane | coco-airplane |
COCO_AIRPLANE |
airplanes |
ped-100 | pednet |
PEDNET |
pedestrians |
multiped-500 | multiped |
PEDNET_MULTI |
pedestrians, luggage |
facenet-120 | facenet |
FACENET |
faces |
Dataset | Resolution | CLI Argument | Accuracy | Jetson Nano | Jetson Xavier |
---|---|---|---|---|---|
Cityscapes | 512x256 | fcn-resnet18-cityscapes-512x256 |
83.3% | 48 FPS | 480 FPS |
Cityscapes | 1024x512 | fcn-resnet18-cityscapes-1024x512 |
87.3% | 12 FPS | 175 FPS |
Cityscapes | 2048x1024 | fcn-resnet18-cityscapes-2048x1024 |
89.6% | 3 FPS | 47 FPS |
DeepScene | 576x320 | fcn-resnet18-deepscene-576x320 |
96.4% | 26 FPS | 360 FPS |
DeepScene | 864x480 | fcn-resnet18-deepscene-864x480 |
96.9% | 14 FPS | 190 FPS |
Multi-Human | 512x320 | fcn-resnet18-mhp-512x320 |
86.5% | 34 FPS | 370 FPS |
Multi-Human | 640x360 | fcn-resnet18-mhp-512x320 |
87.1% | 23 FPS | 325 FPS |
Pascal VOC | 320x320 | fcn-resnet18-voc-320x320 |
85.9% | 45 FPS | 508 FPS |
Pascal VOC | 512x320 | fcn-resnet18-voc-512x320 |
88.5% | 34 FPS | 375 FPS |
SUN RGB-D | 512x400 | fcn-resnet18-sun-512x400 |
64.3% | 28 FPS | 340 FPS |
SUN RGB-D | 640x512 | fcn-resnet18-sun-640x512 |
65.1% | 17 FPS | 224 FPS |
nvpmodel 0
(MAX-N)Network | CLI Argument | NetworkType enum | Classes |
---|---|---|---|
Cityscapes (2048x2048) | fcn-alexnet-cityscapes-hd |
FCN_ALEXNET_CITYSCAPES_HD |
21 |
Cityscapes (1024x1024) | fcn-alexnet-cityscapes-sd |
FCN_ALEXNET_CITYSCAPES_SD |
21 |
Pascal VOC (500x356) | fcn-alexnet-pascal-voc |
FCN_ALEXNET_PASCAL_VOC |
21 |
Synthia (CVPR16) | fcn-alexnet-synthia-cvpr |
FCN_ALEXNET_SYNTHIA_CVPR |
14 |
Synthia (Summer-HD) | fcn-alexnet-synthia-summer-hd |
FCN_ALEXNET_SYNTHIA_SUMMER_HD |
14 |
Synthia (Summer-SD) | fcn-alexnet-synthia-summer-sd |
FCN_ALEXNET_SYNTHIA_SUMMER_SD |
14 |
Aerial-FPV (1280x720) | fcn-alexnet-aerial-fpv-720p |
FCN_ALEXNET_AERIAL_FPV_720p |
2 |
Training GPU: Maxwell, Pascal, Volta, Turing, or Ampere-based GPU (ideally with at least 6GB video memory)
optionally, AWS P2/P3 instance or Microsoft Azure N-series
Ubuntu 16.04/18.04 x86_64
Deployment: Jetson Nano Developer Kit with JetPack 4.2 or newer (Ubuntu 18.04 aarch64).
Jetson Xavier NX Developer Kit with JetPack 4.4 or newer (Ubuntu 18.04 aarch64).
Jetson AGX Xavier Developer Kit with JetPack 4.0 or newer (Ubuntu 18.04 aarch64).
Jetson TX2 Developer Kit with JetPack 3.0 or newer (Ubuntu 16.04 aarch64).
Jetson TX1 Developer Kit with JetPack 2.3 or newer (Ubuntu 16.04 aarch64).
Note that TensorRT samples from the repo are intended for deployment onboard Jetson, however when cuDNN and TensorRT have been installed on the host side, the TensorRT samples in the repo can be compiled for PC.
In this area, links and resources for deep learning are listed:
See DIGITS Workflow
See DIGITS Setup
See JetPack Setup
See DIGITS Setup
See DIGITS Setup
See DIGITS Setup
See DIGITS Setup
See DIGITS Setup
See DIGITS Setup
See DIGITS Setup
See Building the Repo from Source
See Building the Repo from Source
See Building the Repo from Source
See Building the Repo from Source
See Building the Repo from Source
See Classifying Images with ImageNet
See Classifying Images with ImageNet
See Running the Live Camera Recognition Demo
See Re-Training the Recognition Network
See Re-Training the Recognition Network
See Re-Training the Recognition Network
See Re-Training the Recognition Network
See Re-Training the Recognition Network
See Re-Training the Recognition Network
See Downloading Model Snapshots to Jetson
See Loading Custom Models on Jetson
See Locating Object Coordinates using DetectNet
See Locating Object Coordinates using DetectNet
See Locating Object Coordinates using DetectNet
See Locating Object Coordinates using DetectNet
See Locating Object Coordinates using DetectNet
See Locating Object Coordinates using DetectNet
See Locating Object Coordinates using DetectNet
See Locating Object Coordinates using DetectNet
See Locating Object Coordinates using DetectNet
See Downloading the Detection Model to Jetson
See Downloading the Detection Model to Jetson
See Detecting Objects from the Command Line
See Detecting Objects from the Command Line
See Detecting Objects from the Command Line
See Detecting Objects from the Command Line
See Detecting Objects from the Command Line
See Detecting Objects from the Command Line
See Running the Live Camera Detection Demo
See Semantic Segmentation with SegNet
See Semantic Segmentation with SegNet
See Semantic Segmentation with SegNet
See Generating Pretrained FCN-Alexnet
See Training FCN-Alexnet with DIGITS
See Training FCN-Alexnet with DIGITS
See FCN-Alexnet Patches for TensorRT
© 2016-2019 NVIDIA | Table of Contents
此处可能存在不合适展示的内容,页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容,可点击提交进行申诉,我们将尽快为您处理。