NVIDIA DeepStream Technical Deep Dive: DeepStream Inference Options with Triton & TensorRT

NVIDIA’s DeepStream SDK delivers a complete streaming analytics toolkit for AI-based multi-sensor processing, video, audio and image understanding. It provides sensor data analytics solutions from edge to cloud. It also offers extensive AI model inference pipelines for object detection, classification, segmentation and so on. In this video, we will walk through the inference approaches with Triton and TensorRT options and deep dive into each inference plugins, batching policies and custom preprocessing/postprocessing functions and libs. DeepStream supports multiple deep learning runtime models such as Tensorflow, ONNX, Pytorch, TensorRT and custom C++/Python backends. Besides that, with DeepStream, users can also build ONNX/TAO/Caffe models into TensorRT engine file to accelerate model inference on GPU and Jetson DLA. From the video, you will learn how to: 1. Work with DeepStream’s inference options for Tensorflow, Pytorch, and ONNX models. 2. Work with TensorRT and DeepStream for optimized models. 3. Use Triton server to support single or multiple DeepStream pipelines. 4. Use DeepStream’s pre/post-processing plug-ins. More information: To download DeepStream SDK and get started, visit nvda.ws/3RpR4n7 To setup DeepStream Triton environment, read the quick start guide here nvda.ws/3ReC5wb To learn more about DeepStream Triton CAPI and gRPC settings, ensemble models and custom pre/post-processing for LSTM, input/output tensor data parsing,check out gst-nvinferserver documentation nvda.ws/3Y1cicX To learn more about DeepStream TensorRT parameter setting, read gst-nvinfer documentation nvda.ws/3Y69POa To learn more about Triton Inference Server architecture and features, visit nvda.ws/3XFvKfs To configure Triton Model input/output tensors, batching policies and GPU optimization options for TensorRT, TensorFlow, ONNX, Pytorch and custom backends, visit github repo nvda.ws/3wz5tDB Join the NVIDIA Developer Program: nvda.ws/3OhiXfl Read and subscribe to the NVIDIA Tech Blog: nvda.ws/3XHae9F Inference, AI, Deep Learning, Video Analytics, streaming analytics, vision AI, #Inference, #AI, #DeepLearning, #videoanalytics, #streaminganalytics, #visionAI, #NVIDIADeepStream, #NVIDIAMetropolis