dsmertin/dlstreamer_gst

This repository contains a collection of GStreamer* elements to enable CNN model based video analytics capabilities (such as object detection, classification, recognition) in GStreamer* framework.

OpenVINO^™ Toolkit - DL Streamer repository

Overview

This repository contains GStreamer* elements that enable CNN model-based video analytics capabilities using OpenVINO^™ Toolkit Inference Engine across all Intel Hardware. These elements can be used to perform use cases such as object detection, classification, recognition and tracking. The above sample video demonstrates the following GStreamer pipeline that executes face detection and emotion classification, using specific CNN models, on a video file:

gst-launch-1.0 filesrc location=cut.mp4 ! decodebin ! videoconvert ! gvadetect model=face-detection-adas-0001.xml ! gvaclassify model=emotions-recognition-retail-0003.xml model-proc=emotions-recognition-retail-0003.json ! gvawatermark ! xvimagesink sync=false

The solution leverages:

• Open-source GStreamer framework for pipeline management
• GStreamer plugins for input and output, such as media files and real-time streaming from a camera or network
• Video decode and encode plugins, including either CPU-optimized plugins or GPU-accelerated plugins, based on VAAPI

In addition, the solution installs the following Deep Learning-specific elements, also available in this repository:

• Inference plugins leveraging OpenVINO^™ Toolkit for high-performance inference using CNN models
• Visualization of computer vision results (such as bounding boxes and labels of detected objects) on top of video stream

In addition to the samples provided in this repository, please also review the following reference applications that use GStreamer Video Analytics plugin elements:

• Smart City - Traffic and Stadium Management
• Intelligent Ad Insertion

License

The GStreamer Video Analytics Plugin, part of OpenVINO^™ Toolkit - DL Streamer, is licensed under the MIT license.

GStreamer is an open source framework licensed under LGPL. See license terms. You are solely responsible for determining if your use of Gstreamer requires any additional licenses. Intel is not responsible for obtaining any such licenses, nor liable for any licensing fees due, in connection with your use of Gstreamer

Prerequisites

Hardware

• OpenVINO^™ Toolkit has information about the hardware requirements for inference elements
• On platforms with Intel Gen graphics, see the gstreamer-vaapi for hardware accelerated video decode and encode requirements

Software

• OpenVINO^™ Toolkit 2020.4 (Inference Engine 2.1.0) or above
• Linux* system with kernel 4.15 or above
• GStreamer framework 1.14 or above

Getting Started

• Start here: Install Guide
• API reference
• Gstreamer VA plugin on YouTube: Full pipeline simulation using GStreamer; Full pipeline simulation using GStreamer (Samples)

Samples

See the command-line examples and C++ example

Reporting Bugs and Feature Requests

Report bugs and requests on the issues page

Usage and integration into application

Metadata

Details about metadata generated by inference plugins and attached to video frames

Model preparation

Details about how to prepare Tensorflow*, Caffe*, and other models for the inference plugins

Plugins parameters

Elements list and properties list for each element

How to contribute

Pull requests aren't monitored, so if you have bug fix or an idea to improve this project, post a description on the issues page.

---

* Other names and brands may be claimed as the property of others.