This repository contains FFmpeg patches that enable CNN model-based video analytics capabilities in the FFmpeg framework. These patches contain filter based implementation include such things as object detection, classification, and recognition. Following is an example that runs detection & emotion classification, using specific models on a video file:
ffmpeg -i test.mp4 -vf detect=model=face-detection-adas-0001.xml:model_proc=face-detection-adas-0001.json,classify=model=emotions-recognition-retail-0003.xml:model_proc=emotions-recognition-retail-0003.json -an -f null -The solution leverages:
- Open-source FFmpeg framework for pipeline management
- FFmpeg components for input and output, such as media files and real-time streaming from a camera or network
- Video decode and encode plugins, including either CPU-optimized plugins or GPU-accelerated plugins, based on VAAPI
In addition, the solution installs the following Deep Learning-specific elements, also available in this repository:
- Inference plugins leveraging OpenVINO™ Toolkit for high-performance inference using CNN models
- Visualization of computer vision results (such as bounding boxes and labels of detected objects) on top of video stream
The FFmpeg Video Analytics Patches are licensed under the GNU Lesser General Public License (LGPL) version 2.1 or later.
- OpenVINO™ Toolkit has information about the hardware requirements for inference elements
- On platforms with Intel Gen graphics, see the ffmpeg vaapi wiki for Hardware Support
- OpenVINO™ Toolkit 2019 R3 (Inference Engine 2.1.0) or above
- Linux* system with kernel 4.15 or above
- FFmpeg framework version n4.2
See the command-line examples
Report bugs and requests on the issues page
Pull requests aren't monitored, so if you have bug fix or an idea to improve this project, post a description on the issues page
* Other names and brands may be claimed as the property of others.