realtime_object_detection

Realtime Object Detection based on Tensorflow's Object Detection API and DeepLab Project

Version 1: use branch r1.0 for the original repo that was focused on high performance inference of ssd_mobilenet
(x10 Performance Increase on Nvidia Jetson TX2)

Version 2: use branch Master or to be additionally able to run and test Mask-Detection Models, KCF-Tracking and DeepLab Models (merge of the repo realtime_segmenation)

ROS Support: To use this Repo as ROS-Package including detection and segmentation ROS-Nodes use branch ros. Alternativley use the repo objectdetection_ros

About the Project

The Idea was to create a scaleable realtime-capable object detection pipeline that runs on various systems.
Plug and play, ready to use without deep previous knowledge.

The project includes following work:

optional download of tensorflow pretrained models
do Inference with OpenCV, either through video input or on selected test_images.
supported Models are all research/object_detection as well as research/deeplab models
enjoy this project's own ssd_mobilenet speed hack, which splits the model in a mutlithreaded cpu and gpu session.
Results in up to x10 performance increase depending on the running system
⇒ which makes it (one of) the fastest inference piplines out there
run statistic tests on sets of images and get statistical information like mean and median fps, std dev and much more
create timeline files measuring the exact time consumption of each operation in your model
inspect, summarize, quantize, transform and benchmark models with the provided scripts/
Use this Repo as ROS Package. the detection subscirbes a ROS Image topic and publishes the detection as ROS Node.

Inference:

create a copy of config.sample.yml named config.yml and only change configurations inside this file
For example: If you are not interested in visualization: set VISUALIZE to False,
or if you want to switch off the speed hack set SPLIT_MODEL to False,
to be able to use KCF_Tracking inside scripts/ run bash build_kcf.sh to build it and set USE_TRACKER to True to use it
(currently only works for pure object detection models without SPLIT_MODEL)

new class (Model,Config,Visualizer) structure. Simply create your own test file with:

from rod.model import ObjectDetectionModel, DeepLabModel
from rod.config import Config

model_type = 'od'                                              #or 'dl'
input_type = 'video'                                           #or 'image'
config = Config(model_type)
model = ObjectDetectionModel(config).prepare_model(input_type) #or DeepLabModel
model.run()

Alternativley run python + objectdetection_video.py or objectdetection_image.py or deeplab_video.py or deeplab_image.py or allmodels_image.py

Scripts:

To make use of the tools provided inside scripts/ follow this guide:

first change all paths and variables inside config_tools.sh to your needs / according to your system
When using the first time run: source config_tools.sh and in the same terminal run only once source build_tools.sh to build the tools. this will take a while.
For all following uses first run: source config_tools.sh(due to the exported variables) and after that you are able to run the wanted scripts always from the same terminal with source script.sh.
All scripts log the terminal output to test_results/

Setup:

Use the following setup for best and verified performance

Ubuntu 16.04
Python 2.7
Tensorflow 1.4 (this repo provides pre-build tf wheel files for jetson tx2)
OpenCV 3.3.1

Note: tensorflow v1.7.0 seems to have massive performance issues (try to use other versions)

Current max Performance on `ssd_mobilenet`:

Dell XPS 15 with i7 @ 2.80GHZ x8 and GeForce GTX 1050 4GB: 100 FPS
Nvidia Jetson Tx2 with Tegra 8GB: 30 FPS

Related Work:

objectdetection_ros: This Repository as ROS Package ready to use
test_models: A repo for models i am currently working on for benchmark tests
deeptraining_hands: A repo for setting up the ego- and oxford hands-datasets.
It also contains several scripts to convert various annotation formats to be able to train Networks on different deep learning frameworks
currently supports .xml, .mat, .csv, .record, .txt annotations
yolo_for_tf_od_api: A repo to be able to include Yolo V2 in tf's object detection api
realtime_segmenation: This repo was merged into v2.0
Mobile_Mask_RCNN: a Keras Model for training Mask R-CNN for mobile deployment
tf_training: Train Mobile Mask R-CNN Models on AWS Cloud
tf_models: My tensorflow/models fork which includes yolov2 and mask_rcnn_mobilenet_v1_coco
eetfm_automation: Export and Evaluation of TensorFlow Models Automation based on the Object Detection API

gustavz/realtime_object_detection