/Yolov5_StrongSORT_OSNet

Mult-object tracking and segmentation using YOLOv5 with StrongSORT and OSNet

Primary LanguagePythonGNU General Public License v3.0GPL-3.0

Real-time multi-object tracking and segmentation using Yolov5 with StrongSORT and OSNet


CI CPU testing
Open In Colab DOI

Introduction

This repository contains a highly configurable two-stage-tracker that adjusts to different deployment scenarios. It can jointly perform multiple object tracking and instance segmentation (MOTS). The detections generated by YOLOv5, a family of object detection architectures and models pretrained on the COCO dataset, are passed to the tracker of your choice. Supported ones at the moment are: StrongSORT OSNet, OCSORT and ByteTrack. They can track any object that your Yolov5 model was trained to detect.

Installation

git clone --recurse-submodules https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet.git  # clone recursively
cd Yolov5_StrongSORT_OSNet
pip install -r requirements.txt  # install dependencies
Tutorials
Experiments

In inverse chronological order:

Custom object detection architecture

The trackers provided in this repo can be used with other object detectors than Yolov5. Make sure that the output of your detector has the following format:

(x1,y1, x2, y2, obj, cls0, cls1, ..., clsn)

pass this directly to the tracker here:

https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/blob/a4bc0c38c33023fab9e5481861d9520eb81e28bc/track.py#L189

Tracking

$ python track.py --yolo-weights yolov5n.pt      # bboxes only
                                 yolov5n-seg.pt  # bboxes + segmentation masks
Tracking methods
$ python track.py --tracking-method strongsort
                                    ocsort
                                    bytetrack
Tracking sources

Tracking can be run on most video formats

$ python track.py --source 0  # webcam
                           img.jpg  # image
                           vid.mp4  # video
                           path/  # directory
                           path/*.jpg  # glob
                           'https://youtu.be/Zgi9g1ksQHc'  # YouTube
                           'rtsp://example.com/media.mp4'  # RTSP, RTMP, HTTP stream
Select Yolov5 model

There is a clear trade-off between model inference speed and overall performance. In order to make it possible to fulfill your inference speed/accuracy needs you can select a Yolov5 family model for automatic download. These model can be further optimized for you needs by the export.py script

$ python track.py --source 0 --yolo-weights yolov5n.pt --img 640
                                            yolov5s.tflite
                                            yolov5m.pt
                                            yolov5l.onnx 
                                            yolov5x.pt --img 1280
                                            ...
Select ReID model

Some tracking methods combine appearance description and motion in the process of tracking. For those which use appearance, you can choose a ReID model based on your needs from this ReID model zoo. These model can be further optimized for you needs by the reid_export.py script

$ python track.py --source 0 --reid-weights osnet_x0_25_market1501.pt
                                            mobilenetv2_x1_4_msmt17.engine
                                            resnet50_msmt17.onnx
                                            osnet_x1_0_msmt17.pt
                                            ...
Filter tracked classes

By default the tracker tracks all MS COCO classes.

If you want to track a subset of the classes that you model predicts, add their corresponding index after the classes flag,

python track.py --source 0 --yolo-weights yolov5s.pt --classes 16 17  # COCO yolov5 model. Track cats and dogs, only

Here is a list of all the possible objects that a Yolov5 model trained on MS COCO can detect. Notice that the indexing for the classes in this repo starts at zero

Updates with predicted-ahead bbox in StrongSORT

If your use-case contains many occlussions and the motion trajectiories are not too complex, you will most certainly benefit from updating the Kalman Filter by its own predicted state. Select the number of predictions that suits your needs here:

https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/blob/b1da64717ef50e1f60df2f1d51e1ff91d3b31ed4/trackers/strong_sort/configs/strong_sort.yaml#L7

Save the trajectories to you video by:

python track.py --source ... --save-trajectories --save-vid

MOT compliant results

Can be saved to your experiment folder runs/track/<yolo_model>_<deep_sort_model>/ by

python track.py --source ... --save-txt

Contact

For Yolov5 StrongSORT OSNet bugs and feature requests please visit GitHub Issues. For business inquiries or professional support requests please send an email to: yolov5.deepsort.pytorch@gmail.com