Object Tracking with SiamMask

This is the project repository for CS585 final project#3 object tracking

Group Member: Lifu Zhang, Hin Lui Shum, Anjana Srivastava, Trivikram Ranga

Problem Definition

Object Tracking: taking an initial bounding box, creating a unique ID for each of the initial targets, and then tracking each of them as they move around frames in a video, maintaining the ID assignment.

Method

Environment setup
Training SiamMask base model
Training SiamMask refine model

Environment setup

This code has been tested on MacOS

Clone the repository

git clone https://github.com/lifuzhang1108/SiamMaskUpdated.git && cd SiamMaskUpdated
export SiamMask=$PWD

Create Virtual Environment

pip install virtualenv
virtualenv --python=python3.6 .venv
source .venv/bin/activate
pip install -r requirements.txt
bash make.sh

Add the project to your PYTHONPATH

export PYTHONPATH=$PWD:$PYTHONPATH

Add Pretrained Resnet

Download from https://github.com/OlafenwaMoses/ImageAI/releases/download/1.0/resnet50_coco_best_v2.0.1.h5
add under SiamMaskUpdated/experiments/siammask_sharp

Training base model

Download the Youtube-VOS, COCO, ImageNet-DET, and ImageNet-VID.
Preprocess each datasets according the readme files.
Download the pre-trained resnet model (174 MB)
we trained for 17 epoches on 4 GPUs for 10 hours

Training Refine module

In the experiment file, train with the best SiamMask base model checkpoint_16.pth
we trained for 19 epoches on 4 GPUs for 12 hours
select the best model as the final model, in our case checkpoint_19.pth

Results

Testing on DAVIS dataset
Recycling video demo

Testing on Davis dataset

Recycling video demo

currentbest.mov

Run new-demo.py

cd $SiamMask/experiments/siammask_sharp
export PYTHONPATH=$PWD:$PYTHONPATH
python ../../tools/new-demo.py --resume SiamMask_DAVIS.pth --config config_davis.json

Discussion

SiamMask is a robust and powerful MOT model with the ability to produce accurate results with fast speed.
SiamMask generates accurate mask, and simultaneously produce minimal enclosing rectangle as the bounding box.
With reasonable amount of training time, we can achieve results that is comparable to that of pretrained model released by author.
Model fails sometimes when the object being tracked does not have very distinct boundary or intersect with other objects.
As a side effort, we made slight changes to allow multiple initial tracking objects (ROI) in tools/new-demo.py

lifuzhang1108/SiamMask-tracking