SiamMask forked by Augmented Startups

NEW: now including code for both training and inference!

This is the official implementation with training code for SiamMask (CVPR2019). For technical details, please refer to:

Fast Online Object Tracking and Segmentation: A Unifying Approach
Qiang Wang*, Li Zhang*, Luca Bertinetto*, Weiming Hu, Philip H.S. Torr (* denotes equal contribution)
CVPR 2019
[Paper] [Video] [Project Page]

Bibtex

If you find this code useful, please consider citing:

@inproceedings{wang2019fast,
    title={Fast online object tracking and segmentation: A unifying approach},
    author={Wang, Qiang and Zhang, Li and Bertinetto, Luca and Hu, Weiming and Torr, Philip HS},
    booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
    year={2019}
}

Environment Setup
Demo
Testing Models
Training Models

Environment setup

This code has been tested on Ubuntu 16.04, Python 3.6, Pytorch 0.4.1, CUDA 9.2, RTX 2080 GPUs

Clone the repository

git clone https://github.com/foolwood/SiamMask.git && cd SiamMask
export SiamMask=$PWD

Setup python environment

conda create -n siammask python=3.6
source activate siammask
pip install -r requirements.txt
bash make.sh

Add the project to your PYTHONPATH

export PYTHONPATH=$PWD:$PYTHONPATH

Demo

Setup your environment
Download the SiamMask model

cd $SiamMask/experiments/siammask_sharp
wget http://www.robots.ox.ac.uk/~qwang/SiamMask_VOT.pth
wget http://www.robots.ox.ac.uk/~qwang/SiamMask_DAVIS.pth

Run demo.py

cd $SiamMask/experiments/siammask_sharp
export PYTHONPATH=$PWD:$PYTHONPATH
python ../../tools/demo.py --resume SiamMask_DAVIS.pth --config config_davis.json

Testing

Setup your environment
Download test data

cd $SiamMask/data
sudo apt-get install jq
bash get_test_data.sh

Download pretrained models

cd $SiamMask/experiments/siammask_sharp
wget http://www.robots.ox.ac.uk/~qwang/SiamMask_VOT.pth
wget http://www.robots.ox.ac.uk/~qwang/SiamMask_VOT_LD.pth
wget http://www.robots.ox.ac.uk/~qwang/SiamMask_DAVIS.pth

Evaluate performance on VOT

bash test_mask_refine.sh config_vot.json SiamMask_VOT.pth VOT2016 0
bash test_mask_refine.sh config_vot.json SiamMask_VOT.pth VOT2018 0
bash test_mask_refine.sh config_vot.json SiamMask_VOT.pth VOT2019 0
bash test_mask_refine.sh config_vot18.json SiamMask_VOT_LD.pth VOT2016 0
bash test_mask_refine.sh config_vot18.json SiamMask_VOT_LD.pth VOT2018 0
python ../../tools/eval.py --dataset VOT2016 --tracker_prefix C --result_dir ./test/VOT2016
python ../../tools/eval.py --dataset VOT2018 --tracker_prefix C --result_dir ./test/VOT2018
python ../../tools/eval.py --dataset VOT2019 --tracker_prefix C --result_dir ./test/VOT2019

Evaluate performance on DAVIS (less than 50s)

bash test_mask_refine.sh config_davis.json SiamMask_DAVIS.pth DAVIS2016 0
bash test_mask_refine.sh config_davis.json SiamMask_DAVIS.pth DAVIS2017 0

Evaluate performance on Youtube-VOS (need download data from website)

bash test_mask_refine.sh config_davis.json SiamMask_DAVIS.pth ytb_vos 0

Results

These are the reproduction results from this repository. All results can be downloaded from our project page.

_Tracker	_{VOT2016 EAO / A / R}	_{VOT2018 EAO / A / R}	_{DAVIS2016 J / F}	_{DAVIS2017 J / F}	_{Youtube-VOS J_s / J_u / F_s / F_u}	_Speed
_SiamMask-box	_{0.412/0.623/0.233}	_{0.363/0.584/0.300}	- / -	- / -	- / - / - / -	_{77 FPS}
_SiamMask	_{0.433/0.639/0.214}	_{0.380/0.609/0.276}	_0.713/0.674	_0.543/0.585	_{0.602/0.451/0.582/0.477}	_{56 FPS}
_SiamMask-LD	_{0.455/0.634/0.219}	_{0.423/0.615/0.248}	- / -	- / -	- / - / - / -	_{56 FPS}

Note:

Speed are tested on a NVIDIA RTX 2080.
-box reports an axis-aligned bounding box from the box branch.
-LD means training with large dataset (ytb-bb+ytb-vos+vid+coco+det).

License

Licensed under an MIT license.

augmentedstartups/SiamMask