Exploring Lightweight Hierarchical Vision Transformers for Efficient Visual Tracking
accepted by ICCV2023
This is an official pytorch implementation of the ICCV2023 paper Exploring Lightweight Hierarchical Vision Transformers for Efficient Visual Tracking, a new framework for Efficient visual object tracking.
The architecture of HiT is very simple only contains three components: a lightweight hierarchical vision transformer, a bridge Module and a prediction head.
HiT achieves promising speed with competitive performance, it is faster and performs better than previous efficient trackers, It significantly closes the gap between efficient trackers and mainstream trackers. It can be deployed in low-computing power scenarios.
HiT-Base | HiT-Small | HiT-Tiny | FEAR | TransT | |
---|---|---|---|---|---|
LaSOT (AUC) | 64.6 | 60.5 | 54.8 | 53.5 | 64.9 |
GOT-10K (AO) | 64.0 | 62.6 | 52.6 | 61.9 | 72.3 |
TrackingNet (AUC) | 80.0 | 77.7 | 74.6 | - | 81.4 |
PyTorch Speed on 3090 GPU (FPS) | 175 | 192 | 204 | 105 | 63 |
PyTorch Speed on i9-9900K CPU (FPS) | 33 | 72 | 76 | 60 | 5 |
PyTorch Speed on Nvidia Jetson AGX (FPS) | 61 | 68 | 77 | 38 | 13 |
PyTorch Speed on Nvidia Jetson NX (FPS) | 32 | 34 | 39 | 22 | 11 |
ONNX Speed on 3090 GPU (FPS) | 274 | 400 | 455 | - | - |
ONNX Speed on i9-9900K CPU (FPS) | 42 | 98 | 125 | - | - |
ONNX Speed on Nvidia Jetson AGX (FPS) | 75 | 119 | 145 | - | - |
Option1: Use the Anaconda
conda create -n HIT python=3.8
conda activate HIT
pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu113
pip install -r requirements.txt
Put the tracking datasets in ./data. It should look like:
${STARK_ROOT}
-- data
-- lasot
|-- airplane
|-- basketball
|-- bear
...
-- got10k
|-- test
|-- train
|-- val
-- coco
|-- annotations
|-- images
-- trackingnet
|-- TRAIN_0
|-- TRAIN_1
...
|-- TRAIN_11
|-- TEST
Run the following command to set paths for this project
python tracking/create_default_local_file.py --workspace_dir . --data_dir ./data --save_dir .
After running this command, you can also modify paths by editing these two files
lib/train/admin/local.py # paths about training
lib/test/evaluation/local.py # paths about testing
python -m torch.distributed.launch --nproc_per_node 8 lib/train/run_training.py --script HiT --config HiT_Base --save_dir .
(Optionally) Debugging training with a single GPU
python lib/train/run_training.py --script HiT --config HiT_Base --save_dir .
- LaSOT
python tracking/test.py HiT HiT_Base --dataset lasot --threads 2 --num_gpus 2 --debug 0
python tracking/analysis_results.py # need to modify tracker configs and names
- GOT10K-test
python tracking/test.py HiT HiT_Base --dataset got10k_test --threads 2 --num_gpus 2 --debug 0
python lib/test/utils/transform_got10k.py --tracker_name HiT --cfg_name HiT_Base
- TrackingNet
python tracking/test.py HiT HiT_Base --dataset trackingnet --threads 2 --num_gpus 2 --debug 0
python lib/test/utils/transform_trackingnet.py --tracker_name HiT --cfg_name HiT_Base
- NFS
python tracking/test.py HiT HiT_Base --dataset nfs --threads 2 --num_gpus 2 --debug 0
python tracking/analysis_results.py # need to modify tracker configs and names
- UAV123
python tracking/test.py HiT HiT_Base --dataset uav --threads 2 --num_gpus 2 --debug 0
python tracking/analysis_results.py # need to modify tracker configs and names
- LaSOText
python tracking/test.py HiT HiT_Base --dataset lasot_extension_subset --threads 2 --num_gpus 2 --debug 0
python tracking/analysis_results.py # need to modify tracker configs and names
python tracking/transfer_onnx
python tracking/profile_model_hit --script HiT --config HiT_Base
python tracking/profile_model_hit_cpu --script HiT --config HiT_Base
python tracking/profile_model_hit_onnx --script HiT --config HiT_Base
python tracking/profile_model_hit_onnx_cpu --script HiT --config HiT_Base
The trained models, and the raw tracking results are provided in here
- This codebase is implemented on STARK and PyTracking.We would like to thank their authors for providing great libraries.
-
Ben Kang (email:kangben@mail.dlut.edu.cn)
-
Xin Chen (email:chenxin3131@mail.dlut.edu.cn)
Feel free to contact if you have additional questions.