By Tingting Liang*, Xiaojie Chu*, Yudong Liu*, Yongtao Wang, Zhi Tang, Wei Chu, Jingdong Chen, Haibin Ling.
This repo is the official implementation of CBNetV2. It is based on mmdetection and Swin Transformer for Object Detection.
Contact us with tingtingliang@pku.edu.cn, chuxiaojie@stu.pku.edu.cn, wyt@pku.edu.cn.
CBNetV2 achieves strong single-model performance on COCO object detection (60.1 box AP
and 52.3 mask AP
on test-dev) without extra training data.
More results and models can be found in model zoo
Backbone | Lr Schd | box mAP (minival) | #params | FLOPs | config | log | model |
---|---|---|---|---|---|---|---|
DB-ResNet50 | 1x | 40.8 | 69M | 284G | config | github | github |
Backbone | Lr Schd | box mAP (minival) | mask mAP (minival) | #params | FLOPs | config | log | model |
---|---|---|---|---|---|---|---|---|
DB-Swin-T | 3x | 50.2 | 44.5 | 76M | 357G | config | github | github |
Backbone | Lr Schd | box mAP (minival/test-dev) | mask mAP (minival/test-dev) | #params | FLOPs | config | model |
---|---|---|---|---|---|---|---|
DB-Swin-S | 3x | 56.3/56.9 | 48.6/49.1 | 156M | 1016G | config | github |
We use ImageNet-22k pretrained checkpoints of Swin-B and Swin-L. Compared to regular HTC, our HTC uses 4conv1fc in bbox head.
Backbone | Lr Schd | box mAP (minival/test-dev) | mask mAP (minival/test-dev) | #params | FLOPs | config | model |
---|---|---|---|---|---|---|---|
DB-Swin-B | 20e | 58.4/58.7 | 50.7/51.1 | 235M | 1348G | config | github |
DB-Swin-L | 1x | 59.1/59.4 | 51.0/51.6 | 453M | 2162G | config (test only) | github |
DB-Swin-L (TTA) | 1x | 59.6/60.1 | 51.8/52.3 | 453M | - | config (test only) | github |
TTA denotes test time augmentation.
Notes:
- Pre-trained models of Swin Transformer can be downloaded from Swin Transformer for ImageNet Classification.
Please refer to get_started.md for installation and dataset preparation.
In case you get errors use the steps given below for installation.
1.) git clone https://github.com/kartikwar/CBNetV2.git
2.) virtualenv -p python3 cbnet-env
3.) source cbnet-env/bin/activate
4.) pip install torch==1.6.0 torchvision==0.7.0
Note: See the page https://download.openmmlab.com/mmcv/dist/cu101/torch1.6/index.html to select the combatible version in case of errors
6.) pip install mmcv_full-1.3.18-cp36-cp36m-manylinux1_x86_64.whl
7.) pip install mmdet
8.) cd CBNetV2
9.) pip install -r requirements/runtime.txt
# to test on single image
python inference.py
#for simple inference i.e if you are not interested in optimizing saliency mask
checkout to branch plain_inference
# single-gpu testing (w/o segm result)
python tools/test.py <CONFIG_FILE> <DET_CHECKPOINT_FILE> --eval bbox
# multi-gpu testing (w/ segm result)
tools/dist_test.sh <CONFIG_FILE> <DET_CHECKPOINT_FILE> <GPU_NUM> --eval bbox segm
To train a detector with pre-trained models, run:
# multi-gpu training
tools/dist_train.sh <CONFIG_FILE> <GPU_NUM>
For example, to train a Faster R-CNN model with a Duel-ResNet50
backbone and 8 gpus, run:
# path of pre-training model (resnet50) is already in config
tools/dist_train.sh configs/cbnet/faster_rcnn_cbv2d1_r50_fpn_1x_coco.py 8
Another example, to train a Mask R-CNN model with a Duel-Swin-T
backbone and 8 gpus, run:
tools/dist_train.sh configs/cbnet/mask_rcnn_cbv2_swin_tiny_patch4_window7_mstrain_480-800_adamw_3x_coco.py 8 --cfg-options model.pretrained=<PRETRAIN_MODEL>
Following Swin Transformer for Object Detection, we use apex for mixed precision training by default. To install apex, run:
git clone https://github.com/NVIDIA/apex
cd apex
pip install -v --disable-pip-version-check --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./
We list some documents and tutorials from MMDetection, which may be helpful to you.
If you use our code/model, please consider to cite our paper CBNetV2: A Novel Composite Backbone Network Architecture for Object Detection.
@article{liang2021cbnetv2,
title={CBNetV2: A Composite Backbone Network Architecture for Object Detection},
author={Tingting Liang and Xiaojie Chu and Yudong Liu and Yongtao Wang and Zhi Tang and Wei Chu and Jingdong Chen and Haibing Ling},
journal={arXiv preprint arXiv:2107.00420},
year={2021}
}
The project is only free for academic research purposes, but needs authorization for commerce. For commerce permission, please contact wyt@pku.edu.cn.
Original CBNet: See CBNet: A Novel Composite Backbone Network Architecture for Object Detection.