/DCFS

[NeurIPS 2022] Decoupling Classifier for Boosting Few-shot Object Detection and Instance Segmentation

Primary LanguagePythonMIT LicenseMIT

Introduction

This repository is a official PyTorch implementation of Decoupling Classifier for Boosting Few-shot Object Detection and Instance Segmentation (NeurIPS 2022). This repo is created by Bin-Bin Gao.

PWC PWC PWC

Training and Evaluation

  • To reproduce the FSOD/gFSOD results on COCO

    bash run_coco_fsod.sh r101 8 dcfs

  • To reproduce the FSIS/gFSIS results on COCO

    bash run_coco_fsis.sh r101 8 dcfs

  • Model Weighs for Base Pre-Training

    datasets Task Model Weghts
    COCO-Base Detection model
    COCO-Base Instance Segmentation model
    VOC-Base1 Detection model
    VOC-Base2 Detection model
    VOC-Base3 Detection model

Results on COCO Benchmark

  • Few-shot Object Detection

    Method mAPnovel
    Shot 1 2 3 5 10 30
    DeFRCN* 7.7 11.4 13.3 15.5 18.5 22.5
    DCFS 8.1 12.1 14.4 16.6 19.5 22.7

  • Generalized Few-shot Object Detection

    Method mAPnovel
    Shot 1 2 3 5 10 30
    DeFRCN 4.8 8.5 10.7 13.6 16.8 21.2
    DCFS 6.2 10.4 12.9 15.7 18.3 21.9
    Method mAPBase
    Shot 1 2 3 5 10 30
    DeFRCN 30.4 31.4 32.1 32.6 34.0 34.8
    DCFS 34.4 34.7 34.9 35.0 35.7 35.8

  • Few-shot Instance Segmentation

    Method mAPnovel
    Shot 1 2 3 5 10 30
    Mask-DeFRCN 6.7 9.5 11.0 12.7 15.4 18.3
    DCFS 7.2 10.3 13.5 15.7 15.9 18.3

  • Generalized Few-shot Instance Segmentation

    Method mAPnovel
    Shot 1 2 3 5 10 30
    Mask-DeFRCN* 4.5 7.5 9.5 11.6 14.3 17.6
    DCFS 5.7 9.4 11.5 13.5 15.7 18.3
    Method mAPbase
    Shot 1 2 3 5 10 30
    Mask-DeFRCN* 24.6 25.6 26.2 26.8 28.2 29.1
    DCFS 28.0 28.5 28.9 29.1 29.9 30.3
  • Please refer to DeFRCN for data peparation and installation details.
  • * indicates that the results are reproduced by us with the DeFRCN source code.
  • The results of mAPbase and mAPall for gFSOD and gFSIS can be seen at the Supplementary Material and ProjectPage.

Visualization Results

The baseline DeFRCN may tend to incorrectly recognize positive object as background (middle two rows) due to the biased classification. This problem is greatly alleviated using our proposed method (DCFS).

Few-Shot Object Detection Leaderboard

  • MSCOCO FSOD Leaderboard: [html]
  • PASCAL-VOC FSOD Leaderboard: [html]

Citing

If you find this code useful in your research, please consider citing us:

@inproceedings{gao2022dcfs,
  title={Decoupling classifier for boosting few-shot object detection and instance segmentation},
  author={Gao, Bin-Bin and Chen, Xiaochen and Huang, Zhongyi and Nie, Congchong and Liu, Jun and Lai, Jinxiang and Jiang, Guannan and Wang, Xi and Wang, Chengjie},
  booktitle={Advances in Neural Information Processing Systems (NeurIPS 2022)},
  pages={18640--18652},
  year={2022}
}

Acknowledgement

This repo is developed based on DeFRCN and Detectron2.

Discussion