/FastInst

[CVPR2023] FastInst: A Simple Query-Based Model for Real-Time Instance Segmentation

Primary LanguagePythonMIT LicenseMIT

FastInst: A Simple Query-Based Model for Real-Time Instance Segmentation

[arXiv] [BibTeX]

Features

  • A simple query-based model for fast instance segmentation.
  • State-of-the-art real-time performance under the same setting.
  • Support major segmentation datasets: COCO, Cityscapes, ADE20K.

Updates

  • [2023/6] FastInst has been integrated into ModelScope. Try out the Online Demo at FastInst 🚀.
  • [2023/4] We have released the code and checkpoints for FastInst. Welcome to your attention!

Installation

See installation instructions.

Getting Started

See Results.

See Preparing Datasets for FastInst.

See Getting Started.


Results

COCO Instance Segmentation

Name Backbone Epochs Input APval AP Params GFlops FPS (V100) download
FastInst-D1 R50 50 576 34.9 35.6 30M 49.6 53.8 model
FastInst-D3 R50 50 640 37.9 38.6 34M 75.5 35.5 model
FastInst-D3 R101 50 640 38.9 39.9 53M 112.9 28.0 model
FastInst-D1 R50-d-DCN 50 576 37.4 38.0 30M - 47.8 model
FastInst-D3 R50-d-DCN 50 640 40.1 40.5 35M - 32.5 model

Getting Started

This document provides a brief intro of the usage of FastInst.

Please see Getting Started with Detectron2 for full usage.

Evaluate our pretrained models

  • You can download our pretrained models and evaluate them with the following commands.
    python train_net.py --eval-only --num-gpus 4 --config-file config_path MODEL.WEIGHTS /path/to/checkpoint_file
    for example, to evaluate our released the fastest model, you can copy the config path from the table, download the pretrained checkpoint into /path/to/checkpoint_file, and run
    python train_net.py --eval-only --num-gpus 4 --config-file configs/coco/instance-segmentation/fastinst_R50_ppm-fpn_x1_576.yaml MODEL.WEIGHTS /path/to/checkpoint_file
    which can reproduce the model.

Train FastInst to reproduce results

  • Use the above command without eval-only will train the model.
    python train_net.py --num-gpus 4 --config-file config_path
  • For R101 backbone, you need to download and specify the path of the pretrained backbones with MODEL.WEIGHTS /path/to/pretrained_checkpoint. The download link can be found in the above table.
    python train_net.py --num-gpus 4 --config-file config_path MODEL.WEIGHTS /path/to/pretrained_checkpoint
  • For R50-d-DCN backbone, you need to download and convert the pretrained backbones, and specify the path.
    python tools/convert-timm-to-d2.py /path/to/resnet50d_ra2-464e36ba.pth /path/to/resnet50d_ra2-464e36ba.pkl
    python train_net.py --num-gpus 4 --config-file config_path MODEL.WEIGHTS /path/to/resnet50d_ra2-464e36ba.pkl

LICNESE

FastInst is released under the MIT Licence.

Citing FastInst

If you find FastInst is useful in your research or applications, please consider giving us a star 🌟 and citing FastInst by the following BibTeX entry.

@article{he2023fastinst,
  title={FastInst: A Simple Query-Based Model for Real-Time Instance Segmentation},
  author={He, Junjie and Li, Pengyu and Geng, Yifeng and Xie, Xuansong},
  journal={arXiv preprint arXiv:2303.08594},
  year={2023}
}

Acknowledgement

Sincerely thanks to these excellent opensource projects