junjiehe96/FastInst

[CVPR2023] FastInst: A Simple Query-Based Model for Real-Time Instance Segmentation

FastInst: A Simple Query-Based Model for Real-Time Instance Segmentation

[arXiv] [BibTeX]

Features

• A simple query-based model for fast instance segmentation.
• State-of-the-art real-time performance under the same setting.
• Support major segmentation datasets: COCO, Cityscapes, ADE20K.

---

Updates

• [2023/6] FastInst has been integrated into ModelScope. Try out the Online Demo at FastInst 🚀.
• [2023/4] We have released the code and checkpoints for FastInst. Welcome to your attention!

Installation

See installation instructions.

Getting Started

See Results.

See Preparing Datasets for FastInst.

See Getting Started.

---

Results

COCO Instance Segmentation

Name	Backbone	Epochs	Input	APval	AP	Params	GFlops	FPS (V100)	download
FastInst-D1	R50	50	576	34.9	35.6	30M	49.6	53.8	model
FastInst-D3	R50	50	640	37.9	38.6	34M	75.5	35.5	model
FastInst-D3	R101	50	640	38.9	39.9	53M	112.9	28.0	model
FastInst-D1	R50-d-DCN	50	576	37.4	38.0	30M	-	47.8	model
FastInst-D3	R50-d-DCN	50	640	40.1	40.5	35M	-	32.5	model

Getting Started

This document provides a brief intro of the usage of FastInst.

Please see Getting Started with Detectron2 for full usage.

Evaluate our pretrained models

• You can download our pretrained models and evaluate them with the following commands.

  python train_net.py --eval-only --num-gpus 4 --config-file config_path MODEL.WEIGHTS /path/to/checkpoint_file

  for example, to evaluate our released the fastest model, you can copy the config path from the table, download the pretrained checkpoint into /path/to/checkpoint_file, and run

  python train_net.py --eval-only --num-gpus 4 --config-file configs/coco/instance-segmentation/fastinst_R50_ppm-fpn_x1_576.yaml MODEL.WEIGHTS /path/to/checkpoint_file

  which can reproduce the model.

Train FastInst to reproduce results

• Use the above command without eval-only will train the model.

  python train_net.py --num-gpus 4 --config-file config_path

• For R101 backbone, you need to download and specify the path of the pretrained backbones with MODEL.WEIGHTS /path/to/pretrained_checkpoint. The download link can be found in the above table.

  python train_net.py --num-gpus 4 --config-file config_path MODEL.WEIGHTS /path/to/pretrained_checkpoint

• For R50-d-DCN backbone, you need to download and convert the pretrained backbones, and specify the path.

  python tools/convert-timm-to-d2.py /path/to/resnet50d_ra2-464e36ba.pth /path/to/resnet50d_ra2-464e36ba.pkl
  python train_net.py --num-gpus 4 --config-file config_path MODEL.WEIGHTS /path/to/resnet50d_ra2-464e36ba.pkl

LICNESE

FastInst is released under the MIT Licence.

Citing FastInst

If you find FastInst is useful in your research or applications, please consider giving us a star 🌟 and citing FastInst by the following BibTeX entry.

@article{he2023fastinst,
  title={FastInst: A Simple Query-Based Model for Real-Time Instance Segmentation},
  author={He, Junjie and Li, Pengyu and Geng, Yifeng and Xie, Xuansong},
  journal={arXiv preprint arXiv:2303.08594},
  year={2023}
}

Acknowledgement

Sincerely thanks to these excellent opensource projects

• DETR
• Mask2Former