Fully Convolutional Networks for Panoptic Segmentation
Yanwei Li, Hengshuang Zhao, Xiaojuan Qi, Liwei Wang, Zeming Li, Jian Sun, Jiaya Jia
This project provides an implementation for the CVPR 2021 Oral paper "Fully Convolutional Networks for Panoptic Segmentation" based on Detectron2. Panoptic FCN is a conceptually simple, strong, and efficient framework for panoptic segmentation, which represents and predicts foreground things and background stuff in a unified fully convolutional pipeline.
This project is based on Detectron2, which can be constructed as follows.
- Install Detectron2 following the instructions.
- Setup the dataset following the structure.
- Copy this project to
/path/to/detectron2/projects/PanopticFCN
To train a model with 8 GPUs, run:
cd /path/to/detectron2
python3 projects/PanopticFCN/train.py --config-file <config.yaml> --num-gpus 8For example, to launch PanopticFCN training (1x schedule) with ResNet-50 backbone on 8 GPUs, one should execute:
cd /path/to/detectron2
python3 projects/PanopticFCN/train.py --config-file projects/PanopticFCN/configs/PanopticFCN-R50-1x.yaml --num-gpus 8To evaluate a pre-trained model with 8 GPUs, run:
cd /path/to/detectron2
python3 projects/PanopticFCN/train.py --config-file <config.yaml> --num-gpus 8 --eval-only MODEL.WEIGHTS /path/to/model_checkpointWe provide the results on COCO val set with pretrained models. FPS is measured on a single V100 GPU.
| Method | Backbone | Sched | PQ | SQ | RQ | AP | mIoU | FPS | download |
|---|---|---|---|---|---|---|---|---|---|
| PanopticFCN | R50 | 1x | 41.1 | 79.8 | 49.9 | 32.2 | 41.5 | 12.4 | model | metrics |
| PanopticFCN-400 | R50 | 3x | 41.0 | 81.0 | 49.6 | 30.7 | 43.6 | 22.5 | model | metrics |
| PanopticFCN-512 | R50 | 3x | 42.3 | 81.1 | 51.2 | 32.4 | 43.2 | 19.8 | model | metrics |
| PanopticFCN-600 | R50 | 3x | 42.7 | 81.2 | 51.5 | 33.6 | 43.9 | 17.5 | model | metrics |
| PanopticFCN | R50 | 3x | 43.6 | 81.4 | 52.5 | 34.4 | 43.6 | 12.8 | model | metrics |
| PanopticFCN* | R50 | 3x | 44.2 | 81.7 | 52.9 | 35.6 | 43.9 | 9.3 | model | metrics |
A faster version is also provided with higher threshold but similar PQ results, which shares the same model with the corresponding normal one. This version could be suitable for you if the final panoptic results are taken into consideration only.
| Method | Backbone | Sched | PQ | SQ | RQ | AP | mIoU | FPS | download |
|---|---|---|---|---|---|---|---|---|---|
| PanopticFCN | R50 | 1x | 41.1 | 79.8 | 49.9 | 30.2 | 41.4 | 13.6 | model | metrics |
| PanopticFCN-400 | R50 | 3x | 40.8 | 81.1 | 49.4 | 28.9 | 43.5 | 26.1 | model | metrics |
| PanopticFCN-512 | R50 | 3x | 42.3 | 81.1 | 51.2 | 30.7 | 43.2 | 22.0 | model | metrics |
| PanopticFCN-600 | R50 | 3x | 42.7 | 80.8 | 51.4 | 31.6 | 43.9 | 19.1 | model | metrics |
| PanopticFCN | R50 | 3x | 43.6 | 81.4 | 52.5 | 32.4 | 43.6 | 13.5 | model | metrics |
| PanopticFCN* | R50 | 3x | 44.2 | 81.7 | 52.9 | 33.4 | 43.9 | 9.7 | model | metrics |
Consider cite PanopticFCN in your publications if it helps your research.
@inproceedings{li2021panopticfcn,
title={Fully Convolutional Networks for Panoptic Segmentation},
author={Yanwei Li, Hengshuang Zhao, Xiaojuan Qi, Liwei Wang, Zeming Li, Jian Sun, and Jiaya Jia},
booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2021}
}
Consider cite this project in your publications if it helps your research.
@misc{PanopticFCN,
author = {Yanwei Li},
title = {PanopticFCN},
howpublished = {\url{https://github.com/yanwei-li/PanopticFCN}},
year ={2021}
}