This repo is a collection of the challenging panoptic segmentation, including papers, codes, and benchmark results, etc.
Summarize in one sentence : Panoptic Segmentation proposes to solve the semantic segmentation(*Stuff*) and instance segmentation(*Thing*) in a unified and general manner.from UPSNet.
Generally, the datasets which contains both semantic and instance annotations can be used to solve the challenging panoptic task.
PQ
are the standard metrics described in Panoptic Segmentation.
PC
are the standard metrics described in DeeperLab.
- COCO 2019 Panoptic Segmentation Task (ICCV 2019 Workshop, Closed)
- Mapillary 2019 Panoptic Segmentation Task (ICCV 2019 Workshop, Closed)
- Cityscapes Panoptic Semantic Labeling Task (Open)
- COCO 2018 Panoptic Segmentation Task (ECCV 2018 Workshop, Closed)
- Mapillary Vistas 2018 Panoptic Segmentation Task (ECCV 2018 Workshop, Closed)
Method | Backbone | PQ | PQ-Thing | PQ-Stuff | SQ | RQ | mIoU | AP-Mask | PC | e2e |
---|---|---|---|---|---|---|---|---|---|---|
UPSNet | ResNet-50 | 42.5 | 48.6 | 33.4 | - | - | 54.3 | 34.3 | - | ✅ |
OANet | ResNet-101 | 41.3 | 50.4 | 27.7 | - | - | - | - | - | ✅ |
OCFusion | ResNet-50 | 41.0 | 49.0 | 29.0 | 77.1 | 50.6 | - | - | - | ✅ |
Panoptic FPN | ResNet-101 | 40.9 | 48.3 | 29.7 | - | - | - | - | - | ✅ |
AUNet | ResNet-50 | 39.6 | 49.1 | 25.2 | - | - | 45.1 | 34.7 | - | ✅ |
DeeperLab | Xception-71 | 34.3 | 37.5 | 29.6 | 77.1 | 43.1 | - | - | 56.8 | ✅ |
Method | Backbone | PQ | PQ-Thing | PQ-Stuff | SQ | RQ | mIoU | AP-Mask | PC | e2e |
---|---|---|---|---|---|---|---|---|---|---|
Panoptic(Merge) | - | 61.2 | 66.4 | 54.0 | 80.9 | 74.4 | - | - | - | ❎ |
Seamless | ResNet-50 | 59.8 | 53.4 | 64.5 | - | - | 75.4 | 31.9 | - | ✅ |
UPSNet | ResNet-50 | 59.3 | 54.6 | 62.7 | 79.7 | 73.0 | 75.2 | 33.3 | - | ✅ |
TASCNet | ResNet-101 | 59.2 | 56 | 61.5 | - | - | 77.8 | 37.6 | - | ✅ |
AUNet | ResNet-101 | 59.0 | 54.8 | 62.1 | - | - | 75.6 | 34.4 | - | ✅ |
Panoptic FPN | ResNet-101 | 58.1 | 52.0 | 62.5 | - | - | 75.7 | 33.0 | - | ✅ |
DeeperLab | Xception-71 | 56.5 | - | - | - | - | - | - | 75.6 | ✅ |
Method | Backbone | PQ | PQ-Thing | PQ-Stuff | SQ | RQ | mIoU | AP-Mask | PC | e2e |
---|---|---|---|---|---|---|---|---|---|---|
Panoptic(Merge) | - | 38.3 | 41.8 | 35.7 | 73.6 | 47.7 | - | - | - | ❎ |
Seamless | ResNet-50 | 37.2 | 33.2 | 42.5 | - | - | 50.2 | 16.3 | - | ✅ |
TASCNet | ResNet-101 | 32.6 | 31.3 | 34.4 | - | - | 35.0 | 18.5 | - | ✅ |
DeeperLab | Xception-71 | 31.6 | 25.0 | 40.3 | 75.5 | 40.1 | - | - | 55.3 | ✅ |
-
AdaptIS: Konstantin Sofiiuk, Olga Barinova, Anton Konushin.
"AdaptIS: Adaptive Instance Selection Network." ICCV (2019). [paper] -
Bowen Cheng, Maxwell D. Collins, Yukun Zhu, Ting Liu, Thomas S. Huang, Hartwig Adam, Liang-Chieh Chen.
"Panoptic-DeepLab." ICCVW (2019). [paper]
-
Panoptic Segmentation: Alexander Kirillov, Kaiming He, Ross Girshick, Carsten Rother, Piotr Dollár.
"Panoptic Segmentation." CVPR (2019). [paper] -
Panoptic FPN: Alexander Kirillov, Ross Girshick, Kaiming He, Piotr Dollár.
"Panoptic Feature Pyramid Networks." CVPR (2019 oral). [paper] [unofficial code] -
AUNet: Yanwei Li, Xinze Chen, Zheng Zhu, Lingxi Xie, Guan Huang, Dalong Du, Xingang Wang.
"Attention-guided Unified Network for Panoptic Segmentation." CVPR (2019). [paper] -
UPSNet: Yuwen Xiong, Renjie Liao, Hengshuang Zhao, Rui Hu, Min Bai, Ersin Yumer, Raquel Urtasun.
"UPSNet: A Unified Panoptic Segmentation Network." CVPR (2019 oral). [paper] [code] -
DeeperLab: Tien-Ju Yang, Maxwell D. Collins, Yukun Zhu, Jyh-Jing Hwang, Ting Liu, Xiao Zhang, Vivienne Sze, George Papandreou, Liang-Chieh Chen.
"DeeperLab: Single-Shot Image Parser." CVPR (2019 oral). [paper] [project] [code] -
OANet: Huanyu Liu, Chao Peng, Changqian Yu, Jingbo Wang, Xu Liu, Gang Yu, Wei Jiang.
"An End-to-End Network for Panoptic Segmentation." CVPR (2019). [paper] -
Eirikur Agustsson, Jasper R. R. Uijlings, Vittorio Ferrari .
"Interactive Full Image Segmentation by Considering All Regions Jointly." CVPR (2019). [paper]
- Qizhu Li, Anurag Arnab, Philip H.S. Torr.
"Weakly- and Semi-Supervised Panoptic Segmentation." ECCV (2018). [paper] [code]
-
Mark Weber, Jonathon Luiten, Bastian Leibe.
"Single-Shot Panoptic Segmentation." arXiv (2019). [paper] -
Qiang Chen, Anda Cheng, Xiangyu He, Peisong Wang, Jian Cheng.
"SpatialFlow: Bridging All Tasks for Panoptic Segmentation." arXiv (2019). [paper] -
Sagi Eppel, Alan Aspuru-Guzik.
"Generator evaluator-selector net: a modular approach for panoptic segmentation." arXiv (2019). [paper] -
Cheng-Yang Fu, Tamara L. Berg, Alexander C. Berg.
"IMP: Instance Mask Projection for High Accuracy Semantic Segmentation of Things." arXiv (2019). [paper] -
Jasper R. R. Uijlings, Mykhaylo Andriluka, Vittorio Ferrari.
"Panoptic Image Annotation with a Collaborative Assistant." arXiv (2019). [paper] -
OCFusion: Justin Lazarow, Kwonjoon Lee, Zhuowen Tu.
"Learning Instance Occlusion for Panoptic Segmentation." arXiv (2019). [paper] -
PEN: Yuan Hu, Yingtian Zou, Jiashi Feng.
"Panoptic Edge Detection." arXiv (2019). [paper] -
TASCNet: Jie Li, Allan Raventos, Arjun Bhargava, Takaaki Tagawa, Adrien Gaidon.
"Learning to Fuse Things and Stuff." arXiv (2018). [paper] -
Seamless: Lorenzo Porzi, Samuel Rota Bulo, Aleksander Colovic, Peter Kontschieder.
"Seamless Scene Segmentation." arXiv (2019). [paper][code] -
Daan de Geus, Panagiotis Meletis, Gijs Dubbelman.
"Panoptic Segmentation with a Joint Semantic and Instance Segmentation Network." arXiv (2018). [paper] -
Daan de Geus, Panagiotis Meletis, Gijs Dubbelman.
"Single Network Panoptic Segmentation for Street Scene Understanding." arXiv (2019). [paper] -
David Owen, Ping-Lin Chang.
"Detecting Reflections by Combining Semantic and Instance Segmentation." arXiv (2019). [paper] -
Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji.
"PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things." arXiv (2019). [paper]
- CVPR 2019 Tutorial on Visual Recognition and Beyond. [slides] [homepage]
- COCO 2017 Workshop. [slides]
- Megvii(Face++) Detection Team. [zhihu]