/panoptic-segmentation

SOTA Panoptic Segmentation Models in PyTorch

Primary LanguagePythonMIT LicenseMIT

Panoptic Segmentation (WIP)

Easy to use SOTA Panoptic Segmentation models in PyTorch

Model Zoo

COCO-val
Model Backbone PQ PQth PQst Params
(M)		 GFLOPs Weights
Panoptic SegFormer PVTv2-B0 49.6 55.5 40.6 22 156 -
PVTv2-B2 52.6 58.7 43.3 42 219 -
PVTv2-B5 54.1 60.4 44.6 101 391 -
MaskFormer Swin-T 47.7 51.7 41.7 42 179 -
Swin-S 49.7 54.4 42.6 63 259 -
Swin-B 51.1 56.3 43.2 102 411 -
COCO-test (click to expand)
Model Backbone PQ PQth PQst Params
(M)		 GFLOPs Weights
Panoptic SegFormer PVTv2-B5 54.4 61.1 44.3 101 391 -

Supported Datasets

Dataset Type Categories Train
Images		 Val
Images		 Test
Images		 Image Size
(HxW)
COCO General 171 118,000 5,000 20,000 -
CityScapes Street 19 2,975 500 1,525+labels 1024x2048
Datasets Structure (click to expand)

Datasets should have the following structure:

data
|__ CityScapes
    |__ leftImg8bit
        |__ train
        |__ val
        |__ test
    |__ gtFine
        |__ train
        |__ val
        |__ test

|__ COCO
    |__ images
        |__ train2017
        |__ val2017
    |__ labels
        |__ train2017
        |__ val2017

Usage (Coming Soon)

Requirements (click to expand)
  • python >= 3.6
  • torch >= 1.8.1
  • torchvision >= 0.9.1

Other requirements can be installed with pip install -r requirements.txt.


Configuration (click to expand)

Create a configuration file in configs. Sample configuration for Custom dataset can be found here. Then edit the fields you think if it is needed. This configuration file is needed for all of training, evaluation and prediction scripts.


Training (click to expand)

To train with a single GPU:

$ python tools/train.py --cfg configs/CONFIG_FILE.yaml

To train with multiple gpus, set DDP field in config file to true and run as follows:

$ python -m torch.distributed.launch --nproc_per_node=2 --use_env tools/train.py --cfg configs/CONFIG_FILE.yaml

Evaluation (click to expand)

Make sure to set MODEL_PATH of the configuration file to your trained model directory.

$ python tools/val.py --cfg configs/CONFIG_FILE.yaml

To evaluate with multi-scale and flip, change ENABLE field in MSF to true and run the same command as above.


Inference

Make sure to set MODEL_PATH of the configuration file to model's weights.

$ python tools/infer.py --cfg configs/CONFIG_FILE.yaml
References (click to expand)
Citations (click to expand) 
@misc{li2021panoptic,
  title={Panoptic SegFormer}, 
  author={Zhiqi Li and Wenhai Wang and Enze Xie and Zhiding Yu and Anima Anandkumar and Jose M. Alvarez and Tong Lu and Ping Luo},
  year={2021},
  eprint={2109.03814},
  archivePrefix={arXiv},
  primaryClass={cs.CV}
}

@misc{cheng2020panopticdeeplab,
  title={Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation}, 
  author={Bowen Cheng and Maxwell D. Collins and Yukun Zhu and Ting Liu and Thomas S. Huang and Hartwig Adam and Liang-Chieh Chen},
  year={2020},
  eprint={1911.10194},
  archivePrefix={arXiv},
  primaryClass={cs.CV}
}