DeepLab-Pytorch

Pytorch implementation of DeepLab series, including DeepLabV1-LargeFOV, DeepLabV2-ResNet101, DeepLabV3, and DeepLabV3+. The experiments are all conducted on PASCAL VOC 2012 dataset.

Setup

Install Environment with Conda

conda create --name py36 python==3.6
conda activate py36
pip install -r requirements.txt

Clone this repository

git clone https://github.com/rulixiang/deeplab-pytorch.git
cd deeplab-pytorch

Prepare PASCAL VOC 2012 dataset

Download PASCAL VOC 2012 dataset

## download voc 2012 dataset
wget http://host.robots.ox.ac.uk/pascal/VOC/voc2012/VOCtrainval_11-May-2012.tar
## untar
tar –xvf VOCtrainval_11-May-2012.tar

Download the augmented annotations

The augmented annotations are from SBD dataset. Here is a download link of the augmented annotations at DropBox. After downloading SegmentationClassAug.zip, you should unzip it and move it to VOCdevkit/VOC2012. The directory sctructure should thus be

./VOCdevkit/
└── VOC2012
    ├── Annotations
    ├── ImageSets
    ├── JPEGImages
    ├── SegmentationClass
    ├── SegmentationClassAug
    └── SegmentationObject

The root_dir in the .yaml files under directory config should also be updated as your directory.

Todo List

[✓] DeepLabV1-LargeFOV
- [✓] train
- [✓] test
- [✓] crf
[✓] DeepLabV2-ResNet101
- [✓] train
- [✓] test
- [✓] crf
[✗] DeepLabV3
[✗] DeepLabV3+

DeepLabV1

Chen, Liang-Chieh and Papandreou, George and Kokkinos, Iasonas and Murphy, Kevin and Yuille, Alan L, Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs, ICLR 2015.

Train and Test

Before training, you need to download an initial model from this link and move it to weights directory. To train and test a DeepLabV1-LargeFOV network, you need at least 1 gpu device.

## train
python v1/train_deeplabv1.py --gpu 0,1 --config config/deeplabv1_voc12.yaml
## test on trained model
python v1/test_deeplabv1.py --gpu 0 --config config/deeplabv1_voc12.yaml --crf True

Or just run the shell script:

bash run_deeplabv1.sh

Results

The evaulation results are reported in the table below. Random up-down flip and Random rotation cannot improve the performance.

Train set	Val set	CRF	Method	Pixel Accuracy	Mean IoU
trainaug	val	✗	DeepLabv1-step	-	62.25
			Ours-step	89.87	62.20
			DeepLabv1-poly	-	65.88
			Ours-poly	91.31	65.48
		✓	DeepLabv1-step	-	67.64
			Ours-step	92.18	67.96
			DeepLabv1-poly	-	-
			Ours-poly	92.74	69.38

step means the original learning rate decay policy in deeplabv1, poly means the polynomial learning rate decay policy in deeplabv2.

DeepLabV2

Chen, Liang-Chieh and Papandreou, George and Kokkinos, Iasonas and Murphy, Kevin and Yuille, Alan L, Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs, IEEE TPAMI 2017.

Train and Test

Before training, you need to download the initial weights pre-trained on COCO dataset from this link and move it to weights directory. To train and test a DeepLabV2-ResNet101 network, you need at least 3 gpu device with 11GB memory.

## train
python v2/train_deeplabv2.py --gpu 0,1 --config config/deeplabv2_voc12.yaml
## test on trained model
python v2/test_deeplabv2.py --gpu 0 --config config/deeplabv2_voc12.yaml --crf True

Or just run the shell script:

bash run_deeplabv2.sh

Results

Train set	Val set	CRF	Method	Pixel Accuracy	Mean IoU
trainaug	val	✗	DeepLabv2	-	76.35
		✗	Ours	94.61	76.71
		✓	DeepLabv2	-	77.69
		✓	Ours	94.99	77.96

DeepLabV3

DeepLabV3+

Citation

Please consider citing their works if you used DeepLab.

@inproceedings{deeplabv1,
  title={Semantic image segmentation with deep convolutional nets and fully connected crfs},
  author={Chen, Liang-Chieh and Papandreou, George and Kokkinos, Iasonas and Murphy, Kevin and Yuille, Alan L},
  booktitle={International Conference on Learning Representations},
  year={2015}
}

@article{deeplabv2,
  title={Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs},
  author={Chen, Liang-Chieh and Papandreou, George and Kokkinos, Iasonas and Murphy, Kevin and Yuille, Alan L},
  journal={IEEE transactions on pattern analysis and machine intelligence},
  volume={40},
  number={4},
  pages={834--848},
  year={2017},
  publisher={IEEE}
}