DeepLab V3

Lei Mao, Shengjie Lin

University of Chicago

Toyota Technological Institute at Chicago

Introduction

DeepLab is a series of image semantic segmentation models, whose latest version, i.e. v3+, proves to be the state-of-art. Its major contribution is the use of atrous spatial pyramid pooling (ASPP) operation at the end of the encoder. While the model works extremely well, its open source code is hard to read (at least from my personal perspective). Here we re-implemented DeepLab V3, the earlier version of v3+ (which only additionally employs the decoder architecture), in a much simpler and more understandable way.

Dependencies

Python 3.5
TensorFlow 1.8
Tqdm 4.26.0
Numpy 1.14
OpenCV 3.4.3
Pillow 5.3.0

Files

.
├── archieved
├── download.py
├── feature_extractor.py
├── LICENSE.md
├── model.py
├── modules.py
├── nets
├── README.md
├── test_demo.py
├── test_any_image.py
├── train.py
└── utils.py

The nets directory contains network definition files that are directly copied from tensorflow/models/research/slim/nets

Usages

Install Dependencies

To install dependencies, please run the following command to install everything required automatically:

$ chmod +x install_dependencies.sh
$ pip install -r requirements.txt
$ ./install_dependencies.sh

If found permission problems, please run the following command instead:

$ chmod +x install_dependencies.sh
$ pip install -r requirements.txt
$ sudo ./install_dependencies.sh

Download Dataset

Download and extract VOC2012 dataset, SBD dataset, and pretrained models to designated directories.

$ python download.py --help
usage: download.py [-h] [--downloads_dir DOWNLOADS_DIR] [--data_dir DATA_DIR]
                   [--pretrained_models_dir PRETRAINED_MODELS_DIR]
                   [--pretrained_models PRETRAINED_MODELS [PRETRAINED_MODELS ...]]

Download DeepLab semantic segmentation datasets and pretrained backbone
models.

optional arguments:
  -h, --help            show this help message and exit
  --downloads_dir DOWNLOADS_DIR
                        Downloads directory
  --data_dir DATA_DIR   Data directory
  --pretrained_models_dir PRETRAINED_MODELS_DIR
                        Pretrained models directory
  --pretrained_models PRETRAINED_MODELS [PRETRAINED_MODELS ...]
                        Pretrained models to download: resnet_50, resnet_101,
                        mobilenet_1.0_224

For example, to download and extract datasets and models into directories specified:

$ python download.py --downloads_dir ./downloads --data_dir ./data --pretrained_models_dir ./models/pretrained --pretrained_models resnet_50 resnet_101 mobilenet_1.0_224

For simplicity, please just run the following command in terminal:

$ python download.py

Train Model

$ python train.py --help
usage: train.py [-h] [--network_backbone NETWORK_BACKBONE]
                [--pre_trained_model PRE_TRAINED_MODEL]
                [--trainset_filename TRAINSET_FILENAME]
                [--valset_filename VALSET_FILENAME] [--images_dir IMAGES_DIR]
                [--labels_dir LABELS_DIR]
                [--trainset_augmented_filename TRAINSET_AUGMENTED_FILENAME]
                [--images_augmented_dir IMAGES_AUGMENTED_DIR]
                [--labels_augmented_dir LABELS_AUGMENTED_DIR]
                [--model_dir MODEL_DIR] [--log_dir LOG_DIR]
                [--random_seed RANDOM_SEED]

Train DeepLab V3 for image semantic segmantation.

optional arguments:
  -h, --help            show this help message and exit
  --network_backbone NETWORK_BACKBONE
                        Network backbones: resnet_50, resnet_101,
                        mobilenet_1.0_224. Default: resnet_101
  --pre_trained_model PRE_TRAINED_MODEL
                        Pretrained model directory
  --trainset_filename TRAINSET_FILENAME
                        Train dataset filename
  --valset_filename VALSET_FILENAME
                        Validation dataset filename
  --images_dir IMAGES_DIR
                        Images directory
  --labels_dir LABELS_DIR
                        Labels directory
  --trainset_augmented_filename TRAINSET_AUGMENTED_FILENAME
                        Train augmented dataset filename
  --images_augmented_dir IMAGES_AUGMENTED_DIR
                        Images augmented directory
  --labels_augmented_dir LABELS_AUGMENTED_DIR
                        Labels augmented directory
  --model_dir MODEL_DIR
                        Trained model saving directory
  --log_dir LOG_DIR     TensorBoard log directory
  --random_seed RANDOM_SEED
                        Random seed for model training.

For simplicity, please run the following command in terminal:

$ python train.py

With learning rate of 1e-5, the mIOU could be greater 0.7 after 20 epochs, which is comparable to the test statistics of DeepLab V3 in the publication.

Demos

To show some demos, please run the following command in terminal:

$ python test_demo.py

Image	Label	Prediction

Image	Label	Prediction

Image	Label	Prediction

Image	Label	Prediction

Custom Demo

Just put some JPG-format images into demo_dir and run the following command in the terminal.

$ python test_any_image.py

Results will be written into same folder. Make sure that proper model trained and a checkpoint is saved in models_dir. See the script for details.

Contributed by pinaxe1. Will modify to accept arguments and multiple image formats.

References

L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille. Deeplab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. TPAMI, 2017.

L.-C. Chen, G. Papandreou, F. Schroff, and H. Adam. Rethinking Atrous Convolution for Semantic Image Segmentation. arXiv:1706.05587, 2017.

L.-C. Chen, Y. Zhu, G. Papandreou, F. Schroff, H. Adam. Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. arXiv:1802.02611, 2018.

To-Do List

Test script for new arbitrary test images.

leimao/DeepLab-V3