Seed, Expand, Constrain: Three Principles for Weakly-Supervised Image Segmentation

PyTorch implementation of "Seed, Expand, Constrain: Three Principles for Weakly-Supervised Image Segmentation", ECCV2016

This is not the official repository for this paper. For the official, please see the following links.

Paper : https://arxiv.org/abs/1603.06098
Official code : caffe implmentation

Introduction

This is a work that proposes a new composite loss function for training convolutional neural network for the task of weakly-supervised semantic segmentation. Three novel loss functions are introduced:

Seeding loss
Expasion loss
Contrain-to-boundary loss

Updates

19 Jul, 2020: upload PascalVOC pretrained model

02 Jan, 2020: upload COCO implementation

11 Nov, 2019: Initial upload

Prerequisites

Python 3.6
PyTorch >= 1.0.0
Torchvision >= 0.2.2
PIL
opencv-python (OpenCV for Python)
tqdm
tensorboardX

Fully connected CRF wrapper (requires the Eigen3 Package)

apt-get install libeigen3-dev

# this should be done after download the source..
pip install CRF/

Data & Model Preparation

Pascal VOC 2012 dataset (VOC2012) is used for this implementation.

Download VOC2012 from here
You can see the detail of VOC2012 in http://host.robots.ox.ac.uk/pascal/VOC/voc2012/index.html

We use ImageNet pretrained model which is coverted from Caffe.

Download the pretrained model from here
You can convert it on your own. Please see here for more details.

Execution

download the source code & localization cue preparation

git clone https://github.com/halbielee/SEC_pytorch.git
cd SEC_pytorch 

# localizatio-cue preparation
gzip -kd datalist/PascalVOC/localization_cues.pickle.gz

train

# Before executing this, please set the appropriate dataset path
bash script/train.sh

test (generate the prediction map)

# Before executing this, please set the appropriate dataset path and other options..
bash script/test_multiprocess.sh

evaluation (calculate the performance)

# Before executing this, please set the appropriate prediction_map / gt_map path
bash script/evaluation.sh

Performance

We evaluate the PyTorch implementation with hyperparms which the author provided without any other tuning.

Method	Dataset	Backbone	mIOU	Download
SEC	VOC2012-val	DeepLab-LargeFOV	50.6049	⬇️
SEC	VOC2012-val	DeepLab-LargeFOV	49.6978	⬇️

Segmentation Result

Origin : Prediction : Ground Truth

{: widths="100%")