CIAN: Cross-Image Afﬁnity Net for Weakly Supervised Semantic Segmentation

This is the code of:

CIAN: Cross-Image Afﬁnity Net for Weakly Supervised Semantic Segmentation, Junsong Fan, Zhaoxiang Zhang, Tieniu Tan, Chunfeng Song, Jun Xiao, AAAI2020 [paper].

Introuduction

Framework of the approach. Our approach learns the cross image relationships to help weakly-supervised semantic segmentation. The CIAN Module takes features as input from two different images, extract and change information across them to augment the original features.

Prerequisite

Python 3.7, MXNet 1.3.1, Numpy, OpenCV, pydensecrf.
NVIDIA GPUs

Usage

Prepare the dataset and pretrain parameters:

Download the VOC 2012 dataset, and config the path accordingly in the run_cian.sh.
Download the seeds here, untar and put the folder CIAN_SEEDS into CIAN/data/Seeds/ . We use the VGG16 based CAM to generate the foreground and the saliency model DRFI to generate the background. You can also generate the seeds by yourself.
Download the ImageNet pretrained parameters and put them into the folder CIAN/data/pretrained. We adopt the pretrained parameters provided by the official MXNet model-zoo:
- ResNet101 [http://data.mxnet.io/models/imagenet/resnet/101-layers/resnet-101-0000.params]
- ResNet50 [http://data.mxnet.io/models/imagenet/resnet/50-layers/resnet-50-0000.params]

Run:

./run_cian.sh

This script will automatically run the training, testing (on val set), and retraining pipeline. Checkpoints and predictions will be saved in folder CIAN/snapshot/CIAN.

Citation

If you find the code is useful, please consider citing:

@inproceedings{fan2020cian,
  title={CIAN: Cross-Image Afﬁnity Net for Weakly Supervised Semantic Segmentation},
  author={Fan, Junsong and Zhang, Zhaoxiang and Tan, Tieniu and Song, Chunfeng and Xiao, Jun},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
  year={2020}
}

js-fan/CIAN