/PRM

Weakly Supervised Instance Segmentation using Class Peak Response, in CVPR 2018 (Spotlight)

Weakly Supervised Instance Segmentation using
Class Peak Response

[Home] [Project] [Paper] [Supp] [Poster] [Presentation]

Illustration

PyTorch Implementation

The pytorch branch contains:

  • the pytorch implementation of Peak Response Mapping (Stimulation and Backprop).
  • the PASCAL-VOC demo (training, inference, and visualization).

Please follow the instruction below to install it and run the experiment demo.

Prerequisites

  • System (tested on Ubuntu 14.04LTS and Win10)

  • NVIDIA GPU + CUDA CuDNN (CPU mode is also supported but significantly slower)

  • Python>=3.5

  • PyTorch>=0.4

  • Jupyter Notebook and ipywidgets (required by the demo):

    # enable the widgetsnbextension before you start the notebook server
    jupyter nbextension enable --py --sys-prefix widgetsnbextension

Installation

  1. Install Nest, a flexible tool for building and sharing deep learning modules:

    I created Nest in the process of refactoring PRM's pytorch implementation. It aims at encouraging code reuse and ships with a bunch of useful features. PRM is now implemented as a set of Nest modules; thus you can easily install and use it as demonstrated below.

    $ pip install git+https://github.com/ZhouYanzhao/Nest.git
  2. Install PRM via Nest's CLI tool:

    # note that data will be saved under your current path
    $ nest module install github@ZhouYanzhao/PRM:pytorch prm
    # verify the installation
    $ nest module list --filter prm
    # Output:
    #
    # 3 Nest modules found.
    # [0] prm.fc_resnet50 (1.0.0)
    # [1] prm.peak_response_mapping (1.0.0)
    # [2] prm.prm_visualize (1.0.0)

Run demo

  1. Install Nest's build-in Pytorch modules:

    To increase reusability, I abstracted some features from the original code, such as network trainer, to build Nest's built-in pytorch module set.

    $ nest module install github@ZhouYanzhao/Nest:pytorch pytorch
  2. Download the PASCAL-VOC2012 dataset:

    mkdir ./PRM/demo/datasets
    cd ./PRM/demo/datasets
    # download and extract data
    wget http://host.robots.ox.ac.uk/pascal/VOC/voc2012/VOCtrainval_11-May-2012.tar
    tar xvf VOCtrainval_11-May-2012.tar
  3. Run the demo experiment via demo/main.ipynb

    PRM Segmentation

Citation

If you find the code useful for your research, please cite:

@INPROCEEDINGS{Zhou2018PRM,
    author = {Zhou, Yanzhao and Zhu, Yi and Ye, Qixiang and Qiu, Qiang and Jiao, Jianbin},
    title = {Weakly Supervised Instance Segmentation using Class Peak Response},
    booktitle = {CVPR},
    year = {2018}
}