pytorch_MLEM

News. This repo supports pytorch-1.0 and higher version now!!! I borrowed code from mmdetection and also some implementation idea.

This is a simplified version of MELM with context in pytorch for the paper《Min-Entropy Latent Model for Weakly Supervised Object Detection》,which is a accepted paper in CVPR2018 and TPAMI.

This implementation is based on Winfrand's which is the official version based on torch7 and lua. This implementation is also based on ruotianluo's pytorch-faster-rcnn.

And trained on PASCAL_VOC 2007 trainval and tested on PASCAL_VOC test with VGG16 backbone, I got a performance mAP 47.98 a little better than the paper's result

If you find MELM useful and use this code, please cite our paper:

@inproceedings{wan2018min,
  title={Min-Entropy Latent Model for Weakly Supervised Object Detection},
  author={Wan, Fang and Wei, Pengxu and Jiao, Jianbin and Han, Zhenjun and Ye, Qixiang},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  pages={1297--1306},
  year={2018}
}

@article{wan2019Pami,
  author    = {Fang Wan and 
               Pengxu Wei and
               Jianbin Jiao and
               Zhenjun Han and 
               Qixiang Ye},
  title     = {Min-Entropy Latent Model for Weakly Supervised Object Detection},
  journal   = {{IEEE} Trans. Pattern Anal. Mach. Intell.},
  volume       = {DOI:10.1109/TPAMI.2019.2898858},
  year      = {2019}
}

Prerequisites

Nvidia GPU 1080Ti
Ubuntu 16.04 LTS
python 3.6
pytorch 0.4 is required. For pytorch 1.0 or higher version, please go to the pytorch1.0 version.
tensorflow, tensorboard and tensorboardX for visualizing training and validation curve.

Installation

Clone the repository

git clone https://github.com/vasgaowei/pytorch_MELM.git

Compile the modules(nms, roi_pooling, roi_ring_pooling and roi_align)

cd pytorch_MELM/lib
bash make.sh

Setup the data

Download the training, validation, test data and the VOCdevkit

cd pytorch_MELM/
mkdir data
cd data/
wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtrainval_06-Nov-2007.tar
wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtest_06-Nov-2007.tar
wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCdevkit_08-Jun-2007.tar

Extract all of these tars into one directory named VOCdevkit

tar xvf VOCtrainval_06-Nov-2007.tar
tar xvf VOCtest_06-Nov-2007.tar
tar xvf VOCdevkit_08-Jun-2007.tar

Create symlinks for PASCAL VOC dataset or just rename the VOCdevkit to VOCdevkit2007

cd pytorch_MELM/data
ln -s VOCdevkit VOCdevkit2007

It should have this basic structure

$VOCdevkit2007/                     # development kit
$VOCdevkit2007/VOC2007/             # VOC utility code
$VOCdevkit2007/VOCcode/             # image sets, annodations, etc

And for PASCAL VOC 2010 and PASCAL VOC 2012, just following the similar steps.

Download the pre-trained ImageNet models

Downloa the pre-trained ImageNet models from https://drive.google.com/drive/folders/0B1_fAEgxdnvJSmF3YUlZcHFqWTQ or download from https://drive.google.com/drive/folders/1FV6ZOHOxLMQjE4ujTNOObI7lN8USH0v_?usp=sharing and put in in the data/imagenet_weights and rename it vgg16.pth. The folder has the following form.

$ data/imagenet_weights/vgg16.pth
$ data/imagenet_weights/res50.pth

Download the Selective Search proposals for PASCAL VOC 2007

Download it from: https://dl.dropboxusercontent.com/s/orrt7o6bp6ae0tc/selective_search_data.tgz and unzip it and the final folder has the following form

$ data/selective_search_data/voc_2007_train.mat
$ data/selective_search_data/voc_2007_test.mat
$ data/selective_search_data/voc_2007_trainval.mat

Train your own model

For vgg16 backbone, we can train the model using the following commands

./experiments/scripts/train_faster_rcnn.sh 0 pascal_voc vgg16

And for test, we can using the following commands

./experiments/scripts/test_faster_rcnn.sh 0 pascal_voc vgg16

Visualizing some detection results

I have pretrained MLEM_pytorch model on PASCAL VOC 2007 based on vgg16 backbone and you can download it from https://drive.google.com/drive/folders/1FV6ZOHOxLMQjE4ujTNOObI7lN8USH0v_?usp=sharing and put it in the folder output vgg16/voc_2007_trainval/default/vgg16_MELM.pth and run the following commands.

cd pytorch_MELM
python ./tools/demo.py --net vgg16 --dataset pascal_voc

Also you can visualize training and validation curve.

tensorboard --logdir tensorboard/vgg16/voc_2007_trainval/