Faster R-CNN Implementation in Pytorch

This repository implements Faster R-CNN with training, inference and map evaluation in PyTorch. The aim was to create a simple implementation based on PyTorch faster r-cnn codebase and to get rid of all the abstractions and make the implementation easy to understand.

The implementation caters to batch size of 1 only and uses roi pooling on single scale feature map. The repo is meant to train faster r-cnn on voc dataset. Specifically I trained on VOC 2007 dataset.

Faster R-CNN Explanation Video

Faster R-CNN Implementation Video

Faster R-CNN PyTorch Code Walkthrough Video

Sample Output by training Faster R-CNN on VOC 2007 dataset

Ground Truth(Left) | Prediction(right)

Data preparation

For setting up the VOC 2007 dataset:

Download VOC 2007 train/val data from http://host.robots.ox.ac.uk/pascal/VOC/voc2007 and name it as VOC2007 folder
Download VOC 2007 test data from http://host.robots.ox.ac.uk/pascal/VOC/voc2007 and name it as VOC2007-test folder

Place both the directories inside the root folder of repo according to below structure

FasterRCNN-Pytorch
    -> VOC2007
        -> JPEGImages
        -> Annotations
    -> VOC2007-test
        -> JPEGImages
        -> Annotations
    -> tools
        -> train.py
        -> infer.py
        -> train_torchvision_frcnn.py
        -> infer_torchvision_frcnn.py
    -> config
        -> voc.yaml
    -> model
        -> faster_rcnn.py
    -> dataset
        -> voc.py

For training on your own dataset

Copy the VOC config(config/voc.yaml) and update the dataset_params and change the task_name as well as ckpt_name based on your own dataset.

Copy the VOC dataset(dataset/voc.py) class and make following changes:

Update the classes list here (excluding background).

Modify the load_images_and_anns method to returns a list of im_infos for all images, where each im_info is a dictionary with following keys:

 im_info : {
   'filename' : <image path>
   'detections' : 
   	[
   		'label': <integer class label for this detection>, # assuming the same order as classes list present above, with background as zero index.
   		'bbox' : list of x1,y1,x2,y2 for the bboxes.
   	]
   }

Ensure that __getitem__ returns the following:

im_tensor(C x H x W) , 
target{
      'bboxes': Number of Gts x 4,
      'labels': Number of Gts,
      }
file_path(just used for debugging)

Change the training script to use your dataset here
Then run training with the desired config passed as argument.

Differences from Faster RCNN paper

This repo has some differences from actual Faster RCNN paper.

Caters to single batch size
Uses a randomly initialized fc6 fc7 layer of 1024 dim.
Most of the hyper-parameters have directly been picked from official version and have not been tuned to this setting of 1024 dimensional fc layers. As of now using this I am getting ~61-62% mAP.
To improve the results one can try the following:
- Use VGG fc6 and fc7 layers
- Tune the weight of different losses
- Experiment with roi batch size
- Experiment with hard negative mining

For modifications

To change the fc dimension , change fc_inner_dim in config
To use a different backbone, make the change here and also change backbone_out_channels in config
To use hard negative mining change roi_low_bg_iou to say 0.1(this will ignore proposals with < 0.1 iou)
To use gradient accumulation change acc_steps in config to > 1

Quickstart

Create a new conda environment with python 3.8 then run below commands
git clone https://github.com/explainingai-code/FasterRCNN-PyTorch.git
cd FasterRCNN-PyTorch
pip install -r requirements.txt
For training/inference use the below commands passing the desired configuration file as the config argument .
python -m tools.train for training Faster R-CNN on voc dataset
python -m tools.infer --evaluate False --infer_samples True for generating inference predictions
python -m tools.infer --evaluate True --infer_samples False for evaluating on test dataset

Using torchvision FasterRCNN

For training/inference using torchvision faster rcnn codebase, use the below commands passing the desired configuration file as the config argument.
python -m tools.train_torchvision_frcnn for training using torchvision pretrained Faster R-CNN class on voc dataset
- This uses the following arguments other than config file
- --use_resnet50_fpn
  - True(default) - Use pretrained torchvision faster rcnn
  - False - Build your own custom model using torchvision faster rcnn class)
python -m tools.infer_torchvision_frcnn for inference and testing purposes. Pass the desired configuration file as the config argument.
- This uses the following arguments other than config file
- --use_resnet50_fpn
  - True(default) - Use pretrained torchvision faster rcnn
  - False - Build your own custom model using torchvision faster rcnn class)
  - Should be same value as used during training
- --evaluate (Whether to evaluate mAP on test dataset or not, default value is False)
- -- infer_samples (Whether to generate predicitons on some sample test images, default value is True)

Configuration

config/voc.yaml - Allows you to play with different components of faster r-cnn on voc dataset

Output

Outputs will be saved according to the configuration present in yaml files.

For every run a folder of task_name key in config will be created

During training of FasterRCNN the following output will be saved

Latest Model checkpoint in task_name directory

During inference the following output will be saved

Sample prediction outputs for images in task_name/samples/*.png

Citations

@article{DBLP:journals/corr/RenHG015,
  author       = {Shaoqing Ren and
                  Kaiming He and
                  Ross B. Girshick and
                  Jian Sun},
  title        = {Faster {R-CNN:} Towards Real-Time Object Detection with Region Proposal
                  Networks},
  journal      = {CoRR},
  volume       = {abs/1506.01497},
  year         = {2015},
  url          = {http://arxiv.org/abs/1506.01497},
  eprinttype    = {arXiv},
  eprint       = {1506.01497},
  timestamp    = {Mon, 13 Aug 2018 16:46:02 +0200},
  biburl       = {https://dblp.org/rec/journals/corr/RenHG015.bib},
  bibsource    = {dblp computer science bibliography, https://dblp.org}
}

TaiPhillips/FasterRCNN-PyTorch