Faster R-CNN C++ Inference

The above code is an interface inference in c++ against a Faster R-CNN trained network.

From the code developed by Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun (Microsoft Research) for the project py-faster-rcnn, some modules and files from the py-faster-rcnn/lib and py-faster-rcnn/data/scripts/ folders are used directly:

RPN module (py-faster-rcnn/lib/rpn/): Needed to deploy the region proposal network.
NMS module (py-faster-rcnn/lib/nms/): Needed to apply non-maximum suppression step.
Fast_rcnn module (py-faster-rcnn/lib/fast_rcnn/): Contains auxiliary functions.
py-faster-rcnn/lib/Makefile and py-faster-rcnn/lib/setup.py: To compile NMS CUDA and Cython libraries.
py-faster-rcnn/data/scripts/fetch_faster_rcnn_models.sh: To download pre-computed Faster R-CNN detectors.

This code is added as a submodule to the present project for convenience. It also uses their branch of the framework Caffe (caffe-fast-rcnn).

FASTER_RCNN C++ class has adapated (copy/modify/correct) some fuctions from the project FasterRCNN-Encapsulation-Cplusplus. Thanks so much to his SourceCode.

Steps to use the code

First of all, clone this project with --recursive flag:

git clone --recursive https://github.com/HermannHesse/faster_rcnn_cplusplus.git

Requeriments inherited from py-faster-rcnn

Requirements for Caffe and pycaffe (see: Caffe installation instructions)

Note: Caffe must be built with support for Python layers!
```
# In your Makefile.config, make sure to have this line uncommented
WITH_PYTHON_LAYER := 1
# Unrelatedly, it's also recommended that you use CUDNN
USE_CUDNN := 1
```
You can download Ross Girshick Makefile.config for reference.
Python packages you might not have: cython, python-opencv, easydict

Build Caffe and pycaffe

cd $ROOT_DIR/py-faster-rcnn/caffe-fast-rcnn/
# Now follow the Caffe installation instructions here:
#   http://caffe.berkeleyvision.org/installation.html

# If you're experienced with Caffe and have all of the requirements installed
# and your Makefile.config in place, then simply do:
make -j8 && make pycaffe

Build the Cython modules
```
cd $ROOT_DIR/py-faster-rcnn/lib/
make
```

Download pre-computed Faster R-CNN detectors

cd $ROOT_DIR/py-faster-rcnn/
./data/scripts/fetch_faster_rcnn_models.sh

Own requeriments

Set $ROOT_DIR/py-faster-rcnn/caffe-fast-rcnn/python/ and $ROOT_DIR/py-faster-rcnn/lib/ into the enviroment variable PYTHONPATH
```
export PYTHONPATH=$ROOT_DIR/py-faster-rcnn/caffe-fast-rcnn/python/:$ROOT_DIR/py-faster-rcnn/lib/:${PYTHONPATH}
```

Compile the project with CMake

mkdir $ROOT_DIR/build/
cd $ROOT_DIR/build/
cmake ..
make
mv faster_rcnn_cplusplus ..

Run the demo
```
cd $ROOT_DIR
./faster_rcnn_cplusplus
```

Popular issues

Issue 1

"Unknown layer type: Python"

Caffe has been probably compiled without PYTHON_LAYER support. Uncomment the following line

WITH_PYTHON_LAYER := 1

in $ROOT_DIR/py-faster-rcnn/caffe-fast-rcnn/Makefile.config file and recompile it.

cd $ROOT_DIR/py-faster-rcnn/caffe-fast-rcnn/
make clean
make -j8 && make pycaffe

Issue 2

fatal error: caffe/proto/caffe.pb.h: No such file or directory
#include "caffe/proto/caffe.pb.h"
                                  ^
compilation terminated.

caffe.pb.h is a header file generated by Google Protocol Buffer and it is missing for some reason. Let's create it.

cd $ROOT_DIR/py-faster-rcnn/caffe-fast-rcnn/
protoc src/caffe/proto/caffe.proto --cpp_out=.
mkdir include/caffe/proto
mv src/caffe/proto/caffe.pb.h include/caffe/proto

Reference full discussion from here.

Issue 3

When the net is loading Caffe can't find rpn python layer:

ImportError: No module named rpn.proposal_layer

Add lib/ directory to PYTHONPATH:

export PYTHONPATH=$ROOT_DIR/py-faster-rcnn/lib/:${PYTHONPATH}

Issue 4

Python can't find gpu_nms library:

from nms.gpu_nms import gpu_nms
ImportError: No module named gpu_nms

Must compile libraries gpu_nms.so and cpu_nms.sodone by running the fourth step:

cd $ROOT_DIR/py-faster-rcnn/lib/
make

Issue 5

/usr/bin/ld: cannot find -lopencv_dep_cudart 
collect2: error: ld returned 1 exit status

There is a problem with the CMakeCache.txt and the enviroment variable CUDA_USE_STATIC_CUDA_RUNTIME. To solve this issue, set it OFF in CMakeList.txt:

# Add the following line to CMakeList.txt and recompile
set(CUDA_USE_STATIC_CUDA_RUNTIME "OFF")

or generate CMake files with:

cd $ROOT_DIR/build/
cmake .. -D CUDA_USE_STATIC_CUDA_RUNTIME=OFF

Reference full discussion from here.