You may want to compare with similar steps listed on Train Fast-RCNN on Another Dataset and How to train fast rcnn on imagenet. Below is my dataset directory:
## Demo
<tr>
<td>
<img src="result/test_img/w_126_whale.png" width="45%" />
<img src="result/test_img/w_276_whale.png" width="45%" />
</td>
</tr>