This code comes from the author of the paper: "Graph R-CNN for Scene Graph Generation". It is just used here for my master thesis. The author has also already uploaded their official code finally, please check there own repostory(https://github.com/jwyang/graph-rcnn.pytorch)
This is the pytorch implementation of our graph rcnn model. It completes three tasks: object detection, attribute recognition and relation detection for each image jointly, and obtain the graph representations meanwhile. Besides a good performance on these three seperate tasks, it is also expected to help a bunch of high level (downstream) tasks, such as image captioning, visual question answering, expression reference, etc. It is developed based on the following two projects:
- jwyang/faster-rcnn.pytorch, a pure, fast and memory efficient pytorch implementation of faster-rcnn.
- peteanderson80/bottom-up-attention, a more sophisticated visual representation for image captioning and visual question answering based on bottom-up attention model.
- danfeiX/scene-graph-TF-release, a secene graph detection model based on iterative message passsing.
Download Visual Genome. For this project, you will need:
- Images (part1, part2)
- Metadata
- Object Annotations, Object Alias
- Attribute Annotations
- Relation Annotations, Relation Alias
- Scene Graph
After downloading the above data, unzip all of them and put into a single folder. Then make a soft link to it via:
cd $REPO_ROOT
ln -s PATH/TO/YOUR/DATA_DIR data/vg
Then, prepare the pascal voc like xmls for visual genome using:
cd $REPO_ROOT
python data/genome/setup_vg.py
Create a folder:
cd $REPO_ROOT
mkdir data/pretrained_model
Download pretrained resnet101 and vgg16 models and put them into the folder.
To train a resnet101, run:
CUDA_VISIBLE_DEVICES=0 python trainval_grcnn.py --dataset vg1 --net res101
Alternatively, to train a vgg16, run:
CUDA_VISIBLE_DEVICES=0 python trainval_grcnn.py --dataset vg1 --net vgg16
CUDA_VISIBLE_DEVICES=0 python test_grcnn.py --dataset vg1 --net res101 --checksession 1 --checkepoch 11 --checkpoint 6224 --imdb test --mGPUs True