Swapped test results shown in eval_fastrcnn_voc2007.sh?

[desertnaut]

Hi all,

Is it possible that the test results of eval_fastrcnn_voc2007.sh shown here are swapped between the two models? I run the script with test_model=data/models/caffenet_fast_rcnn_iter_40000.t7, and get the results reported for the vgg16_fast_rcnn_iter_40000.t7 model:

ubuntu@ip-172-31-24-199:~/multipathnet$ scripts/eval_fastrcnn_voc2007.sh
{
  year : "2007"
  test_nsamples : -1
  test_augment : false
  proposals : "selective_search"
  test_model : "./data/models/caffenet_fast_rcnn_iter_40000.t7"
  max_size : 1000
  test_best_proposals_number : 2000
  test_save_res : ""
  test_add_nosoftmax : false
  test_save_raw : ""
  disable_memory_efficient_forward : false
  test_data_offset : -1
  proposal_dir : "./data/proposals"
  test_save_res_prefix : ""
  test_bbox_voting_nms_threshold : 0.5
  test_num_iterative_loc : 1
  scale : 600
  dataset : "pascal"
  test_min_proposal_size : 2
  test_nGPU : 4
  test_load_aboxes : ""
  test_bbox_voting : false
  test_just_save_boxes : false
  transformer : "RossTransformer"
  test_bbox_voting_score_pow : 1
  test_set : "test"
  test_use_rbox_scores : false
  test_nms_threshold : 0.3
}
[...]
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.264
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.559
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.217
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.034
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.132
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.318
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.304
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.400
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.408
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.140
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.285
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.456

(Unfortunately, I cannot cross-check this with test_model=data/models/vgg16_fast_rcnn_iter_40000.t7, as I get a CUDA out of memory error)

[szagoruyko]

@desertnaut you're right, thanks for pointing out.

[desertnaut]

OK, just confirmed it also with test_model=data/models/vgg16_fast_rcnn_iter_40000.t7 (changing to test_nGPU=1 I was able to run it without memory issues).