wkentaro/chainer-mask-rcnn

Qualitative differences when running the examples

Closed this issue · 4 comments

m3at commented

Hi, first thanks a ton for that clean implementation of Mask-RCNN, it's very helpful! 🙂

When trying to reproduce the results shown in the inference example, I get qualitatively worst results (see bellow).

I'm running this on a 1070Ti on Ubuntu 18.04, with:

Platform: Linux-4.15.0-39-generic-x86_64-with-Ubuntu-18.04-bionic
Chainer: 5.0.0
NumPy: 1.15.4
CuPy:
  CuPy Version          : 5.0.0
  CUDA Root             : /usr
  CUDA Build Version    : 9010
  CUDA Driver Version   : 10000
  CUDA Runtime Version  : 9010
  cuDNN Build Version   : 7102
  cuDNN Version         : 7102
  NCCL Build Version    : 2115
iDeep: Not Available

chainercv: 0.11.0
numpy: 1.15.4
fcn: 6.4.8

It seems that you developed this on Chainer 4.0, but I could not find in the release notes what change in v5 could make this kind of impact.
Do you have any ideas about what specific change could have affect the results?

Alternatively, some other things could be happening on my side that impact the results, I will investigate as soon as I get the time this week and report it here.

Weights from Google Drive

using_gdrive_weights

Weights from Caffe2 (image 1)

using_caffe_weights_2

Weights from Caffe2 (image 2)

using_caffe_weights

wkentaro commented

Sorry. I have no idea at the moment.

wkentaro commented

@m3at Did you find anything that might cause this?

wkentaro commented

I've found that the problem is caused by some commits between 4.5.0 to 5.1.0, and the problem is fixed in 5.2.0.
I will filter 5.0.0 and 5.1.0 in requirements.txt via #37

m3at commented

@m3at Did you find anything that might cause this?

Sorry I had no opportunity to dig into it.

Thanks for finding the issue 👍