fartashf/vsepp

problem in finetune

136823xuewei opened this issue · 6 comments

136823xuewei commented

I use the original flickr30k dataset rather than precomputed features ,and use the pretrained model resnet50.I set finetune True. But, i get poor result of recall. R@1,R@5,R@10 is nearly zero.I can't find the reason about the bad result.How do you set the parameters when you use resnet with finetune?
Thank you.

fartashf commented

Try learning rate 0.00002. Look at section 3.2 for more details of training.

136823xuewei commented

Thanks for your reply!

shengyuzhang commented

@136823xuewei Did you succeed in getting good results using raw images and finetuning rather than using precomputed features?

@fartashf I trained on the COCO dataset without precomputed features but failed to get good results (@1 less than 1.). Using precomputed features is all good.

The learning rate is 0.00002 when training on raw images.

Could someone help me?

fartashf commented

I can still reproduce the result of the following command using the latest commit on the master:

python train.py --data_name coco --logger_name runs/X --max_violation

It takes at least 1000 iterations to reach above 1% R@1 that is expected.

By default the learning rate is 0.0002. I reproduced it using python 2.7.14 and pytorch 0.3.1. For python 3 and pytorch 1 checkout the details on their corresponding branches.

winnechan commented

if we fix the image full encoder in the first epoch and train the joint embedding model, do we need to write some code to force the image cnn encoder always in evaluation mode? i.e. no dropout and ...

fartashf commented

Not necessarily. Even if the image encoder is not being trained, dropout might help in training robust embeddings.

  • 👍1