MILVLG/mcan-vqa

Overfitting on val dataset

cikers opened this issue · 2 comments

Hi, i have one question.
I train the model, details as follow: model='small', train_split='train+val', 13 epoches.
The val acc is overall=84.18, yes/no=97.67, num=71.03, other=77.39.
Is it overfitting on the val dataset?
I tried to add dropout rate to 0.5, it works(val: overall=72.31),
but the test score is overall=66.95 (the same implements).
What should i do for this problem?
Thank you!

You are using train+val split for training, which means the val split is augmented in the training set. This will definitely lead to overfitting. The train+val split is only used to report the test-dev performance. If you want to evaluate the performance on val split, you should use train_split=train

ok,thank you!
it's my fault.