How to make evaluation on VQA?

[yangapku]

Hi, I have trained the fine-tuned visualbert model on VQA following the instructions in the readme. But I have no idea how to make predictions on the official VQA v2.0 test-dev set and compute the accuracy score which is comparable with the mentioned performance in your paper (70.80)? Thank you very much!

[liunian-harold-li]

Hi, to valuate on VQA, just change "skip_training" in the config to False, and "do_test" in the config to True. And run "train.py" as before.

But when running "train.py", the "-folder" should be the same as the folder that you stored your fine-tuned VQA model (the code will look for "model_state_epoch_X.th" and "training_state_epoch_X.th" in the folder).

Thanks!