About token bank selection module

[wenqi-wang20]

Hi, thank you for your great work. I was trying to reproduce the pre-training and fine-tuneing process. However, I have several questions:

1. Are the pre-training model and fine-tuning model with the same arch? I saw there is only one model in blip2_3dvqa_t5_elm.py. But I think there is no temporal data in pre-training dataset? Is there still a token selection module?
2. Is the blip2_3dvqa_t5_elm model for only pretraining? Could you please point out that where is the proposed token bank module? I am so sorry that there are too many files and I can not find them by my own.

Thank you very much!

[zhouyunsong]

Hello, thank you for your interest.

1. The model arch for pre-training and fine-tuning is the same, except that the latter will have a token bank, which was not published in the previous version of the code and has been updated.
2. advqa_t5_elm.yaml is for pre-training only, you can check advqa_t5_lr.yaml for token bank.