microsoft/TAP

No targets for training

abhinavkcs11 opened this issue · 1 comments

When I train the VQA model I get the warning "Sample list has not field 'targets', are you sure that your ImDB has labels? you may have wanted to run with --evalai_inference 1"

I executed the same command as mentioned:
python -m torch.distributed.launch --nproc_per_node 4 tools/run.py --tasks vqa --datasets m4c_textvqa --model m4c_split --seed 13 --config configs/vqa/m4c_textvqa/tap_refine.yml --save_dir save/m4c_split_refine_test --resume_file save/pretrained/textvqa_tap_base_pretrain.ckpt training_parameters.distributed True

Can you provide additional details on this and how to train the model with the targets? And can you point out where the targets and the predictions are getting compared to compute loss?

Hi @abhinavkcs11 ,

The loss function is here: https://github.com/microsoft/TAP/blob/main/pythia/modules/losses.py.

Maybe we could trace from the "sample_list" in the loss functions, back to the data loader (e.g.,

sample.targets = processed_answers["answers_scores"]
) and see which part is not working as expected? Thank you.