airsplay/R2R-EnvDrop

Test Question

xiran2018 opened this issue · 2 comments

xiran2018 commented

After I train the model， i use the test environment to evaulate，the success rate result is below， i dont understand why the result is so low？ please help me， is there something wrong when i test ？
image
the test script is：
name=agent
flag="--train validlistener --featdropout 0.3 --angleFeatSize 128
--feedback argmax
--mlWeight 0.2
--subout max --dropout 0.5 --optim rms --lr 1e-4 --iters 80000 --submit"
CUDA_VISIBLE_DEVICES=$1 python r2r_src/train.py $flag --name $name

airsplay commented

Thanks.

It is because the test data does not provide the ground truth label. It only gives the start viewpoint so the accuracy is actually the frequency of [final-viewpoint == start-viewpoint].

To get the test results, please use the testing servers here https://evalai.cloudcv.org/web/challenges/challenge-page/97/overview. However, there is a limitation of 5 trails.

xiran2018 commented

Thanks.

It is because the test data does not provide the ground truth label. It only gives the start viewpoint so the accuracy is actually the frequency of [final-viewpoint == start-viewpoint].

To get the test results, please use the testing servers here https://evalai.cloudcv.org/web/challenges/challenge-page/97/overview. However, there is a limitation of 5 trails.

thank u very much for your reply, I appreicate u work