AmoebaNet does not use the full validation set
ddkang opened this issue · 8 comments
@bignamehyp as per the other thread, setting "eval_batch_size=1024" is incorrect, as it will skip parts of the validation set. Could you rerun the validation checkpoints using a correct batch size and update the TSVs and json files?
I'm on vacation util June 4th with limited Internet connectivity. Is there a deadline for this? I will try to squeeze some time to run the evaluation again.
@bignamehyp we don't have an explicit deadline, but we would like to resolve this in a timely manner. Can anyone else from your team re-evaluate your submissions?
I will start rerun the evaluation with eval_batch_size=1000 and update the accuracies soon.
@ddkang @codyaustun please take a look. Thanks
Great, thanks. The updated TSVs look good, but can you also update the JSON files to use eval_batch_size=1000?
Done
Thanks!