Consistently Low Acc for Jester Pretrained models

Question

Consistently Low Acc for Jester Pretrained models

gtangg12 opened this issue 4 years ago · 2 comments

gtangg12 commented 4 years ago

Hi,

The pretrained Jester models give around 50% acc at best. I tried MobileNetV2, MobileNet, and ShuffleNet. Any tips?

Thanks

Answer 1 · 2021-05-18T02:53:49.000Z

This has happened to me as well, with the default run script for jester and 0.5x MobileNet on the validation set!

Answer 2 · 2022-11-26T06:37:26.000Z

Someone found the issue a while back:

"I found the mistake that the authors had made that caused me these results. The authors to generate the model invoke the "generate_model" method which always inserts an untrained Linear level as the last level, so I added an "if statement "which in the test case returns the model as it was loaded from the checkpoint. With these changes the performances are similar to those declared by the authors, that is '0.950561' (top5) and '0.624915' on validation. The only consideration is that the results declared by the authors are on the top5 accuracy and not the top1."