AssertionError: Metric file does not exist
ryoma-yataka opened this issue · 2 comments
I used the execution command you provided to train and evaluate the model.
Currently, I am running problems of 3DROOM_SMALL and 3D_ROOM_LARGE and getting the evaluation results.
However, on other problems (SPACE Invader and Atari), the following error occurs even if the evaluation command is executed.
Traceback (most recent call last):
File "main.py", line 16, in
task_dicttask
File "/import/homes/username/project/dnn/detection/SPACE/src/engine/eval.py", line 40, in eval
checkpoint = checkpointer.load_best(cfg.eval.metric, model, None, None, use_cpu)
File "/import/homes/username/project/dnn/detection/SPACE/src/utils.py", line 130, in load_best
assert osp.exists(metric_file), 'Metric file does not exist'
AssertionError: Metric file does not exist
I presume that the cause is that the "best_ap_avg.pth" file etc. is not generated. (I only got model_xxxx.pth and .pkl files as a result of learning for these atari and invader problems)
How can I get best_ap_avg.pth on the atari and invader training?
Thank you.
Hi @ryoma-yataka, this is because we don't have ground truth bounding boxes for the Atari games, so it's not possible to evaluate the AP on these tasks.
Thank you for your answer, @zhixuan-lin. I had overlooked the quantitative comparison section of the paper. And I understood that Atari and other problems conduct qualitative comparison only. Thank you very much for your kind answer.