poor results on teamtime
feixue94 opened this issue · 13 comments
Dear author,
Thanks for your excellent work.
I evaluated the released pretrained_model on teatime. The localization accuracy is only 0.1017.
Do you have any idea what the problem is.
2024-04-25 16:47:42,245 - teatime - INFO - trunc thresh: 0.4
INFO:teatime:trunc thresh: 0.4
2024-04-25 16:47:42,245 - teatime - INFO - iou chosen: 0.0245
INFO:teatime:iou chosen: 0.0245
2024-04-25 16:47:42,248 - teatime - INFO - chosen_lvl:
[array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0)]
INFO:teatime:chosen_lvl:
[array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0)]
2024-04-25 16:47:42,248 - teatime - INFO - Localization accuracy: 0.1017
INFO:teatime:Localization accuracy: 0.1017
Hi,
Thanks for your attention.
All of your chosen level is 0, which means that the model are selecting only Level S results. Have you correctly loaded the models for all three levels?
thanks for your reply. I figured it out.
Hi, @feixue94 and @minghanqin ! I also obtained similar bad results after runing eval.sh. Could you please tell me how to fix it? This was how I executed:
python render.py -m output/teatime_1 --feature_level 1
python render.py -m output/teatime_2 --feature_level 2
python render.py -m output/teatime_3 --feature_level 3
cd eval
bash eval.sh
Hi @davidpengiupui ,
My problem was that the only features of level 1 was extracted.
Hi @feixue94 ! Thank you for your reply! Do you mean that I need to extract things like teatime_1, teatime_2 and teatime_3 in the argument "feat_dir"? I have done that but still got poor results. Do you change something in the eval.sh regarding this?
I think you need to to extract multi-level features again.
I think you need to to extract multi-level features again.
I understand. Thank you!
I think you need to to extract multi-level features again.
I tried the command below and still got the same error.
python render.py -m output/teatime_1
python render.py -m output/teatime_2
python render.py -m output/teatime_3
What is the right command to extract multi-level features?
Hi @yquantao please check if you have 3 encoders for 3 different levels of feature
Hi, @feixue94 and @minghanqin ! I also obtained similar bad results after runing eval.sh. Could you please tell me how to fix it? This was how I executed:
python render.py -m output/teatime_1 --feature_level 1 python render.py -m output/teatime_2 --feature_level 2 python render.py -m output/teatime_3 --feature_level 3
cd eval bash eval.sh
how to extract multi-level features? I have the same problem here, have you fixed it?
hi, after I runned:
python render.py -m output/teatime_1 --feature_level 1
python render.py -m output/teatime_2 --feature_level 2
python render.py -m output/teatime_3 --feature_level 3
sh eval.sh
I got the poor result like yours, how to make sure my "3 encoders for 3 different levels of feature"?
I have the same problem. How to fix it?