about the test data
genius9527 opened this issue · 7 comments
Hello, I used your method to train, and then used test.py to test, but found that the test accuracy is very low, I downloaded the pre-training model, tested, still very low.
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets= 20 ] = 0.188.
I suspect that my test data is wrong, I am using COCO2017/val2017,COCO_2017_val.json,person_keypoints_val2017.json
The data is stored according to data/README.md. I want to ask what your test data is. The test results are so good. Thank you.
Hi @genius9527
I also used COCO 2017 Keypoint data for training and test.
Sorry I don't know where your problem actually is. Have you modified any code?
Maybe you can make sure you have downloaded the full validation dataset and delete COCO_2017_val.json file(in annotations folder) and run label_transform.py(in ROOT_DIR) again to regenerate it.
ok,I will try. Can you pass your pre-training model to Baidu Cloud? I can't download anything on Google Cloud Drive.Thanks
@genius9527 Sorry to tell that I cannot upload the model to Baidu Cloud due to some limitations. If you still have the problem on downloading the pre-trained model, I can provide you the pre-trained model by Tencent weiyun: COCO.res50.256x192.CPN
@GengDavid thank you，I used the pre-training model you provided and found that the current test results are normal. The AP exceeded 0.7.But now I have a new problem, I try to use COCO2017/test2017 as the test data, and image_info_test-dev2017.json as the annotations. When I use test.py to test, the following problem occurs:
=> loaded checkpoint 'checkpoint/CPN256x192.pth.tar' (epoch 29)
testing...
0it [00:00, ?it/s]
loading annotations into memory...
Done (t=0.05s)
creating index...
index created!
Loading and preparing results...
Traceback (most recent call last):
File "test.py", line 159, in
main(parser.parse_args())
File "test.py", line 137, in main
eval_dt = eval_gt.loadRes(result_file)
File "/media/uestc/4T_1/pytorch-cpn-master/256.192.model/../cocoapi/PythonAPI/pycocotools/coco.py", line 318, in loadRes
if 'caption' in anns[0]:
IndexError: list index out of range
Have you ever encountered such a problem? Thanks.
I have not met this problem before. But it seems that there was something wrong in testing process since the test time was 00:00.
@GengDavid emmmm，So, have you tested it on the test dataset, how is it tested?
The test codes provided are not suit for evaluation on COCO test dataset. If you want to test it on test-dev, you may need to prepare the human detection results on test-dev and save it as the format shown in #2 first. And then you need to comment out line 136-141(which use coco tool to evaluate the results)in test.py. Finally, you can submit the results to COCO submission platform to get the evaluation result.
This is a pipeline for testing on test-dev. Some other changes are also needed. For example, you need to modify test_config to load the right dataset and detection bbox.