why performance of Mask_rcn tensorrt-fp16_dynamic-320x320-1344x1344 is bad
azuryl opened this issue · 1 comments
python ./tools/test.py configs/mmdet/instance-seg/instance-seg_tensorrt-fp16_dynamic-320x320-1344x1344.py /data/azuryl/mmdetection_2.27.0/configs/mask_rcnn/mask_rcnn_r50_fpn_2x_coco.py --model /data/azuryl/mmdeploy_model/maskrcnn_f16_d320_1344/end2end.engine --metrics segm --device cuda:0
/data/azuryl/mmdetection_2.27.0/mmdet/datasets/utils.py:70: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
'data pipeline in your config file.', UserWarning)
loading annotations into memory...
Done (t=2.09s)
creating index...
index created!
2021-07-23 02:54:25,221 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /data/azuryl/mmdeploy_0.7.0/mmdeploy/lib/libmmdeploy_tensorrt_ops.so
2021-07-23 02:54:25,222 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /data/azuryl/mmdeploy_0.7.0/mmdeploy/lib/libmmdeploy_tensorrt_ops.so
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 4952/4952, 4.0 task/s, elapsed: 1242s, ETA: 0s
Evaluating segm...
/data/azuryl/mmdetection_2.27.0/mmdet/datasets/coco.py:474: UserWarning: The key "bbox" is deleted for more accurate mask AP of small/medium/large instances since v2.12.0. This does not change the overall mAP calculation.
UserWarning)
Loading and preparing results...
DONE (t=8.15s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type segm
DONE (t=154.80s).
Accumulating evaluation results...
DONE (t=21.92s).
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.196
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=1000 ] = 0.394
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=1000 ] = 0.173
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.009
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.160
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.466
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.278
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.278
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=1000 ] = 0.278
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.027
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.253
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.586
2021-07-23 03:19:01,735 - test - INFO - OrderedDict([('segm_mAP', 0.196), ('segm_mAP_50', 0.394), ('segm_mAP_75', 0.173), ('segm_mAP_s', 0.009), ('segm_mAP_m', 0.16), ('segm_mAP_l', 0.466), ('segm_mAP_copypaste', '0.196 0.394 0.173 0.009 0.160 0.466')])