inference error

[DuZzzs]

Thank you for your excellent work. I built environment according to ducuments. But when I try ATSS, I got this error:
base env:

torch=1.6.0
tensorrt=7.0.11
mmdetection=2.3.0
cuda=10.0
cudnn=7.6

run this:

python demo/inference.py tests/images/test_2.jpg ~/dzws/mmdetection/configs/atss/atss_r50_fpn_1x_coco.py checkpoints/atss_r50_fpn_1x_coco.pth detouts/atss.wts

error:

(mmdetection) vs@vx:~/dzws/tmp-file/mmdet_trt/mmdetection-to-tensorrt$ python demo/inference.py tests/images/test_2.jpg ~/dzws/mmdetection/configs/atss/atss_r50_fpn_1x_coco.py checkpoints/atss_r50_fpn_1x_coco.pth detouts/atss.wts
[TensorRT] ERROR: Parameter check failed at: engine.cpp::setBindingDimensions::949, condition: profileMinDims.d[i] <= dimensions.d[i]
[TensorRT] ERROR: Parameter check failed at: engine.cpp::resolveSlots::1092, condition: allInputDimensionsSpecified(routine)
Traceback (most recent call last):
  File "demo/inference.py", line 62, in <module>
    main()
  File "demo/inference.py", line 31, in main
    result = inference_detector(trt_model, image_path, cfg_path, args.device)
  File "/home/visi/dzws/tmp-file/mmdet_trt/mmdetection-to-tensorrt/mmdet2trt/apis/inference.py", line 35, in inference_detector
    result = model(tensor)
  File "/home/visi/miniconda3/envs/mmdetection/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/visi/miniconda3/envs/mmdetection/lib/python3.7/site-packages/torch2trt/torch2trt.py", line 415, in forward
    shape = tuple(self.context.get_binding_shape(idx))
ValueError: __len__() should return >= 0

Input 800x1344. Any suggestions? Thanks.

[grimoire]

Hi,

I set the default input shape range as

    if opt_shape_param is None:
        img_scale = cfg.test_pipeline[1]['img_scale']
        min_scale = min(img_scale)
        max_scale = max(img_scale)
        opt_shape_param = [
            [
                [1, 3, min_scale, min_scale], 
                [1, 3, img_scale[1], img_scale[0]],
                 [1, 3, max_scale, max_scale]
            ]
        ]

img_scale comes from the cfg file of mmdet model. opt_shape_param mark the min/opt/max shape feed to the network. In most case, (include the atss config file)

opt_shape_param=[
[1, 3, 800, 800],
[1, 3, 800, 1333],
[1, 3, 1333, 1333]
]

800x1344 seems out of the limit. you can manually set the opt_shape_param to meet your demond. for example:

opt_shape_param=[
    [
        [1,3,320,320],      # min shape
        [1,3,800,1344],     # optimize shape
        [1,3,1344,1344],    # max shape
    ]
]
trt_model = mmdet2trt(cfg_path, weight_path, opt_shape_param=opt_shape_param)

[DuZzzs]

opt_shape_param = [[[1, 3, 800, 800], [1, 3, 800, 1333], [1, 3, 1333, 1333]]]

    trt_model = mmdet2trt(cfg_path, args.checkpoint, fp16_mode=args.fp16, device=args.device, opt_shape_param=opt_shape_param)

1. modify config file too, but same error.
2. config set 800x1333, do not set opt_shape_param, same error.

[grimoire]

Errr... I means the max_shape_param[2] should have a larger size, such as [1, 3, 1376, 1376], height and width must be larger than the input tensor(after preprocess, not the origin size) you feed to the network. Just try the second opt_shape_param I provided. It should works.

If you prefer to modify mmdet config(not recommend), you should set the width greater than 1344 inorder to feed 800x1344 input. Because width and height of input tensor should be mod32. for exmple. if the config is 800x1333, preprocess pipeline of mmdet will enlarge the input image to a tensor with shape 1x3x800x1344 ( ensure that width%32==0 and height%32==0).

[DuZzzs]

It works. Thank you very much! I should pay more attention to source code.
Can this wts file be used in cpp file? or it should be build and serialized first, and then deserialize in cpp file? Is this the correct way to use it?

[grimoire]

The wts file is a warp of tensorrt engine, you can get the serialized engine as follow in python:

with open(engine_path, mode='wb') as f:
    f.write(model_trt.state_dict()['engine'])

Link the amirstan_plugin lib and deserialize engine in cpp should works.
If the plugins are not loaded, use initLibAmirstanInferPlugins(); in amirstan_plugin/plugin/amirInferPlugin.h to load them manually.

The engine does not include preprocess and postprocess(divide scale factor), these part should be implemented in your cpp.

[DuZzzs]

Got it. Thanks!

[vedrusss]

Hi @DuZzzs , I've tried different values of opt_shape_param and here are results of mine:

1. I've found I must use the same values for min, optimize and max shapes to provide serialized engine be usable in C++ (when used different it had bad input dims).
2. I tried different width and height values which all were of mod32. mmdet2trt couldn't convert DCN model for [1,3,320,320] as well as for any value less than 800. I succeed converting the model only for lowest size [1,3,800,800] (that gave me the highest possible inference speed).
3. Convertion and inference (in C++) worked as well for sizes [1,3,1024,1024] and [1,3,800,1344]. But inference latency was bigger than in #2.
4. Often I had to re-run mmdet2trt to convert the model because it some runs failed (some instability?)

If this message is not the reason to re-open the issue please refer me to the right issue.