Paddle训练UNet转ONNX后在Windows上推理速度变慢

[zhiminwang1]

我训练完2分类的UNet模型后转为onnx格式，推理速度变慢了很多

导出为onnx

def paddle_onnx(params_path, onnx_path):
model = UNet(2, 3)
model.set_state_dict(paddle.load(params_path))
model.eval()
input_spec = paddle.static.InputSpec(shape=[None, 3, 512, 512], dtype='float32', name='image')
paddle.onnx.export(model, onnx_path, input_spec=[input_spec], opset_version=12)

测试

net = UNet(2, 3)
net.set_state_dict(paddle.load("F:\AI\onnx_test\pd_unet.pdparams"))
net.eval()

model = InferenceSession('F:\AI\onnx_test\pd_unet.onnx')
x = np.random.random((1, 3, 512, 512)).astype('float32')
d0 = model.run(output_names=None, input_feed={'image': x})

start = time.time()
d0 = model.run(output_names=None, input_feed={'image': x})
end = time.time()
print('onnx predict time: %.04f s' % (end - start))

start = time.time()
paddle_outs = net(paddle.to_tensor(x))
end = time.time()
print('paddle predict time: %.04f s' % (end - start))

测试结果

onnx predict time: 13.4033 s
W1207 17:16:43.671676 17220 gpu_resources.cc:201] WARNING: device: . The installed Paddle is compiled with CUDNN 8.2, but CUDNN version in your machine is 8.0, which may cause serious incompatible bug. Please recompile or reinstall Paddle with compatible CUDNN version.
paddle predict time: 5.4835 s