fp16精度在/norm/ReduceMean_output_0节点返回nan
13484835805 opened this issue · 1 comments
13484835805 commented
我在fp16精度,onnx==1.16,onnxconverter-common==1.14.0,gpu==A800-SXM4-80GB,节点readcemean输出结果为nan;
13484835805 commented
/layers.5/patch_embed/Transpose_output_0 -8680.0 24160.0 -5012.0 5644.0 6320.0 31400.0 18580.0 1565.0 22260.0 -2586.0
/layers.5/Add_output_0 -8400.0 24850.0 -6420.0 4896.0 5572.0 34720.0 18700.0 2198.0 22690.0 -2404.0
/norm/ReduceMean_output_0 nan nan nan nan nan nan nan nan nan nan
git:
https://github.com/XPixelGroup/HAT