deeplearning4j/deeplearning4j

AdamUpdater producing NaN and Infinity results if FP16 is used

partarstu opened this issue a year ago · 0 comments

partarstu commented a year ago

Issue Description

expected behavior : idempotent results with both DataType.FLOAT and DataType.FLOAT16 as default data types except rounding cases in order to avoid over/underflow
encountered behavior: NaN and Infinity results while using DataType.FLOAT16

Version Information

Deeplearning4j version - SNAPSHOT
Platform information: Windows 11
CUDA version - N/A
NVIDIA driver version - N/A

Additional Information

The reproducer has been created based on UpdaterTest and could be found here: https://gist.github.com/partarstu/d1ad1c5060ddaa19186d405b2379917e