loss become NAN with Taichi 0.4.0

Question

SJHNJU opened this issue 4 years ago · 7 comments

I install taichi-nightly on OS X and try to run the demo: diffmpm.py, diffmpm3d.py, liquid.py.

However the loss become NAN on the second iteration.

WHY???

Answer 1 · 2020-01-30T16:02:13.000Z

Hi, thanks for reporting. Which taichi version, device (CPU/GPU), and OS are you using?

Answer 2 · 2020-01-30T16:07:39.000Z

Thanks! The configurations are as follows:
taichi-nightly (0.4.0)
CPU
MacOS Mojave(10.14.3)

Answer 3 · 2020-01-30T16:09:17.000Z

Thank you. I can reproduce this on my end now. I'll fix it later. Please try Taichi v0.3.25 and see if it works for you.

Answer 4 · 2020-01-30T19:20:13.000Z

Fixing related issues. Taichi v0.3.18 is the version that works.

Answer 5 · 2020-01-30T20:03:36.000Z

Confirmed that the change between v0.3.23 and v0.3.24 introduced something that caused ill behavior in diffmpm.py

Answer 6 · 2020-01-31T06:17:00.000Z

Update: this should be fixed in Taichi v0.4.1. Please update and let me know if your issue is resolved. Thanks so much for reporting!

Answer 7 · 2020-01-31T07:49:47.000Z

The issue is resolved, Great job!!!
Lots of respect, Thank you!