NaN output when running manta

[joepareti54]

when executing following:

./manta ../scenes/_trainingData.py --dim 3 --addModelGeometry True --addSphereGeometry True

the ouptut has entries such as:

FluidSolver::solvePressure skipping CorrectVelocity since res is nan!

According to #8 a possible workaround is turn down the gradient clipping magnitude first and see if that works. By default it's 1, but I would try as low as 0.2.

Any idea how to do that? Or any suggestions how to resolve this NaN problem?

[RiLights]

Hi Joepareti,

What if you go further and try to train your fluid_net ? And for example with '-batchSize' equal to 10 or even less.

Cheers
Ostap

[jonathantompson]

Likely this issue is so old that no one cares anymore :-)

However, just to be clear, this is from running manta and doesn't have anything to do with the torch training code. IIRC, Manta is unstable for some initial conditions (a know bug) and NaN is due to residual blow up when the PGC loop fails. You can try playing with manta max iterations.

The current manta settings are a good trade off of not too many of these events (they're just skipped when we find them), vs simulation speed.