😱 GPU memory leak for 2D compressible models
ali-ramadhan opened this issue · 1 comments
ali-ramadhan commented
Kinda scary, not even sure where this could be coming from.
192^3
Memory at iteration 0: 1685 MiB
Memory at iteration 1: 1695 MiB
Memory at iteration 10: 1695 MiB
400x1x400
Memory at iteration 0: 519 MiB
Memory at iteration 1: 779 MiB
Memory at iteration 2: 1057 MiB
Memory at iteration 3: 1585 MiB
Memory at iteration 4: 2113 MiB
Memory at iteration 5: 2641 MiB
...
400x2x400
Memory at iteration 0: 519 MiB
Memory at iteration 1: 523 MiB
Memory at iteration 2: 525 MiB
Memory at iteration 3: 527 MiB
Memory at iteration 10: 527 MiB
ali-ramadhan commented
Seen on both Tartarus and Satori. Although perhaps intermittent as 400x2x400
leaked memory on Tartarus but not on Satori...