ali-ramadhan/Atmosfoolery.jl

😱 GPU memory leak for 2D compressible models

ali-ramadhan opened this issue · 1 comments

Kinda scary, not even sure where this could be coming from.

192^3
Memory at iteration  0: 1685 MiB
Memory at iteration  1: 1695 MiB
Memory at iteration 10: 1695 MiB
400x1x400
Memory at iteration  0: 519 MiB
Memory at iteration  1: 779 MiB
Memory at iteration  2: 1057 MiB
Memory at iteration  3: 1585 MiB
Memory at iteration  4: 2113 MiB
Memory at iteration  5: 2641 MiB
...
400x2x400
Memory at iteration  0: 519 MiB
Memory at iteration  1: 523 MiB
Memory at iteration  2: 525 MiB
Memory at iteration  3: 527 MiB
Memory at iteration 10: 527 MiB

Seen on both Tartarus and Satori. Although perhaps intermittent as 400x2x400 leaked memory on Tartarus but not on Satori...