Investigate CUDA memory leak
Closed this issue · 2 comments
LocalToasty commented
When training for many targets, the pipeline tends to crash after a while due to insufficient GPU memory.
jnkather commented
oops that's weird. I am interested to hear what the problem is. Is it possible to reset the CUDA device before each run?
jnkather commented
@narminGhaffari did you experience this in your implementation as well?