Cuda Out of Memory
hamidrezafazlali opened this issue · 2 comments
Hello,
I am have been using the SECOND method based on the sparse 3D CNN, with around 16 Million parameters in my whole model, I do not get the "Cuda Out of Memory". However, when I replace the sparse 3D CNN in the backbone with your VoTr, although my model number of parameters is around 10 Million, I get the "Cuda Out of Memory" error.
I've also tried to make VoTr even simpler than what it is, but it still gives the "Cuda Out of Memory" error.
I really appreciate it if you help me.
@hamidrezafazlali
This is mainly because we are using 48 attending voxels for each self-attention, while sparse convs use less than 27 voxels. So the GPU memory consumption is larger than the convolutional counterparts.
A solution is to decrease the number of attending voxels (SIZE parameter in the yaml config file), but it will harm the detection performance.
Thank you so much, for the quick response:)