xuelunshen/gim

RuntimeError: CUDA out of memory during 3D reconstruction.

sean880304 opened this issue · 4 comments

Hi, first of all thank you for the amazing work. I've tried the 3d reconstruction but encountered "CUDA out of memory" problem. I get this error while it's about to perform dense matching.

RuntimeError: CUDA out of memory. Tried to allocate 5.17 GiB (GPU 0; 9.76 GiB total capacity; 2.49 GiB already allocated; 4.22 GiB free; 3.50 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.

I though it could be batch size is too large, later i found out that batch_size is already set to 1 in match_dense.py. I've also tried playing around parameters like max_kps, dfactor, cell_size, resize_max, etc. but none worked. Perhaps you could shed some light on how to reduce the cuda memory needed? Much appreciated!

Could you tell me which line of code raise this error ?

i managed to reduce to memory needed by setting upsample_preds=False and changing height=480, width=640 in hloc/matchers/dkm.py. but the error remains. Full traceback is as following:

$ sh reconstruction.sh test
Screenshot from 2024-09-20 16-04-25

Thank you for your time.

sparse_matches, mconf = self.net.sample(dense_matches, dense_certainty, 8192)

Try to change 8192 to 4096, this will reduce the number of matches participating in reconstruction and may lead to worse reconstruction results

sparse_matches, mconf = self.net.sample(dense_matches, dense_certainty, 8192)

Try to change 8192 to 4096, this will reduce the number of matches participating in reconstruction and may lead to worse reconstruction results

It works! I really appreciate it!!