Out-Of-Memory regardless of setting batch_size
haithanhp opened this issue · 10 comments
I run your code in main.py of TransMatcher but the Out-Of-Memory issue is still there regardless of setting batch_size. Do you know how to modify it to make it efficient? The code line of this issue:
score = einsum('q t d, k s d -> q k s t', query, key) * self.score_embed.sigmoid()
What is the GPU card you used? I was using V100 with 32GB memory without issues, but I did not check those with lower memory. You can try lower the batch size, --num_trans_layers, --dim_feedforward, and --neck.
I used 8 GPUs of V-100 Tesla 16GB but when I set to batch_size of 8, it still has an out-of-memory error.
I used only one V100, with a batch size of 64. It seems that you should be able to run it with batch size of 8 in one of your GPU.
How many GB memory for your V100? Is it 32 GB? I just have 16GB and it says that 9.0 GB is required.
Yes, as I said, 32GB.
I see. So, we are unable to run the code with lower memory even multiple GPUs and small batch size. But it is weird that the efficient model can not be trained with batch size of 1. Could you show me where to update the code to make it run with lower memory? Thanks
I'm not sure. I still feel strange that you cannot run the code with batch size of 8 in one V100 (16GB), since I'm able to run with batch size of 64 in one V100 (32GB). Did you try one GPU?
For further suggestions, please see the below.
What is the GPU card you used? I was using V100 with 32GB memory without issues, but I did not check those with lower memory. You can try lower the batch size, --num_trans_layers, --dim_feedforward, and --neck.
I have encountered this issue and it seems that you need to change the test_batch smaller. Because the default size of the test batch is 256.
Thanks. Yes, could be this reason. I thought @haithanhp encountered this during training.
Thanks for your help. I used the server with larger GPU memory. So, the issue is no longer there.