CUDA out of memory in evaluation step
Closed this issue · 1 comments
alasokolova commented
Hi, I am trying to run the training code, training process goes well, but it fails as soon as starts evaluating. I decreased the batch size (although I as far as I see the batch size during evaluation is 1).
Have you faces such a problem? Why could that happen?
dhbloo commented
Hi, can you elaborate on your GPU's memory size? We run the evaluation on an RTX3090 GPU, so i suppose 24GiB memory is sufficient.