Regarding the VRAM usage of each GPU
Closed this issue · 1 comments
zhaosonghui commented
This is a very valuable study! During the training process, how much VRAM does each A800 GPU approximately use? I would like to know if it is feasible to replicate the experiment using a 4090 (24GB) version of the GPU
Hon-Wong commented
We use A100-80G for training. You may need some tricks to run the experiment on 4090, e.g., lora, 8-bit, etc.