CompVis/attribute-control

How much GPU memory is needed for training devices?

Closed this issue · 4 comments

liujingwen-bmil commented

What an excellent work! Congratulations！I wanna create a new Attribute Delta with one 3090(24G), but it reports that my GPU memory is insufficient. I would like to ask what device you use for training and how much memory is required. Thanks for your reply.

stefan-baumann commented

Hi, thanks a lot!
I've trained the deltas on A100s (with increased microbatch size and fewer gradient accumulation steps to compensate), but it should absolutely be viable to train with 24GB VRAM. Theoretically, the training requirements should be less than those for training LoRAs, which are trainable on that VRAM budget. I don't know if I can look into it today, but I'll try to look into what needs to be changed to make training work with 24GB of VRAM in the coming days at least.

liujingwen-bmil commented

Thank you for your reply, and I look forward to your further research.

  • ❤️1
stefan-baumann commented

Sorry for taking so long to get back to you on this. I've added activation checkpointing in 302c272, which yielded a 11.5GB VRAM utilization for me, at the cost of slower training obviously. I've added instructions to the readme that explain how to enable it. Please let me know if this works for you on your system :)

liujingwen-bmil commented

Sorry for taking so long to get back to you on this. I've added activation checkpointing in 302c272, which yielded a 11.5GB VRAM utilization for me, at the cost of slower training obviously. I've added instructions to the readme that explain how to enable it. Please let me know if this works for you on your system :)

Thanks for your reply, it could be run with your new readme, thanks again.

  • 👍1