Changing the stable diffusion to AITemplate Sd
DhruvThu opened this issue · 3 comments
Hi, I am new to a Pytorch lightning and started with the stable-diffusion-deploy. I have cloned it and try to run it and its work perfectly. But when I change the stable diffusion model to other implementation of SD such as AITemplate pipeline (not written in pytorch), it started giving error like this,
Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method
I am not sure but I guess when I run stable diffusion deploy, it will call AItemplate pipeline and create one separate process in gpu and after that when load balancer try to run this process available in gpu, it cause an error.
If someone can help to resolve this issue, it will be really helpful.
hi @DhruvThu, it is a known issue with PyTorch. You might have to set the multiprocessing method to spawn. Also, this shouldn't happen if you initialize all the cuda related operation inside the run method of the LightningWork.
https://pytorch.org/docs/stable/notes/multiprocessing.html#cuda-in-multiprocessing
Thank you so much for your suggestion. I have tried to run the SD pipeline inside run method and Its working for me.
really glad to be of help @DhruvThu! Would love to see what you build 💜