inference.py
Opened this issue · 6 comments
Do you happen to have another way to run inference.py? Main.py is too much for my computer to handle so I ran and created a training directory using train.py, however, when I try to run inference.py, it simply won't recognize the arguments. I've tried anywhere from inference.py --captions.txt --training_20221106_101837 to inference.py [-c] [-d]. Nothing I've ran seems to work, I've had to add torch imports in order to get it to work on its own but otherwise I'm at a loss. Any help would be greatly appreciated.
Hi @Shikamaru5 ,
I hope you found the solution of your problem. If not, I faced similar problem and I solve it by using this command
python inference.py -c captions.txt --TRAINING_DIRECTORY training_20221207_125134
I hope it works for you.
hey @DataDetactive thank you for this, I may have to look into it, I've gone onto working with the actual Imagen but I really do appreciate it.
hey @Shikamaru5 , Great that you started on the original Imagen. I have question for you since you started the original imagen. Do you know how I make the model work on 2 GPUs because one GPU is not sufficient. since I saw the guidelines which the authors added I did not know how I use it.
hey @DataDetactive, something I've been looking into is called DeepSpeed it's a program that allows you to pass data from the gpu to the cpu back to the gpu, or something along those lines. Basically it can allow a single GPU to train LLM up to the billions of parameters. Unfortunately as I use Windows it is not supported even though it's developed by Microsoft, so I've been researching how to use it with Docker. I believe that I have figured out how to do this but it did take significant digging. As for your question about multi-gpus, the documentation for Imagen or the proxy I found says that for training on multi-gpus it can be done with a program called Accelerate. It has me curious now because it says that it looks like this program has a number of useful functions including using multi-gpus in a way that you could have functions wait to be processed before continuing with further instructions and it can even utilize DeepSpeed. You can also use it to deal with the out-of-memory error to some degree by using a function called find_executable_batch_size. I hope this helps at all, this has been a challenge to deal with since I just have one gpu but hopefully through experimentation and a little duct tape someone may be able to solve these limitations.
@Shikamaru5 , Great thank you for the information. I will try my best with accelerate to know how I could make it works. As soon as I know I will tell you. Thanks again for your help. Really appreciate it.