Anyway to cache the loading of stable diffusion between runs?
hoalarious opened this issue · 4 comments
Hi @neggles,
Glad to see active development into AnimateDiff. You've got some cool ideas going forward. I'm also working on an AnimateDiff repo focusing on the UI side. I think it would be good if I can just call your CLI for the UI to consolidate some development efforts however I'm wondering if there's a way to cache the loading of stable diffusion as that would save about a min between each generation.
Valid point, I have been meaning to refactor things to allow that... Let me see what I can do.
If I pull model loading and the actual inference loop out into separate functions (which i've been meaning to do for a while) you'd be able to call those independently; would that work?
I'm also currently recreating the entire motion-model'd unet on every startup which is a bit silly. Let me take a look at some bits and pieces...
OK! You should now be able to import generate() from animatediff.cli and call it multiple times without it having to reload the model every single time. Bonus: when called as a function, it will return the path to the generation output folder 😄
I sort of cheated and just used a couple of global variables (ew) to make it happen, but, well, it works! Example script at scripts/test_persistent.py shows how it works.
That's perfect! Much appreciated. I'll give it a whirl over the weekend!
It's kind of hard to not do the quick and dirty in a field that is moving so fast hahah. It makes premature optimization a bit more relevant until we know this needs LTS. Though it is good to train ourselves to do it the right way from the start. Ughh the dilemmas of being a programmer.
Converting the pipeline variable to be a part of a class could be a solution. I don't think that'll be difficult to do even later down the track so it's probably not a priority.
I've got an implementation for multiple loras and init image on my fork that I should be able to migrate over to your CLI if you don't already have plans for that. The original code doesn't deal with backing up the unet parameters that were modified by the LoRAs so the unet needs to loaded again each time or the model decays after each run.
True that. A class would probably be the way to go, yeah, but there's not a huge difference between that and just using the global variables (which don't persist in CLI mode anyway) so, ehhhh. They're still module-scoped, so it's fiiiiiiiiiiine 😅
Would love to have some functioning LoRA + init image code, that's just about the only major couple of pieces still missing - would be happy to take that PR (though I reserve the right to refactor things since i'm a little picky about code style 😝)