v0xie/sd-webui-incantations

Question on generation speed

tusharbhutt opened this issue · 3 comments

tusharbhutt commented

Hi,

not a bug, but a question: is the generation speed supposed to be halved when using this? I went from 2:15 for four 1024x1024 SDXL images at 40 steps (in one batch) to 4:29. Is this normal?

v0xie commented

That's normal for the current implementation.

Normal SDXL batched forward pass computes cond and uncond in one call to the inner model. PAG requires a third cond to be computed, so we're doing so in a second call to the inner model.

Ideally we would batch cond, uncond, and PAG cond together so we get it all done in one call to the inner model.

  • 👍2
tusharbhutt commented

Oh cool, hopefully you can find a means of addressing it :)

tusharbhutt commented

I'm an idiot, I shouldn't have closed this. The speed issue remains in that latest version.