p1atdev/LECO

Penultimate CLIP layer

catboxanon opened this issue · 2 comments

catboxanon commented

AFAIK, Waifu Diffusion is trained using the penultimate CLIP layer, just as NovelAI did. Currently there isn't a config option for that. Shouldn't there be one? I would imagine that would improve the results for the WD1.5 examples.

p1atdev commented

Hi,

SDv2.x models always use the penultimate layer, so it is no problem for SDv2.x like WD1.5. (see huggingface/diffusers#1721 (comment))

But some v1.x models prefer CLIP skip 2, so I'll add an option for it.

Thanks for pointing out.

  • 👍1
catboxanon commented

Looks like this was implemented as part of #13 so I'm gonna close this.