Incompatibility with stable-diffusion.cpp for SD3.X models
Opened this issue · 1 comments
This incompatibility goes both ways, GGUFs generated with this tool won't run in stable-diffusion.cpp, and GGUFs generated with stable-diffusion.cpp refuse to load in your comfyui node. This happens only with SD3 and SD3.5 models as far as I know, for example Flux GGUFs are cross-compatible.
This is pretty annoying for the few people like me who use both comfyui and sdcpp, especially considering SD3 quants have been working in sdcpp for moths now.
I'm guesssing this is realated to this "special fix" of the llama.cpp patch, which I don't fully grasp the point of.
loading sdcpp's quants in the node:
SD3:
SD3.5:
Loading the "fixed" quants in sdcpp just crashes the program with no special logs.
That fix is there to avoid the dim0 (which is 1) being stripped by the quantization. The comfy ones are 1:1 to the reference model (which they have to be since the comfy code fails to load them otherwise - see your first error message).
More info in this comment in the original SD3.5 thread/issue: #136 (comment) / #136 (comment)
(the key order is reversed in the GGUF files, but I've numbered the dimensions. Note the missing one in the sdcpp version.)
Since it's just one key, I think it should be fairly easy to add some logic to pad the missing dimension in comfyui + drop that dim or support the full key shape in sdcpp.
I assume your second image is for SD3.5 large (K quant?). I think that'll need extra logic due to some keys not being divisible by 256. I haven't looked at how sdcpp handles that logic yet, and we currently don't have any logic in place, so only legacy quants work in comfy for that model.