Batch_size > 1 seems to break controlnet models

[herval]

I'm getting the weirdest bug with ControlNet 1.1 (used to work on an earlier release, broken for a week or so). This is through the sdapi/v1/txt2img endpoint.

I'm trying to use 2 controlnet steps (webui is configured for 2). Seed is fixed at 12345.

I'm using the following configuration (excerpt from the logs):

Loading preprocessor: scribble_xdog
Pixel Perfect Mode Enabled.
resize_mode = ResizeMode.INNER_FIT
raw_H = 512
raw_W = 512
target_H = 512
target_W = 512
estimation = 512.0
preprocessor resolution = 512
Loading model from cache: t2iadapter_style_sd14v1 [202e85cc]
Loading preprocessor: clip_vision
Pixel Perfect Mode Enabled.
resize_mode = ResizeMode.INNER_FIT
raw_H = 512
raw_W = 512
target_H = 512
target_W = 512
estimation = 512.0
preprocessor resolution = 512
Data shape for DDIM sampling is (1, 4, 64, 64), eta 0.0

Things work okay-ish when I set the batch_size to 1. Here's an example:

If I keep the exact same parameters but change the batch_size to 4, images come out more and more distorted on each iteration. Here's the outputs in that case:

It's as if the batch_size is somehow re-feeding the outputs on the batch?

[herval]

Actually, I noticed an even weirder issue with the shuffle model:

if I call the API with only the shuffle model (in my case, passing the second controlnet as disabled), the generation only works if the batch_size is 1:

Loading model from cache: control_v11e_sd15_shuffle_fp16
Loading preprocessor: shuffle
Pixel Perfect Mode Enabled.
resize_mode = ResizeMode.INNER_FIT
raw_H = 512
raw_W = 512
target_H = 512
target_W = 512
estimation = 512.0
preprocessor resolution = 512
Data shape for DDIM sampling is (1, 4, 64, 64), eta 0.0

Setting batch_size to anything else gets me a completely random image.

So I suspect the issue isn't with an interaction between cn + t2i models (my original suspicion), but rather something in the way the batch_size is being handled?

[herval]

For reference, the same issue happens through the webui:

vs batch size of 1:

[lllyasviel]

please attach the version and your a1111 cmd flag

[herval]

please attach the version and your a1111 cmd flag

Tested on v1.1.137 and v1.1.150 (latest?)

webui args:

webui.sh --disable-safe-unpickle --listen --api --api-log --disable-console-progressbars --skip-torch-cuda-test --reinstall-torch

(ran with torch2+c118 and torch2 cpu, also with and without --opt-sdp-attention)

[lllyasviel]

i see that you have other tiling extensions in your screenshots. can you try disable other extensions?

[lllyasviel]

btw i cannot reproduce your problem

[herval]

Disabling all other extensions seems to fix it 😮 😢

[lllyasviel]

then just manage them on you own

[herval]

thanks for debugging