rotate360 + headbanging

[j0seo]

Congrats on this amazing work and thank you for the quick release of the code!

I am running the code with the rotate360 + headbanging setting using images I prepared. But, sometimes I encounter a phenomenon like the attached photo and video. Are there any tips or appropriate hyperparameter values to avoid this?

[ironjr]

You may try the following:

1. Manipulate negative prompts to set harder constraints for the frame object. You may try adding tags like twitter thumbnail, profile image, instagram image, watermark, text to the negative prompt. In fact, negative prompts are the best thing to try if you want some things not to be appeared in the resulting image.
2. Actually, our SD1.5 model uses the standard stable-diffusion-inpainting-v1.5 model, which I think has vulnerability to generate framed photos. To avoid this, I'd rather try other custom checkpoint models, which employs different pipeline methods: LaMa inpainting -> ControlNet-inpaint guided image inpainting. I empirically find that this setting is more stable than the naive application of standard inpainting model.

I hope you find this guide useful.

[j0seo]

Thanks a lot! I've changed the prompt a little bit and now it works well. In my case, it seems that there was a bias regarding proper nouns like 'Shibuya'.

[ironjr]

Very interesting! Thanks for the report!