shonenkov-AI/rudalle-aspect-ratio

How to change window and image_size when using dwt vae (RUDALLE's output will become 512)?

Opened this issue · 0 comments

Hi, I experimented with the codes, and both Malevich and Surreal work fairly well. When I enabled dwt=True for the get_vae function, however, the horizontal ratio yields extremely bad results. To be consistent with the vae upscale, I scaled the window and image_size to 256 and 512 respectively. In contrast to the vertical ratio, the horizontal ratio yields very poor results. Do you know how I should modify the parameters or codes?

Here is the codes for generate vertical ratio.

rudalle_ar = RuDalleAspectRatio(
    dalle=dalle_m, vae=vae, tokenizer=tokenizer,
    aspect_ratio=9/32,window=256,image_size=512, bs=2,device=device
)
_, result_pil_images = rudalle_ar.generate_images('облако', 768, 0.99, 2)
show(result_pil_images, 2)

The results are here and it looks amazing.

And here is the codes for generate horizontal ratio.

rudalle_ar = RuDalleAspectRatio(
    dalle=dalle_m, vae=vae, tokenizer=tokenizer,
    aspect_ratio=32/9,window=256,image_size=512, bs=2,device=device
)
_, result_pil_images = rudalle_ar.generate_images('облако', 768, 0.99, 2)
show(result_pil_images, 1)

Now the result here looks very bad, the transition between windows looks very awkward.