ExponentialML/ComfyUI_VisualStylePrompting

Can not get outcome as shown in the examples with the provided workflow

RudyB24 opened this issue · 6 comments

Hi. I am looking forward to using VSP, the examples shown in the papers look awesome, and look indeed better than the 'competition'. However ... I can not get the expected outcome. I installed as described, via git clone, then opened the provided workflow, in which I chose model realisticvison4, loaded an image, wrote 'purple fur' in the style prompt and 'dog' in the pos prompt, expecting to get a purple fur dog. Please see the image I attached. Can you point me in the proper direction to get it working?

Greetings, Ruud.

VSP problem

Hey @RudyB24! Please refer to the discussion here, thanks!

Thanks for your effort and the work and time you put in.

Unfortunately here with me the results have not changed after the update. I again tried the purple fur dog. Looking at the examples shown in the paper, like with the fire or the white clouds, I would expect to get a purple fur dog. The dog that I do get has only a hint of purple fur and is deformed, with two noses. See the attached image. As a comparison, there's also an IPAdapter output (at 1024 px)

Then there's this other thing. You combined the pos- and the style prompt into one new node. That is a pity. Before I was able to implement the VSP into my existing workflows, that may contain a Prompt Styler and may use Integrated Nodes. With this new combined node I can not simply add VSP, I have to do a lot more rework of the workflows.

I hope you'll return to the separate pos, neg, style prompts again in a next update.

Best regards, Rudy.

purple_fur_dog
purple_fur_dog_IPAdapter

Another PR has been pushed that updates the functionality. Could you check to see if it works for you?

First, thanks for having separate prompts for style/reference and positive. It's also useful to have the style prompt separate when using an image analyzer > prompt generator like WD14 Tagger for the style/reference image.

The output images are getting better, but we're still nowhere near the images that are shown in the paper https://curryjung.github.io/VisualStylePrompt/

Images: VSP workflow used, 8 output images of purple fur dog, Japanese Geisha with VSP, Japanese Geisha with IPAdapter.

Best regards, Rudy (from https://www.youtube.com/playlist?list=PLyC6aoYnRBZbU7RDv3bvnXjDHTn1zoyuY)

Version20240323
Purple_Fur_Dogs
Geisha_VSP
Geisha_IPA

After today's update, can not get "a dog made out of cloud"
workflow (5)

Please give more workflows with the images shown in the paper including thonse with controlnets.