Perp-Neg/Perp-Neg-stablediffusion

Perp-Neg for Editing Models

andreemic opened this issue · 1 comments

Hey!

I really loved your work on this and think this can be very powerful in the context of image editing with diffusion models, like done by InstructPix2Pix.
There, the authors propose to have two-way CFG between

  1. unconditional generation,
  2. only image-conditional generation, and
  3. image- and text-conditional generation

So you end up with something like this:
Screenshot 2023-06-22 at 00 25 27

How could we apply the Perp-Neg idea there? How would the idea map to this three-way CFG?