A set of nodes for editing images using Flux in ComfyUI
See example_workflows directory for examples.
No ControlNets are used in any of the following examples.
Rectified Flow Inversion (Unsampling from RF Inversion)
Admittedly this has some small differences between the example images in the paper, but it's very close. Will be updating as I find the issue. It's currently my recommended way to unsample an image for editing or style transfer.
Use this workflow for RF-Inversion.
Stylization now works!
It can also be used to mix or style images (although I'm still working out the settings for this)
- Ensure "reverse_ode" is set to True on the "Outverse Flux Model Pred" node. Sometimes when users upgrade this repo it doesn't load the workflow correctly.
- latent_image -- the image to guide the sampling
- start_step -- the step that the sampler starts guiding the sampling towards the image in "latent_image"
- end_step -- the last step for guiding the sampling (not inclusive)
- eta -- the strength of the guidance. The paper does not decrease this below 0.7
- eta_trend -- how the eta should increase/decrease/stay constant between start_step and end_step
- gamma -- the paper leaves this at 0.5
- For sampling normal flux guidance works (~3.5)
- For unsampling use 0
- Overlayed images -- try changing your start step and/or Eta. A start step that is too late won't be able to influence the image generation properly
- Not following edits -- try fewer steps (change start/end step) or lower eta
- Make sure your steps on the Forward (unsampling) and Reverse (sampling) samplers are the same (recommended 28 each)
See example workflow for how to use this one. It's similar to inverse noise/unsampling, but has better adherence to the input image.
@article{rout2024rfinversion,
title={Semantic Image Inversion and Editing using Rectified Stochastic Differential Equations},
author={Litu Rout and Yujia Chen and Nataniel Ruiz and Constantine Caramanis and Sanjay Shakkottai and Wen-Sheng Chu},
journal={arXiv preprint arXiv:2410.10792},
year={2024}
}