Flux AI Toolkit preset generate completly noisy image in Comfy UI.
axel578 opened this issue · 3 comments
It might not necessarely be a Comfy UI issue since all flux lora training I did on kohya/onetrainer seems to work, and the issue seems to be pretty recent seens it was working with simpletuner before, I used two days old simpletuner version, yet the new version did'nt seem to fix it.
I used your default flux lora training especially with quanto (let me know if its the issue) and use flux target ai-toolkit
Just so you can test it a lora: https://drive.google.com/file/d/1mpFHMTOfbAXVLKKj8A5IfWLr0VIoT32K/view?usp=sharing
ai-toolkit target is not really supported. it is probably safe to remove now, in fact. i would recommend just all or all+ffs for LoRA.
but really LoRA is kinda outdated tech by now. LyCORIS LoKr works much better.
ai-toolkit target is not really supported. it is probably safe to remove now, in fact. i would recommend just
allorall+ffsfor LoRA.but really LoRA is kinda outdated tech by now. LyCORIS LoKr works much better.
Yeah, I know the popularity of LyCORIS LoKr and why its know, its very good on paper, but there are many properties (especially on multi resolution) where it doesnt work that well, yet its still pretty good.
Don't remove ai-toolkit, really the more layers the lora is applied on, the easier it is to train a lora on very complex and far away from original learned distribution images, people are judging lora quality only on very easy image dataset (someone's face ...), but they really forget about the capacity of the lora for learning hard styles/image compositons, to be fair, I modified adapter.py to target near every layers of the model and its where I get the best result really.
LoKr has nothing to do with multi-resolution, that's the underlying model at play. unless @KohakuBlueleaf can explain if I am overlooking something.