How to add VAE to training

Question

How to add VAE to training

Eliont opened this issue a year ago · 6 comments

Eliont commented a year ago

Hello and good time of day.

How to add custom VAE to training when training with custom model?

Thanks for you work, best lora training colab i've ever seen.

cavit99 commented a year ago

Thank you

Answer 1 · 2023-05-06T08:42:22.000Z

I second this, please add, excellent work

Answer 2 · 2023-05-07T06:47:09.000Z

Oh I'm sorry that I missed this - I haven't added it before as my sources say using an external VAE when training leads to worse results. If you really want to, I can look into it.

Answer 3 · 2023-05-08T13:34:21.000Z

Interesting, do you have any references to that claim re VAEs? Would be great to know. Also Kohya's modules have regularisation images section and I see advice regarding using this. In the colab there is no option to add regularisation images, just the training images. Any intention of implementing this? Referenced here: https://youtu.be/TpuDOsuKIBo?t=861

Answer 4 · 2023-05-08T17:16:46.000Z

You can work with multiple folders in the Extras below the main cell of the trainer. You can make one of them a regularisation folder with is_reg=true

Answer 5 · 2023-06-27T02:39:45.000Z

Closed as not planned for now