Wonderful work and hi from 🧨 diffusers
sayakpaul opened this issue · 4 comments
Hi folks!
Simply amazing work here 🔥
I am Sayak, one of the maintainers of 🧨 diffusers at HF. I see all the weights of BK-SDM are already diffusers-compatible. This is really amazing!
I wanted to know if there is any plan to also open-source the distillation pre-training code. I think that will be beneficial to the community.
Additionally, any plans on doing for SDXL as well?
Thanks for your kind words about our work😊
We plan to open-source the pre-training codes; however, as of now, we have not finalized specific dates for its release.
As for SDXL, we regret to inform you that we do not have any immediate plans to apply our method to SDXL at the moment.
Please stay tuned for further updates. We truly appreciate all the contributions to diffusers!
Hi Sayak @sayakpaul (Cc: @patrickvonplaten)
Thank you once again for showing interest in our work!
We wanted to bring to your attention that we recently came across an unofficial implementation of distill_training.py from Segmind. This implementation appears to be quite similar to ours, as it is based on the Diffusers great example code, train_text_to_image.py. It also incorporates feature hooking and defines KD losses, just like our code.
One notable difference is that they have trained their models using much larger datasets and longer training iterations, which might result in better performance compared to our models. <- (+) We will happily check their data size and model performance.
[updated, Aug/05/2023] Difference to our work, based on the blog
- Teacher: Realistic-Vision 4.0
- Training data: LAION Art Aesthetic dataset with image scores above 7.5, because of their high quality image descriptions.
- Student type: two models on 1M images for 100K steps for the Small and 125K steps for the Tiny mode respectively.
The good news is that their models are also compatible with Diffusers.
As for the delay in immediate releasing our own training code, the main reason is the lack of resources 😔 for code clean-up. While we haven't thoroughly tested their repository for runnability, it seems workable, and we believe their code could be a valuable complement to ours.
You may already be aware of this, but if not, I hope this information can be helpful in any way🤗
Yes, actually they also wrote a blog post about it: https://huggingface.co/blog/sd_distillation.
Would be great if you could mention their work from this repository :)
Oh, I didn't know about that. That's really cool! Thank you very much for letting me know.
Today, I will add a mention of their repository and the related post in our Readme :)