This repository is created to fine-tune your VAE model of Stable Diffusion model, which you can change input image size, or with a new dataset.
Note: I follow the guidance here, in which some first epochs are trained with (l1 + Lpips), later epochs are trained with (l2 + 0.1*Lpips) loss.
Please download pre-trained SD model here, and put into sd_model folder.
Copy ldm folder from this repo into your current directory.
For training, run the following script:
python train.py \
--data_dir <YOUR DATA FOLDER>
--batch_size 2 \
--num_epochs 20 \
--lr 2e-5 \
--val_size 0.1 \
--precision 16 \
--image_size <YOUR DESIRED SIZE> \
--lpips_loss_weight 1.0 \
--ema_decay 0.99\