/Dreambooth-Stable-Diffusion-Tweaked

Implementation of DreamBooth with Stable Diffusion (tweaked)

Primary LanguageJupyter NotebookMIT LicenseMIT

DreamBooth on Stable Diffusion (with some customizations)

If you just installed conda (or you're running on a fresh VM):

% conda init bash

% bash

(If your shell isn't bash, substitute the correct shell command.)

Create the conda environment:

% conda env create -f environment.yaml

Activate:

% conda activate dream

Then...

  1. Move your model.ckpt into the models directory. This is likely a ckpt file from Hugging Face.
  2. Put your training data into inputs/training. Training data should be between 5 - 20 512x512 PNGs of the subject you want to train on.
  3. Put your regularization data into inputs/regularization. Regularization data is anywhere from 100 - 200 512x512 PNGs representing subjects in the same class as the subject in your training data. For instance, if you're training a model on your dog, you want divers images of dogs. (Note that these images can be generated by Stable Diffusion.)

Copy and paste the command below, but make the appropriate changes to --identifier and --class_word:

  • --identifier: The unique word that you want to use to reference your subject in your prompts. It should be something unlikely to already be in the training data.
  • --class_word: The class of what you're training on. For example: man, woman, person, dog, etc.
python main.py --base configs/stable-diffusion/v1-finetune_unfrozen.yaml -t --actual_resume models/model.ckpt -n DreamBoothFineTune --gpus 0, --data_root inputs/training --reg_data_root inputs/regularization --identifier UNIQUE_IDENTIFIER --class_word SUBJECT_CLASS

(Optional) When you're done, you can prune the checkpoints to create a smaller ckpt file (credit):

% python scripts/prune.py --input path/to/model.ckpt

(Optional) If you did the training on a remote machine, you can copy the ckpt file to your local machine with a command that looks something like this (replace the port number, IP address, and paths):

% scp -P PORT_NUM root@123.456.789.000:/root/Dreambooth-Stable-Diffusion-Tweaked/logs/training2022-10-25T21-25-03_DreamBoothFineTune/checkpoints/last.ckpt .

(Optional) Turn the ckpt file into the diffusers file format (credit) compatible with the Stable Diffusion Photoshop REST API Server:

% python .scripts/convert_original_stable_diffusion_to_diffusers.py --checkpoint_path /path/to/model.ckpt --dump_path /output/path