/debiasing_gen_models

This repo is for codebase and experiments for debiasing project on diffusion models.

Primary LanguageJupyter NotebookOtherNOASSERTION

'Balancing Act: Distribution-Guided Debiasing in Diffusion Models', CVPR 2024

arXiv Webpage slides

Set Up

Create the environment by running the following:

- conda env create -f environment.yml
- pip install https://github.com/podgorskiy/dnnlib/releases/download/0.0.1/dnnlib-0.0.1-py3-none-any.whl

Download the pretrained models from here and store it in pretrained/

You have to match data.dataset in custom.yml with your data domain. For example, if you want to use CelebaHQ, data.dataset should be CelebA_HQ.

Using Pretrained Models to generate debiased images

We provide pretrained H classifiers for Gender, Race and age for Celeba-HQ images

Run the ./s_u_r.sh file to generate images:

  • exp: Path that the iamges should be stored in.
  • edit_attr: Attribute to edit. But not used for now. you can use ./utils/text_dic.py to predefined source-target text pairs or define new pair.
  • n_test_img : How many images should be generated?
  • attribute_list : Attribute to be balanced: [1,0,0,0 - Eyeglasses, 0,1,0,0 - Gender, 0,0,1,0 - Race] [For multi attributes, add 1's accordingly, ex: 1,1,0,0 = Eyeglasses+ Gender]
  • scale : Guidance scale [Hyperparameter] Refer to __ section for scale of the attributes in the paper, if not present, needs to be tuned
  • just_precompute : False

Vanilla Generation:

  • vanilla_generation : True

Sample based Generation:

  • vanilla_generation : False
  • sample : True
  • bs_test : 1
  • male : 0/1 [Binary, for each class of the attribute. Example: for gender, 0->Female, 1->Male]

Distribution based Generation:

  • vanilla_generation : False
  • sample : False
  • bs_test : 100 (if any other, scale needs to be tuned accordingly) (higher the batch size, better is the guidance)
  • male : Fraction of each class. example: for gender, 0.5 = 50% Male, 50% Female generations

Training h-classifiers for other attributes/ other datasets

  • A directory of images of all the classes of the particular attribute needs to be made. Example: If attribute is Gender, images of males and females need to be kept in 2 directories (Around 1000-2000 images for each class)
  • Generated their h vectors : Run the script_extract_he.sh file to extract the h vectors of images.
- `just_precompute`: True
- `test_path_one` : Path containing the images whose h-vectors should be generated.
- `savepath` : Path where the h vectors need to be saved.
  • Train a Linear classifier on these images. Run __________ file to do this
  • Use the checkpoints of the classifier for generating images. Copy the checkpoint path into h_classification.py (for sample based) and multi_classifier.py for distribution based, and use the respective index in the attribute_list parameter while generating

Acknowledge

Codes are based on Asyrp

Bibtex

@InProceedings{Parihar_2024_CVPR,
    author    = {Parihar, Rishubh and Bhat, Abhijnya and Basu, Abhipsa and Mallick, Saswat and Kundu, Jogendra Nath and Babu, R. Venkatesh},
    title     = {Balancing Act: Distribution-Guided Debiasing in Diffusion Models},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2024},
    pages     = {6668-6678}
}