M-Control: Improving Text-Image Consistency via Mask-Guided ControlNet (paper will be released later)
To achieve M-Control, please follow two steps:
Our codebase is built on huggingface/diffusers and has shared environmental dependencies.
conda env create -f environment.yaml
conda activate m-control
pip install .
If you already have a condition input and corresponding mask, first set the hyper-parameters for the generation in a yaml config file. An example of extraction configs can be found in configs/test.yaml. Once the arguments are set, run:
python inference.py --config <extraction_config_path>