This is the official repository for the following paper:
Deep Image Harmonization with Learnable Augmentation [arXiv]
Li Niu, Junyan Cao, Wenyan Cong, Liqing Zhang
Accepted by ICCV 2023.
SycoNet can generate multiple plausible synthetic composite images based on a real image and a foreground mask, which is useful to construct pairs of synthetic composite images and real images for harmonization. We release the SycoNet inference code.
Clone the repository:
git clone git@github.com:bcmi/SycoNet-Adaptive-Image-Harmonization.git
Install Anaconda and create a virtual environment:
conda create -n syconet python=3.6
conda activate syconet
Install PyTorch:
conda install pytorch==1.10.1 torchvision==0.11.2 torchaudio==0.10.1 cudatoolkit=11.3 -c pytorch -c conda-forge
Install necessary packages:
pip install -r requirements.txt
Build Trilinear:
cd trilinear_cpp
sh setup.sh
Modify CUDA_HOME as your own path in setup.sh. You can refer to this repository for more solutions.
Download SycoNet model pretrained_net_Er.pth and 3D LUTs pretrained_net_LUTs.pth pretrained on the whole iHarmony4 training set from Baidu Cloud (access code:o4rt) or GoogleDrive. Put them in the folder checkpoints\syco.
Modify real and mask in demo_test.sh as your own real image path and foreground mask path respectively. Modify augment_num as your expected number of generated composite images per pair of real image and foreground mask. Then, run the following command:
sh demo_test_single.sh
Our SycoNet could generate composite images for the input real image and foreground mask in the folder results\syco\test_pretrained.
Download iHarmony4 and modify dataset_root, dataset_name in demo_test_iHarmony4.sh as your own dataset path. Then, run the following command:
sh demo_test_iHarmony4.sh
Our SycoNet could generate composite images for the input real images and foreground masks in the specified dataset in the folder results\syco\test_pretrained.