/Face-Sketch-SCG

Semi-supervised Cycle-GAN for face photo-sketch translation in the wild, CVIU2023

Primary LanguagePythonOtherNOASSERTION

Face-Sketch-SCG

This is the official PyTorch codes for the journal extension of our previous work:

Semi-supervised Cycle-GAN for face photo-sketch translation in the wild.
Chaofeng Chen, Wei Liu, Xiao Tan, Kwan-Yee K. Wong

arXiv visitors Citation LICENSE

Framework Noise-injection

⚙️ Dependencies Configuration

  • PyTorch >= 1.8.0
  • CUDA >= 10.1
  • Install other dependencies with 
    # git clone this repository
git clone https://github.com/chaofengc/Face-Sketch-SCG.git
cd Face-Sketch-SCG 

# create new anaconda env
conda create -n sketch python=3.8
source activate sketch 

# install python dependencies
pip3 install -r requirements.txt

⚡ Quick Test

You may test with aligned face images using the following command:

python test.py -i ./test_images -o results/ --direction AtoB

The available options are

name default description
-i, --input input image path or folder path
-o, --output results output folder path
-s, --style cufs sketch styles: cufs, cufsf, wildsketch
-d, --direction AtoB AtoB: RGB to gray sketch; BtoA: gray sketch to RGB
-w, --weight None custom weight path
-g, --ground_truth None ground truth images for calculating metrics: fsim, lpips, dists

🖼️ Example Results

📊 Performance on Public Benchmarks

Because we improve the training details after cleaning the codes, results of the released models are slightly different from numbers in the paper:

Datasets CUFS (FSIM↑/LPIPS↓/DISTS↓) CUFSF (FSIM↑/LPIPS↓/DISTS↓) WildSketch (FSIM↑/LPIPS↓/DISTS↓)
Paper Results (AtoB) 0.7343/0.3232/0.1967 0.7261/0.3489/0.184 0.7010/-/-
Released 0.7349/0.3140/0.1875 0.7258/0.3473/0.1777 0.7010/0.3165/0.2528
Paper Results (BtoA) 0.7652/0.3374/0.1710 0.7777/0.3527/0.2082 N/A
Released 0.7789/0.2874/0.1633 0.7816/0.3592/0.2229 N/A

Here is the example benchmark test script for photo-to-sketch (AtoB) with CUFS dataset:

python test.py -i ./dataset/CUFS/test_photos/ -o tmp_test_results -s cufs -d AtoB -g ./dataset/CUFS/test_sketches/

🛠️ Train the Model

You may first download the datasets and use the provided example script to train the model.

⏬ Download Datasets

For convenient, we put all the training dataset here. Please download and extract them under the dataset/ folder. You may also download the dataset with the following commands:

wget https://github.com/chaofengc/Face-Sketch-SCG/releases/download/v0.1/dataset.tgz

👩🏻‍💻 Train Commands

Here is an example training command with cufs as reference dataset:

python train.py --gpus 1 --name FaceSCG_v001 --model semicycle_gan --total_epochs 100 \
    --lr 0.002 --beta1 0.9 \
    --lr_policy 'cosine' --n_epochs 20 \
    --Gnorm "gn" --Dnorm "gn" --act_type silu --n_layers_D 3 --D_num 3 \
    --lambda_mrf 1 --lambda_sty 1 --lambda_g 1 --lambda_A 1 --lambda_pcp 1 --sigma 20 --patch_k 3 \
    --train_style cufs --vggface ./dataset/vggface \
    --dataset_name fsvgg --batch_size 4 --Gin_size 256 --topk 3 \
    --visual_freq 50 --print_freq 10 --save_iter_freq 200 --save_latest_freq 100

Please refer to the codes for details of these options.

📑 Citation

If you find this code or the provided data useful in your research, please consider cite:

@inproceedings{chen2023face-sketch-scg,
    title={Semi-supervised Cycle-GAN for face photo-sketch translation in the wild},
    author={Chen, Chaofeng and Liu, Wei and Tan, Xiao and Wong, KKY},
    booktitle={Computer Vision and Image Understanding (CVIU)},
    year={2023},
    doi={https://doi.org/10.1016/j.cviu.2023.103775},
}

📜 License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Creative Commons License