Code Appendix of Idempotence and Perceptual Image Compression
Environments:
python 3.10, pytorch 2.0.1, compressAI 1.1.5
other versions might work as well, but this is the version we have tested on.
Pre-trained models:
all the Pre-trained model should be downloaded and put in ./bins
Pre-trained MSE codec:
Pre-trained unconditional generative model by the authors of DPS:
We provide the first 5 image of FFHQ, ImageNet, COCO and CLIC dataset in data/*_samples. Those images are un-curated. They are provide for demo purpose.
To actually reproduce the number, the first 1000 samples of those datasets are needed. They can be obtained by:
After downloading the pre-trained, the prepared dataset sample can be played with
python -u run_codec.py --model_config=configs/ffhq_model_config.yaml --diffusion_config=configs/diffusion_config.yaml --task_config=configs/elic_config.yaml --save_dir=results
The result is put in save_dir, with 4 sub-dir
label: the source image
mse_recon: the reconstruction of MSE codec
per_recon: the perceptual reconstruction of our proposed codec
re_recon: the recompressed perceptual reconstruction
The correct behaviour is:
mse_recon looks very blurry
per_recon looks sharp
re_recon looks very similar with mse_recon
we provide an example of run results on 5 images of FFHQ, using elic bitrate 1, in results_example_ffhq_elic_q1
label:
mse_recon:
per_recon:
re_recon:
This example run should be able to be reproduced almost exactly by the above command
To run a codec with DDPM trained on FFHQ and ELIC base codec
python -u run_codec.py --model_config=configs/ffhq_model_config.yaml --diffusion_config=configs/diffusion_config.yaml \
--task_config=configs/elic_config.yaml --save_dir=results
To run a codec with DDPM trained on Imagenet and ELIC base codec
python -u run_codec.py --model_config=configs/imagenet_model_config.yaml --diffusion_config=configs/diffusion_config.yaml \
--task_config=configs/elic_config.yaml --save_dir=results
To run a codec with DDPM trained on FFHQ and Hyperprior base codec
python -u run_codec.py --model_config=configs/ffhq_model_config.yaml --diffusion_config=configs/diffusion_config.yaml \
--task_config=configs/hyper_config.yaml --save_dir=results
To run a codec with DDPM trained on Imagenet and Hyperprior base codec
python -u run_codec.py --model_config=configs/imagenet_model_config.yaml --diffusion_config=configs/diffusion_config.yaml \
--task_config=configs/hyper_config.yaml --save_dir=results
To run a codec with DDIM trained on Imagenet and ELIC base codec by DSG method
python -u run_codec.py --model_config=configs/imagenet_model_config.yaml --diffusion_config=configs/diffusion_ddim200_config.yaml \ --task_config=configs/elic_dsg_config.yaml --save_dir=results
The dataset root and bitrate is in task_config.yaml.
Evaluation:
to evaluate bpp, run the base codec test code:
python test_elic.py
python test_gg18.py
to evaluate mse, psnr and FID, run