Neural estimation of the rate-distortion function and its applications to reverse channel coding and compression. Provides a neural estimator (NERD) for estimating the rate-distortion function R(D) from i.i.d. samples. Uses NERD to implement single-shot lossy compression with guarantees on achievable rate-distortion.
For full details, see:
Eric Lei, Hamed Hassani, and Shirin Saeedi Bidokhti. "Neural Estimation of the Rate-Distortion Function With Applications to Operational Source Coding." arXiv preprint arXiv:2204.01612 (2022).
Eric Lei, Hamed Hassani, and Shirin Saeedi Bidokhti. "Neural Estimation of the Rate-Distortion Function For Massive Datasets," in 2022 IEEE International Symposium on Information Theory (ISIT), June 2022.
Trained networks have been saved in the trained_lagr/
folder. These can be used to plot RD curves in plotRDcurves_release.ipynb
. To train NERD from scratch on image datasets, it is easier to pretrain a GAN on the dataset first, and use the GAN to initialize the trained_gan/
folder for MNIST, FMNIST, and SVHN datasets. To run NERD with these pretrained GANs, simply run bash scripts/NERD_{dataset}.sh
. If you wish to pretrain a GAN yourself, we have the wgan_gp.py
file which trains a Wasserstein GAN with gradient penalty.
Once you have trained NERD, the RCC methods can be run via bash scripts/RCC_{dataset}.sh
.
- torch
- numpy
- scipy
- pytorch-lightning==1.9.0
- pykeops
- huffman
- tqdm
- argparse
- matplotlib
@ARTICLE{lei2022neuralrd,
author={Lei, Eric and Hassani, Hamed and Bidokhti, Shirin Saeedi},
journal={IEEE Journal on Selected Areas in Information Theory},
title={Neural Estimation of the Rate-Distortion Function With Applications to Operational Source Coding},
year={2022},
volume={3},
number={4},
pages={674-686},
doi={10.1109/JSAIT.2023.3273467}
}