[AAAI'21] Modeling Deep Learning Based Privacy Attacks on Physical Mail

Introduction

PyTorch implementation of Neural-STE (Neural-See-Through-Envelope). Please refer to supplementary material (~68M) for more results.

Prerequisites

PyTorch compatible GPU
Python 3
See requirements.txt

Benchmark dataset

Real and synthetic dataset (~840MB)

Usage

Clone this repo:

 git clone https://github.com/BingyaoHuang/Neural-STE
 cd Neural-STE

Install required packages by typing
```
 pip install -r requirements.txt
```
Download CompenNeSt++ Real and synthetic dataset and extract to data/
Start visdom by typing
```
 visdom -port 8098
```
Once visdom is successfully started, visit http://localhost:8098 (train locally) or http://serverhost:8098 (train remotely).
Open train_Neural-STE.py and set which GPUs to use. An example is shown below, we use GPU 0, 2 and 3 to train the model.
```
 os.environ['CUDA_VISIBLE_DEVICES'] = '0, 2, 3'
 device_ids = [0, 1, 2]
```
Run train_Neural-STE.py to start training and testing
```
 cd src/python
 python train_Neural-STE.py
```
The training and validation results are updated in the browser during training. An example is shown below, where the 1st figure shows the training and validation loss, rmse and ssim curves. The 2nd and 3rd montage figures are the training and validation pictures, respectively. In each montage figure, the 1st rows are the camera-captured envelope (I), the 2nd rows are Neural-STE inferred hidden content (\hat{J}) and the 3rd rows are ground truth of the hidden content (J).
The quantitative comparison results will be saved to log/%Y-%m-%d_%H_%M_%S.txt after training.

If you use the dataset or this code, please consider citing our work

Citation

@inproceedings{huang2021Neural-STE,
    title={Modeling Deep Learning Based Privacy Attacks on Physical Mail},
    author={Bingyao Huang and Ruyi Lian and Dimitris Samaras and Haibin Ling},
    booktitle = {AAAI Conference on Artificial Intelligence (AAAI)},
    year={2021}
}

Acknowledgments

The PyTorch implementation of SSIM loss is modified from Po-Hsun-Su/pytorch-ssim. We thank the anonymous reviewers for valuable and inspiring comments and suggestions. We thank the authors of the colorful textured sampling images.

License

This software is freely available for non-profit non-commercial use, and may be redistributed under the conditions in license.