Communication Algorithms via Deep Learning

This repository is an implementation of "Communication Algorithms via Deep Learning" https://arxiv.org/abs/1805.09317.

Main Idea: This paper claims that a Recurrent Neural Network can learn from data to decode noisy signal over Additive White Gaussian Noise (AWGN) Channel as good as Viterbi and BCJR algorithm.
Reproduced Result (Test data = 10,000 sequences, K = 100):

Paper Result (Appendix A, page 12):

Usage

1. Install dependencies

conda env create -f environment.yml
source activate deepcom

2. (Recommend) IPython Notebook for training/benchmarking RNN with Viterbi Decoder.

reproduce_result.ipynb: A Jypyter notebook demonstrates how to train a Neural Decoder and compare the performance with Viterbi Decoder.

3. (Optional) Steps to reproduce the result yourself.

Please see at the bottom of this README file.

Network Architecture:

Why Bi-directional, and not uni-directional, RNN? Similar to dynamic programming, it usually consists of a forward and backward steps. The Bi-directional RNN architecture allows the network to learn the feature representation in both direction. I demonstrated a fail case, when using Uni-directional RNN, in unidirection_fail_not_converge.ipynb notebook.

Proper training data matters. Given message bit sequence K, transmitted codeword sequence of length c and data rate r. Then, the paper provides an emperical method to select SNR_train as:

For example, the paper uses r=1/2 and block length c=2K. Then SNR_{train} =min(SNR_{test}, 0). However, I ran an experiment and found that the model still converge when training model on higher SNR. In this example, we trained on SNR=4.0 and SNR_test = [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0]:

Steps to reproduce the result

Generate synthetic data for training/testing. This script will generate a pickle file rnn_12k_bl100_snr0.dataset

python generate_synthetic_dataset.py \
--snr 0 \
--block_length 100 \
--num_training_sequences 12000\
--num_testing_sequences  10000  \
--num_cpu_cores 8 \
--training_seed 2018 \
--testing_seed 1111

Train the network

For GPU supported machine

python train_rnn.py \
--dataset ./rnn_12k_bl100_snr0.dataset \
--batch_size 200
--epochs 50
--dropout_Rate 0.7

For CPU, properly take a long time to converge

python train.py \
--dataset ./rnn_12k_bl100_snr0.dataset \
--batch_size 4
--epochs 50
--dropout_Rate 0.7

Benchmark the result, there are two ways
- Use a script to only benchmark the Neural Decoder (over multiple SNRs).
```
python evaluate.py \
--checkpoint_dir ./reports/logs/BiGRU-2-400::dropout0.7::epochs-50
--dataset ./rnn_12k_bl100_snr0.dataset \
--batch_size 200 \
```
- Use an existing benchmark notebook in reports/benchmark.ipynb