R-Transformer
Pytorch implementation of R-Transformer. Some parts of the code are adapted from the implementation of TCN and Transformer.
For more details about R-Transformer, Please read our paper. If you find this work useful and use it on your research, please cite our paper.
@article{wang2019rtransf,
title={R-Transformer: Recurrent Neural Network Enhanced Transformer},
author={Wang, Zhiwei and Ma, Yao and Liu, Zitao and Tang, Jiliang},
journal={arXiv preprint arXiv:1907.05572},
year={2019}
}
Usage
Our repository is arranged as follows:
[Task Name] /
data/ # contains the datasets
experiment.py #run experiment
model.py # comtains the model
utils.py # utility functions including dataset downloading and preprocessing
models /
RTransformer.py # RTransformer model
The dataset for the "polyphonic music modeling" experiment is already included in audio/data/. For other experiments that are based on much larger datasets, the data needs to be downloaded (from torchvision.datasets or observations) and then put into the "data" folder which should be created firstly.
When data is ready, the code can directly run with PyTorch 1.0.0.
Final Words
We will keep this repo updated and hope it is useful to your research.