In this repo, we provide the source code for you to reproduce the collapse of Pareto front phenomena and the visualization result as in our paper (https://arxiv.org/abs/2304.03216).
News: Our work has been accepted to NeurIPS 2023!
conda create -n ParetoMNMT python=3.8.15
conda activate ParetoMNMT
bash setup.sh
We provide the training scripts for reproducing the 2d and 3d trade-off front in our paper.
We also provide preprocessed binary data at GoogleDrive, which is needed to conduct following training.
cd scripts
bash frdezh_trade_off.sh # 3d-trade-off front
bash frzh_trade_off.sh # 2d-trade-off front
# you can split the training to different GPU to speed up
The training log and checkpoint will be saved at ./logs and ./checkpoints directories.
- you can also use the
scripts/inference.shto compute the BLEU score of each models
cd scripts
bash inference.sh <checkpoint_dir> # you can change the inferenced directions in the script
We provide a jupyter notebook ./scripts/3d-vis.ipynb to visulize the 3d Pareto front after training all models.
The results:
3d trade-off front of fr-de-zh with different data-adequacy
Please kindly cite our paper if you find it helpful in your work.
@article{Chen2023OnTP,
title={On the Pareto Front of Multilingual Neural Machine Translation},
author={Liang Chen and Shuming Ma and Dongdong Zhang and Furu Wei and Baobao Chang},
journal={ArXiv},
year={2023},
volume={abs/2304.03216}
}