Code and data for AAAI 2022 paper "Multilingual Code Snippets Training for Program Translation"
The code is adapted from https://github.com/facebookresearch/CodeGen. Please refer to this repo for setting up the environment.
Unzip the CoST_data.zip
python data_prepro.py
python train_commands.py
You can modify the train_commands.py file to change experiment name, saving path etc.
First get the checkpoint
wget https://dl.fbaipublicfiles.com/transcoder/pre_trained_models/dobf_plus_denoising.pth
Get into the training path
cd codegen_sources/model
Run the commands from "python train_commands.py", or construct them by yourself.
Snippet Translation:
python train.py --exp_name exp_snippet_<lang1>_<lang2> --dump_path dumppath1 --data_path CoST_data/snippet_data/<lang1>_<lang2>/ --mt_steps <lang1>_sa-<lang2>_sa --encoder_only False --n_layers 0 --lgs <lang1>_sa-<lang2>_sa --max_vocab 64000 --gelu_activation true --roberta_mode false --amp 2 --fp16 true --tokens_per_batch 3000 --group_by_size true --max_batch_size 128 --epoch_size 10000 --split_data_accross_gpu global --has_sentences_ids true --optimizer 'adam_inverse_sqrt,warmup_updates=10000,lr=0.0001,weight_decay=0.01' --eval_bleu true --eval_computation false --generate_hypothesis true --validation_metrics valid_<lang1>_sa-<lang2>_sa_mt_bleu --eval_only false --max_epoch 50 --beam_size 5 --max_len 100 --n_layers_encoder 12 --n_layers_decoder 6 --emb_dim 768 --n_heads 12 --reload_model dobf_plus_denoising.pth,dobf_plus_denoising.pth
Program Translation:
Replace the "snippet_data" with "program_data" in data_path. Note that you may want to change the exp_name accordingly also.
MuST Training:
python train.py --exp_name exp_<lang1>_<lang2> --dump_path dumppath1 --data_path CoST_data/<data_type>/<lang1>_<lang2>/ --mt_steps <lang1>_sa-<lang2>_sa --encoder_only False --n_layers 0 --lgs <lang1>_sa-<lang2>_sa --max_vocab 64000 --gelu_activation true --roberta_mode false --amp 2 --fp16 true --tokens_per_batch 3000 --group_by_size true --max_batch_size 128 --epoch_size 10000 --split_data_accross_gpu global --has_sentences_ids true --optimizer 'adam_inverse_sqrt,warmup_updates=10000,lr=0.0001,weight_decay=0.01' --eval_bleu true --eval_computation false --generate_hypothesis true --validation_metrics valid_<lang1>_sa-<lang2>_sa_mt_bleu --eval_only false --max_epoch 50 --beam_size 5 --max_len 100 --n_layers_encoder 12 --n_layers_decoder 6 --emb_dim 768 --n_heads 12 --reload_model dumppath1/exp_<lang1>_<lang2>/<exp_id>/best-valid_<lang1>_sa-<lang2>_sa_mt_bleu.pth,exp_<lang1>_<lang2>/<exp_id>/best-valid_<lang1>_sa-<lang2>_sa_mt_bleu.pth
DAE pre-training (using Java as an example):
python train.py --exp_name all_2_Java --dump_path dumppath1 --data_path all_2_one/Java/ --mt_steps cpp_sa-java_sa,c_sa-java_sa,python_sa-java_sa,javascript_sa-java_sa,php_sa-java_sa,csharp_sa-java_sa --encoder_only False --n_layers 0 --lgs cpp_sa-c_sa-python_sa-javascript_sa-php_sa-csharp_sa-java_sa --max_vocab 64000 --gelu_activation true --roberta_mode false --amp 2 --fp16 true --tokens_per_batch 3000 --group_by_size true --max_batch_size 128 --epoch_size 10000 --split_data_accross_gpu global --has_sentences_ids true --optimizer 'adam_inverse_sqrt,warmup_updates=10000,lr=0.0001,weight_decay=0.01' --eval_bleu true --eval_computation false --generate_hypothesis true --validation_metrics valid_cpp_sa-java_sa_mt_bleu,valid_c_sa-java_sa_mt_bleu,valid_python_sa-java_sa_mt_bleu,valid_javascript_sa-java_sa_mt_bleu,valid_php_sa-java_sa_mt_bleu,valid_csharp_sa-java_sa_mt_bleu --eval_only false --max_epoch 200 --beam_size 10 --max_len 100 --ae_steps java_sa --lambda_ae 0:1,30000:0.1,100000:0 --n_layers_encoder 12 --n_layers_decoder 6 --emb_dim 768 --n_heads 12 --reload_model dobf_plus_denoising.pth,dobf_plus_denoising.pth
Evaluation:
Change "--eval_only" in the training command from false to true.
In CoST_data.zip, there are two folders, raw_data and processed_data. The raw_data contains the .csv files of the programming problems, where each file contains the aligned snippets and programs from different language. The processed_data contains tokenized data that has been splitted into train, validation and test sets.
Ming Zhu, Karthik Suresh and Chandan K. Reddy, "Multilingual Code Snippets Training for Program Translation". Proceedings of the 36th AAAI Conference on Artificial Intelligence (AAAI), Feb 22- Mar 1, 2022. Acceptance Rate 15%.
@article{zhu2022multilingual,
title={Multilingual Code Snippets Training for Program Translation},
author={Zhu, Ming and Suresh, Karthik and Reddy, Chandan K},
year={2022}
}