Data path (os.path.isdir(path) returning false when it exists) using FB XLM

[bala1802]

I'm trying to run an evaluation with the below configuration,

python XLM/train.py
--n_heads 8
--bt_steps 'python_sa-cpp_sa-python_sa,cpp_sa-python_sa-cpp_sa,java_sa-cpp_sa-java_sa,cpp_sa-java_sa-cpp_sa,python_sa-java_sa-python_sa,java_sa-python_sa-java_sa' # The evaluator will use this parameter to infer the languages to test on
--max_vocab '-1'
--word_blank '0.1'
--n_layers 6
--generate_hypothesis true
--max_len 512
--bptt 256
--fp16 true
--share_inout_emb true
--tokens_per_batch 6000
--has_sentences_ids true
--eval_bleu true
--split_data false
--data_path '/content/drive/MyDrive/TransCoder/repository/TransCoder/data/XLM-cpp-java-python-with-comments/'
--eval_computation true
--batch_size 32
--reload_model 'model_1.pth,model_1.pth'
--amp 2
--max_batch_size 128
--ae_steps 'cpp_sa,python_sa,java_sa'
--emb_dim 1024
--eval_only True
--beam_size 10
--retry_mistmatching_types 1
--dump_path '/tmp/'
--exp_name='eval_final_model_wc_30'
--lgs 'cpp_sa-java_sa-python_sa'
--encoder_only=False

Looks like the data_path params is not getting updated when passed as an argument, coz of this the assertion statements in loader.py

assert os.path.isdir(params.data_path), params.data_path this assertion failed

[IgWod]

Hi,

I guess you may have already fixed the problem, but for anyone facing a similar issue in the future the fix is to download the pre-train model before running the evaluation from here.

If model is not available then the application crashes, because of the following option:

--reload_model 'model_1.pth,model_1.pth'

Hope that helps!

[malachaux]

You should provide abolute path on your machine (so downloaded) for both the data_path and reload_model

[malachaux]

This repo is now deprecated. Please now refer to our new repository https://github.com/facebookresearch/CodeGen.