zhuchen03/FreeLB

Errors generated during data preprocessing

yupeijei1997 opened this issue · 2 comments

yupeijei1997 commented

hi,

When i execute this command source ./examples/roberta/preprocess_GLUE_tasks.sh glue_data ALL, the following error is generated.

  File "/root/FreeLB/fairseq-RoBERTa/fairseq_cli/preprocess.py", line 2
    ../preprocess.py
    ^
SyntaxError: invalid syntax

https://github.com/zhuchen03/FreeLB/blob/master/fairseq-RoBERTa/fairseq_cli/preprocess.py#L1

I don't understand why this script is written like this （ ../preprocess.py）

But after finishing the data processing, I can run run_glue.sh successfully.

so what is the purpose of this script？

Hope to get your answer, thanks.

zhuchen03 commented

Hi,

Most of the code in this repo comes from fairseq and I did not write this data preprocessing script. It is actually a symbolic link, not a python file, as in previous fairseq releases: https://github.com/pytorch/fairseq/tree/v0.6.2

  • 👍1
yupeijei1997 commented

Thanks for your answer！