Portion of translations from select_successful_tests.py fails testcases
AlexShypula opened this issue · 0 comments
AlexShypula commented
I have had issues being able to improve the pre-trained model after fine-tuning. If I run this sanity check on the output dataframe from select_successful_tests.py in this zipped file, I have found that on a subset of 500 translations, only around 53% are successful.
My suggestion is to remove the following line
https://github.com/facebookresearch/CodeGen/blob/main/codegen_sources/test_generation/select_successful_tests.py#L187
I will also submit a PR to correct this.