About evaluation result in Table 1
Closed this issue · 2 comments
Hello, thank you for such a great project. I have a question when reading Table 1 of the paper:
Are the evaluation data on APE and AVE shown in Table 1 the average of multiple evaluation results? If so I would like to ask how many experiments you did?
Because I changed the random number and trained the model multiple times, the results obtained each time were not as good as shown in the paper. Even if I averaged multiple experiments, I did not achieve the effect shown in the paper.
Hello @fyyakaxyy,
In Table 1, it is not an average of multiple evaluation results. It corresponds to only one random generation. I actually made only one experiment. In Table 2, I am generating it 10 times and do the average (avg) or take the best.
The training of such models are not 100% deterministic so it may be normal that the results differ.
Hello @fyyakaxyy,
In Table 1, it is not an average of multiple evaluation results. It corresponds to only one random generation. I actually made only one experiment. In Table 2, I am generating it 10 times and do the average (avg) or take the best.
The training of such models are not 100% deterministic so it may be normal that the results differ.
Thank you!