THUDM/HGB

The random seed problem in experimental setting

wjwangppt opened this issue · 3 comments

wjwangppt commented

Dear author:
I noticed that the code you provide didn't fix the random seed. Can you provide the detail random seed corresponding to the tabular in the paper? I think it will bring the deviation of experimental results because the validation set is split in code, and different seed will cause different results.
Thanks a lot if you provide the random seed your benchmark set.

1049451037 commented

We have already considered the random deviation by repeatedly running the code five times. The results in the tabular is the average and standard deviation of 5 times running.

wjwangppt commented

We have already considered the random deviation by repeatedly running the code five times. The results in the tabular is the average and standard deviation of 5 times running.

Thanks

NovelinUp commented

That mean use the seed 1 in the code or can set other seed each run? @1049451037