microsoft/evodiff

Scaffolding benchmark produces different sequences every time it is run

abhi-senthilkumar opened this issue · 1 comments

I'm not sure if this is expected behavior, but the scaffolding benchmark code produces different sequences every time it is run. It appears to be because in conditional_generation.py, while the numpy and pytorch seeds are set to 0, the random seed is not set and random.randint is used to generate the scaffold length. When I manually set random.seed(0), I got the same sequences on every run.

this is the expected behavior. We do not want the same sequences for each generation, and we sample sequence length in the paper.