automl/DACBench

Unexpected Change of Seed in SDG due to Instance Seeds (logging)

maximilianreimer opened this issue · 1 comments

Since Commit f511fd9 the logged seed is automatically loaded from initial_seed variable of the the environment. Since SDG reseeds the environment in the reset()-a method the leads also to a change of the logged seed.

Is this behaviour intendent?

Due to this the tests TestRunBaselines.test_run_static_SGDBenchmark and TestRunBaselines.test_run_random_SGDBenchmark currently fail and have been disbaled for now.

This is not how the seed method is supposed to be used, really.
@Steven-Adriaensen I'm not sure if the env seed is supposed to change with the training seed or not, can you maybe clarify what's happening?