Results of module 'background_fit' are not reproducible
Closed this issue · 6 comments
BerndDoser commented
Repeated execution of 'background_fit' with identical input shows different results.
Behavior can be reproduced with
python3 ../../src/background_fit.py --pds test_background_pds.csv --summary test_background_summary.csv --ofac_pds test_background_ofac_pds.csv --bins 300
Deviation in 'Hmax' is significant:
1st run: 10443.844542299583
2nd run: 10449.024796593862
3rd run: 10430.003248623914
BerndDoser commented
FYI @saskiahekker
saskiahekker commented
could you elaborate a bit on the none-reproducibility?
BerndDoser commented
I assume you didn't see the explanation of the issue (see #12), because I was only mention instead of assigning you.
I would expect the identical result (here Hmax for example) within numerical accuracy if the module is executed with the same input, if there is no random process anywhere.
saskiahekker commented
no I indeed did not see this. The background fitting is a bayesian fit and so the result should be in line within uncertainties and may not be exactly the same.
Best,
Saskia
BerndDoser commented
Add np.random.seed
as parameter to get reproducible results needed for unit testing.
saskiahekker commented
Great that you could solve it!