tests/test_hitlet.py test too stringent?

Question

tests/test_hitlet.py test too stringent?

Closed this issue 3 years ago · 5 comments

Master failed on https://github.com/AxFoundation/strax/runs/3787298221?check_suite_focus=true for test_conditional_entropy(data, size_template_and_ind_max_template) with:

data = array([-3.8847911, -0.5546908, -0.5546908, -8.079916 ,  7.364694 ,
        7.378992 , -0.5546908, -0.5546908, -0.5546908], dtype=float32)
size_template_and_ind_max_template = [1, 0]

Perhaps rtol is too small?

Answer 1 · 2021-10-07T12:33:45.000Z

Hmm sure, if this starts to become an issue we can loosen the tolerance a bit. I have a different question though Since currently our tests are passing, I was always in the belief hypothesis would retry a certain test if it failed in the past. Is this only true for tests run locally?

Answer 2 · 2021-10-07T12:34:36.000Z

Are you fine with observing the situation a bit before loosing the test? If so feel free to close.

Answer 3 · 2021-10-07T13:07:03.000Z

Yeah, that's only locally. We install it every time, so there is no database or so that keeps track of fails.

Answer 4 · 2022-01-10T07:47:27.000Z

@WenzDaniel would you mind having a look, it seems that it's still an issue
https://github.com/AxFoundation/strax/runs/4758294267?check_suite_focus=true

Answer 5 · 2022-01-10T08:55:14.000Z

Sure let me loosen the test requirement a bit.