test_hypothesis.py::TestList::test fails on 32-bit systems
ginggs opened this issue · 6 comments
As seen in Debian bullseye testing:
https://ci.debian.net/packages/p/python-fakeredis/testing/i386/
https://ci.debian.net/packages/p/python-fakeredis/testing/armhf/
=================================== FAILURES ===================================
________________________________ TestList.test _________________________________
self = <test_hypothesis.TestList object at 0xf5fb19b8>
@pytest.mark.slow
def test(self):
class Machine(CommonMachine):
create_command_strategy = self.create_command_strategy
command_strategy = self.command_strategy
> hypothesis.stateful.run_state_machine_as_test(Machine)
test/test_hypothesis.py:468:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib/python3/dist-packages/hypothesis/stateful.py:75: in run_state_machine_as_test
def run_state_machine_as_test(state_machine_factory, *, settings=None):
/usr/lib/python3/dist-packages/hypothesis/internal/reflection.py:654: in accept
return func(*bound.args, **bound.kwargs)
/usr/lib/python3/dist-packages/hypothesis/stateful.py:200: in run_state_machine_as_test
run_state_machine(state_machine_factory)
/usr/lib/python3/dist-packages/hypothesis/stateful.py:92: in run_state_machine
@given(st.data())
test/test_hypothesis.py:454: in one_command
self._compare(command)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = Machine({}), command = Command('lindex', b'', 2147483648)
def _compare(self, command):
fake_result, fake_exc = self._evaluate(self.fake, command)
real_result, real_exc = self._evaluate(self.real, command)
if fake_exc is not None and real_exc is None:
raise fake_exc
elif real_exc is not None and fake_exc is None:
> assert real_exc == fake_exc, "Expected exception {} not raised".format(real_exc)
E AssertionError: Expected exception value is out of range not raised
E assert ResponseError...out of range') == None
E +ResponseError('value is out of range')
E -None
test/test_hypothesis.py:412: AssertionError
----------------------------- Captured stdout call -----------------------------
Falsifying example:
state = Machine()
state.init_attrs(attrs={'fields': [b'', b'\x00'],
'keys': [b'', b'\x00'],
'scores': [0.0, 1.0],
'values': [b'', b'\x00']})
state.init_data(commands=[Command('rpush', b'', b'')])
state.one_command(command=Command('lindex', b'', 2147483648))
state.teardown()
I wasn't aware of this, but it appears that the redis server has different semantics depending on whether it is 32-bit or 64-bit, and fakeredis is faking a 64-bit server, so running the hypothesis tests against a 32-bit server is probably going to fail in lots of obscure ways.
The hypothesis tests have always been somewhat brittle because they probe undocumented corner-case behaviour that varies even between patch releases of redis, and they're only really expected to pass on whatever Redis server is deployed on Travis.
I can look into pytest markers for a way to disable them by default and enable them only on Travis (and document the above), but as a workaround I'd suggest just running pytest test/test_aioredis.py test/test_fakeredis.py. That'll test the stuff normal humans actually use.
@bmerry Thanks for the explanation, that makes sense.
Hope to see you at PyConZA again some day!
@ginggs would you mind testing the latest master? In theory I've disabled the hypothesis tests for 32-bit redis servers, but I don't actually have a 32-bit system to test on.
I built latest master using the current packaging from Debian. On 64-bit (amd64), I see:
test/test_hypothesis.py::TestTransaction::test FAILED
However, on 32-bit (armhf) I see the following, which I believe means 919d951 does the right thing.
test/test_hypothesis.py::TestConnection::test SKIPPED
test/test_hypothesis.py::TestString::test SKIPPED
test/test_hypothesis.py::TestHash::test SKIPPED
test/test_hypothesis.py::TestList::test SKIPPED
test/test_hypothesis.py::TestSet::test SKIPPED
test/test_hypothesis.py::TestZSet::test SKIPPED
test/test_hypothesis.py::TestZSetNoScores::test SKIPPED
test/test_hypothesis.py::TestTransaction::test SKIPPED
test/test_hypothesis.py::TestServer::test SKIPPED
test/test_hypothesis.py::TestJoint::test SKIPPED
test/test_hypothesis.py::TestFuzz::test SKIPPED
If I'm understanding that 64-bit error correctly, the redis server refused a connection at one point, but not repeatably, and hypothesis gets upset if the test doesn't behave consistently. It's probably worth rerunning it in case it was a transient error from the redis server.