Python random tests are failing but CI is passing
Opened this issue · 0 comments
marler8997 commented
The regex_test.py
and regex_test_neg.py
scripts are failing but they don't return a non-zero exit code. This causes the Makefile to continue on like nothing went wrong thinking that everything passed. This can be fixed by adding the following lines to the end of the regex python scripts:
if nfails != 0:
sys.exit(1)
The problem is that this will cause the CI to fail because the tests currently aren't passing, see alll the "FAIL" tests in this run: https://github.com/kokke/tiny-regex-c/runs/1939275804?check_suite_focus=true
[..snip..]
pattern '[1-5-]+[-1-2]-[-]': FAIL : doesn't match '-5--34522125-3131---' as expected [0x2d, 0x35, 0x2d, 0x2d, 0x33, 0x34, 0x35, 0x32, 0x32, 0x31, 0x32, 0x35, 0x2d, 0x33, 0x31, 0x33, 0x31, 0x2d, 0x2d, 0x2d].
FAIL : doesn't match '31433--3-33325121--' as expected [0x33, 0x31, 0x34, 0x33, 0x33, 0x2d, 0x2d, 0x33, 0x2d, 0x33, 0x33, 0x33, 0x32, 0x35, 0x31, 0x32, 0x31, 0x2d, 0x2d].
FAIL : doesn't match '12-32342324113--12--' as expected [0x31, 0x32, 0x2d, 0x33, 0x32, 0x33, 0x34, 0x32, 0x33, 0x32, 0x34, 0x31, 0x31, 0x33, 0x2d, 0x2d, 0x31, 0x32, 0x2d, 0x2d].
FAIL : doesn't match '455354554121-4515---' as expected [0x34, 0x35, 0x35, 0x33, 0x35, 0x34, 0x35, 0x35, 0x34, 0x31, 0x32, 0x31, 0x2d, 0x34, 0x35, 0x31, 0x35, 0x2d, 0x2d, 0x2d].
FAIL : doesn't match '---5551121353354--2--' as expected [0x2d, 0x2d, 0x2d, 0x35, 0x35, 0x35, 0x31, 0x31, 0x32, 0x31, 0x33, 0x35, 0x33, 0x33, 0x35, 0x34, 0x2d, 0x2d, 0x32, 0x2d, 0x2d].
FAIL : doesn't match '215535-35551-442--' as expected [0x32, 0x31, 0x35, 0x35, 0x33, 0x35, 0x2d, 0x33, 0x35, 0x35, 0x35, 0x31, 0x2d, 0x34, 0x34, 0x32, 0x2d, 0x2d].
FAIL : doesn't match '5-342342452523-52-41--' as expected [0x35, 0x2d, 0x33, 0x34, 0x32, 0x33, 0x34, 0x32, 0x34, 0x35, 0x32, 0x35, 0x32, 0x33, 0x2d, 0x35, 0x32, 0x2d, 0x34, 0x31, 0x2d, 0x2d].
FAIL : doesn't match '55515455412-1--' as expected [0x35, 0x35, 0x35, 0x31, 0x35, 0x34, 0x35, 0x35, 0x34, 0x31, 0x32, 0x2d, 0x31, 0x2d, 0x2d].
FAIL : doesn't match '2-3---' as expected [0x32, 0x2d, 0x33, 0x2d, 0x2d, 0x2d].
FAIL : doesn't match '444-2-3412--45433-1--' as expected [0x34, 0x34, 0x34, 0x2d, 0x32, 0x2d, 0x33, 0x34, 0x31, 0x32, 0x2d, 0x2d, 0x34, 0x35, 0x34, 0x33, 0x33, 0x2d, 0x31, 0x2d, 0x2d].
[..snip..]