CI does not report error when unit tests fail
Closed this issue · 1 comments
phate commented
I just noticed it in PR #387 that the CI build job (which also runs the unit tests) does not fail even though a unit test fails. The only thing that failed was the valgrind-check job (that is how I noticed it).
The rule is as following:
$(RUNTESTS): run-% : %
@echo -n "Running test $^ ... " ; \
if $^ >$^.log 2>&1 ; then \
echo "success" ; \
else \
echo "failed. See $^.log:" ; cat $^.log ; \
fi
I have not really investigated, but I believe that the issue arises because the only thing that really fails is the test invocation, but not the actual make invocation that runs this test.
FYI: @caleridas @sjalander @haved
haved commented
You're absolutely right, this is the second time I have pushed changes to GitHub only to discover that some tests were failing locally