Problems not identified in string-comparison errors on PHPUnit 5.7
Closed this issue · 2 comments
For my sins, I'm using an older version of PHPUnit. I've noticed that problems aren't always picked up.
Repo - working case
This output marks a problem correctly:
PHPUnit 5.7.27 by Sebastian Bergmann and contributors.
...F.......... 14 / 14 (100%)
Time: 652 ms, Memory: 8.00MB
There was 1 failure:
1) SilverStripe\Config\Tests\Transformer\YamlTransformerTest::testNoNameStatement
Failed asserting that false is true.
/Users/samminnee/Projects/silverstripe-config/tests/Transformer/YamlTransformerTest.php:183
FAILURES!
Tests: 14, Assertions: 23, Failures: 1.
Generating code coverage report in Clover XML format ... done
The terminal process "/bin/bash '-c', ''/Users/samminnee/Projects/silverstripe-config/vendor/bin/phpunit' --colors=always '/Users/samminnee/Projects/silverstripe-config/tests/Transformer/YamlTransformerTest.php''" terminated with exit code: 1.
Terminal will be reused by tasks, press any key to close it.
Outcome
The failure is linked as a problem
Repo - broken case
But this output does not:
> Executing task: '/Users/samminnee/Projects/silverstripe-config/vendor/bin/phpunit' --colors=always '/Users/samminnee/Projects/silverstripe-config/tests/Transformer/YamlTransformerTest.php' <
PHPUnit 5.7.27 by Sebastian Bergmann and contributors.
...F.......... 14 / 14 (100%)
Time: 734 ms, Memory: 8.00MB
There was 1 failure:
1) SilverStripe\Config\Tests\Transformer\YamlTransformerTest::testNoNameStatement
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'blahy'
+'blah'
/Users/samminnee/Projects/silverstripe-config/tests/Transformer/YamlTransformerTest.php:184
FAILURES!
Tests: 14, Assertions: 23, Failures: 1.
Generating code coverage report in Clover XML format ... done
The terminal process "/bin/bash '-c', ''/Users/samminnee/Projects/silverstripe-config/vendor/bin/phpunit' --colors=always '/Users/samminnee/Projects/silverstripe-config/tests/Transformer/YamlTransformerTest.php''" terminated with exit code: 1.
Terminal will be reused by tasks, press any key to close it.
Outcome
No problems are linked
Expected outcome
The failure is linked as a problem
Thanks for the bug report, unfortunately I'm not super actively maintaining this so I can't support older versions. At least if it requires different problem matchers for different versions it sounds quite tricky to fix.
Perhaps the current problem matcher could be patched though. I'd happily accept a PR that works for both cases :)
I'll close this as I'll aim to only support later versions of phpunit and v5 is end of life https://phpunit.de/supported-versions.html