szepeviktor/phpstan-wordpress

Coverage calculation problem

szepeviktor opened this issue · 6 comments

szepeviktor commented

Contributions are welcome!

Code Coverage Report:
  2022-06-01 21:26:20

 Summary:
  Classes: 13.64% (3/22)
  Methods: 53.23% (33/62)
  Lines:   32.20% (133/413)

SzepeViktor\PHPStan\WordPress\ApplyFiltersDynamicFunctionReturnTypeExtension
  Methods:  66.67% ( 2/ 3)   Lines:  91.67% ( 11/ 12)
SzepeViktor\PHPStan\WordPress\CurrentTimeDynamicFunctionReturnTypeExtension
  Methods:  50.00% ( 1/ 2)   Lines:   7.69% (  1/ 13)
SzepeViktor\PHPStan\WordPress\EchoParameterDynamicFunctionReturnTypeExtension
  Methods:  50.00% ( 1/ 2)   Lines:   5.88% (  1/ 17)
SzepeViktor\PHPStan\WordPress\GetCommentDynamicFunctionReturnTypeExtension
  Methods:  50.00% ( 1/ 2)   Lines:   6.25% (  1/ 16)
SzepeViktor\PHPStan\WordPress\GetListTableDynamicFunctionReturnTypeExtension
  Methods:  50.00% ( 1/ 2)   Lines:   8.33% (  1/ 12)
SzepeViktor\PHPStan\WordPress\GetObjectTaxonomiesDynamicFunctionReturnTypeExtension
  Methods:  50.00% ( 1/ 2)   Lines:   7.14% (  1/ 14)
SzepeViktor\PHPStan\WordPress\GetPostDynamicFunctionReturnTypeExtension
  Methods:  50.00% ( 1/ 2)   Lines:   6.25% (  1/ 16)
SzepeViktor\PHPStan\WordPress\GetPostsDynamicFunctionReturnTypeExtension
  Methods:  50.00% ( 1/ 2)   Lines:   4.35% (  1/ 23)
SzepeViktor\PHPStan\WordPress\GetTaxonomiesDynamicFunctionReturnTypeExtension
  Methods:  50.00% ( 1/ 2)   Lines:   6.67% (  1/ 15)
SzepeViktor\PHPStan\WordPress\GetTermsDynamicFunctionReturnTypeExtension
  Methods:  10.00% ( 1/10)   Lines:   1.59% (  1/ 63)
SzepeViktor\PHPStan\WordPress\HasFilterDynamicFunctionReturnTypeExtension
  Methods:  50.00% ( 1/ 2)   Lines:   7.14% (  1/ 14)
SzepeViktor\PHPStan\WordPress\HookDocBlock
  Methods: 100.00% ( 3/ 3)   Lines: 100.00% ( 13/ 13)
SzepeViktor\PHPStan\WordPress\HookDocsParamException
  Methods:  ( 0/ 0)   Lines:  (  0/  0)
SzepeViktor\PHPStan\WordPress\HookDocsRule
  Methods: 100.00% ( 7/ 7)   Lines: 100.00% ( 57/ 57)
SzepeViktor\PHPStan\WordPress\HookDocsVisitor
  Methods: 100.00% ( 2/ 2)   Lines: 100.00% ( 11/ 11)
SzepeViktor\PHPStan\WordPress\IsWpErrorFunctionTypeSpecifyingExtension
  Methods:  33.33% ( 1/ 3)   Lines:  37.50% (  3/  8)
SzepeViktor\PHPStan\WordPress\IsWpErrorRule
  Methods:  66.67% ( 2/ 3)   Lines:  86.96% ( 20/ 23)
SzepeViktor\PHPStan\WordPress\MySQL2DateDynamicFunctionReturnTypeExtension
  Methods:  50.00% ( 1/ 2)   Lines:   8.33% (  1/ 12)
SzepeViktor\PHPStan\WordPress\RedirectCanonicalDynamicFunctionReturnTypeExtension
  Methods:  50.00% ( 1/ 2)   Lines:  50.00% (  1/  2)
SzepeViktor\PHPStan\WordPress\ShortcodeAttsDynamicFunctionReturnTypeExtension
  Methods:  50.00% ( 1/ 2)   Lines:   6.67% (  1/ 15)
SzepeViktor\PHPStan\WordPress\StringOrArrayDynamicFunctionReturnTypeExtension
  Methods:  50.00% ( 1/ 2)   Lines:   6.67% (  1/ 15)
SzepeViktor\PHPStan\WordPress\TermExistsDynamicFunctionReturnTypeExtension
  Methods:  50.00% ( 1/ 2)   Lines:  11.54% (  3/ 26)
SzepeViktor\PHPStan\WordPress\WPErrorParameterDynamicFunctionReturnTypeExtension
  Methods:  33.33% ( 1/ 3)   Lines:   6.25% (  1/ 16)
herndlm commented

jfyi I took a look and wanted to add that the coverage metric is a bit misleading here. E.g. DynamicReturnTypeExtensionTest has the dataprovider dataFileAsserts that gathers assert types of many files which uses and actually covers many of the DynamicFunctionReturnTypeExtension classes implemented here, e.g. ShortcodeAttsDynamicFunctionReturnTypeExtension.
And in the actual test method testFileAsserts it asserts those types. But apparently this way no coverage is collected for those extension classes, even if they were executed. I'm not a 100% sure, but I assume this is because they are not executed via test method.

Long story short, many of the extension should already have quite good coverage, but the data we see does not reflect that. And right now I'm not sure how to fix this :/ CC @johnbillion

szepeviktor commented

I assume this is because they are not executed via test method

How could we become phpunit coverage friendly?
Maybe @ondrejmirtes could help.

herndlm commented

fyi this was also briefly discussed in phpstan/phpstan#8238 and apparently it is a known problem, the code coverage via dataproviders is currently just wrong. I don't have any fix for this and I think Ondrej doesn't have either atm since he's not pursuing code coverage

  • 👀1
herndlm commented

But I was thinking into looking this on phstan side a bit more. I'll let you know if I find out more

szepeviktor commented

I'm not willing to work on this package as WordPress core makes it impossible to work normally.

szepeviktor commented

This was a simple package back then.
Now it needs a LOT of PHP code to handle weird WordPress core functions.