Ensure every package's test can complete without errors

[Leo3418]

GSoC Idea

Resolving this issue is a good objective for a Gentoo GSoC project. Interested in participating in GSoC? You are invited to apply as a GSoC contributor! Please find more information on Gentoo Wiki as well as Google's program homepage.

You are still welcome to resolve this issue without participating in GSoC - just submit a pull request containing the fix!

Issue Description

Although many ebuilds in the Spark overlay perform some sort of verification in the src_test phase, some of those ebuilds will fail src_test. For each of the ebuilds, either the underlying issue causing the src_test failures should be fixed, or RESTRICT="test" should be added to the ebuild if it is not fixable.

In general, the ebuilds that fail src_test contain JAVA_TESTING_FRAMEWORKS="pkgdiff" - which will make java-pkg-simple.eclass run japi-compliance-checker and then pkgdiff on the JAR built from source and the pre-built binary JAR specified by JAVA_BINJAR_FILENAME - and the checks run by any of those tools fail.

So far, commonly-observed causes of those check failures include but are not limited to:

• japi-compliance-checker
  • Discrepancy in the sets of source files used to build the two JARs being compared: sometimes, the sources JAR whose contents are used to build the package from source might contain less files than the set of files the upstream used to build the binary JAR
  • Minor API incompatibilities, such as generic type vs. raw type in parameter/return types, addition/removal of the abstract keyword
  • Inability to reproduce a JAR equivalent to the pre-built binary one, even if the sets of source files used to build those JARs are identical
• pkgdiff
  • Inclusion of additional files in the pre-built binary JAR, like XML files, HTML files
  • Use of fields that are expected to change between builds: in some packages, there is a file that records the build timestamp, which is inherently different between builds executed at different times

The bare minimum acceptance criterion of a resolution to this issue is that the leaf-ebuilds-test jobs run in the daily CI tests pass. Currently, they always fail due to this issue, but their failures are ignored.

leaf-ebuilds-binary-test might require more effort to fix but might also be not very useful - please look at issue #12 for more details.

After any CI jobs have been fixed, please remove may-fail: true for the fixed jobs from the GitHub Actions workflow file for the tests, so their failures would no longer be ignored.