jakartaee/data

[Bug]: maven-surefire-plugin version and configuration issues

Closed this issue · 1 comments

starksm64 commented

Specification Version

SNAPSHOT (Locally built)

Bug report

The tck docs provide examples of using a <maven.surefire.plugin.version>2.22.2</maven.surefire.plugin.version> and incomplete examples of how to set the groups property, and do not use the jakarta.tck.profile setting, which is a required property by the tck. The current version of the surefire plugin is 3.2.5.

Failure to set the jakarta.tck.profile results in:

[ERROR] Errors: 
[ERROR]   ExtensionTests » IllegalState Could not obtain a value for system property: jakarta.tck.profile
[ERROR]   EntityTests » IllegalState Could not obtain a value for system property: jakarta.tck.profile
[ERROR]   StandaloneExampleTests » IllegalState Could not obtain a value for system property: jakarta.tck.profile
[ERROR]   PersistenceEntityTests » IllegalState Could not obtain a value for system property: jakarta.tck.profile
[ERROR]   SignatureTests » IllegalState Could not obtain a value for system property: jakarta.tck.profile
[ERROR]   ExtensionTests » IllegalState Could not obtain a value for system property: jakarta.tck.profile
[ERROR]   ComplexServletTests » IllegalState Could not obtain a value for system property: jakarta.tck.profile
[ERROR]   SimpleServletExampleTests » IllegalState Could not obtain a value for system property: jakarta.tck.profile
[ERROR]   PersistenceTests » IllegalState Could not obtain a value for system property: jakarta.tck.profile
[ERROR]   ValidationTests » IllegalState Could not obtain a value for system property: jakarta.tck.profile

There seem to be more tests run than specified in the groups setting. If I just specify persistence in the in groups setting and core for jakarta.tck.profile, I'm seeing all tests still being run:

starksm@Scotts-Mac-Studio runner-web % grep 'Tests run:' core.log
[ERROR] Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 4.585 s <<< FAILURE! -- in ee.jakarta.tck.data.core.cdi.ExtensionTests
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.142 s -- in ee.jakarta.tck.data.standalone.signature.SignatureTests
[INFO] Tests run: 73, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.693 s -- in ee.jakarta.tck.data.standalone.entity.EntityTests
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.872 s -- in ee.jakarta.tck.data.standalone.example.StandaloneExampleTests
[ERROR] Tests run: 10, Failures: 0, Errors: 10, Skipped: 0, Time elapsed: 0.869 s <<< FAILURE! -- in ee.jakarta.tck.data.standalone.persistence.PersistenceEntityTests
[ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.512 s <<< FAILURE! -- in ee.jakarta.tck.data.web.transaction.PersistenceTests
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.852 s -- in ee.jakarta.tck.data.web.example.ComplexServletTests
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.119 s -- in ee.jakarta.tck.data.web.example.SimpleServletExampleTests
[ERROR] Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 1.023 s <<< FAILURE! -- in ee.jakarta.tck.data.web.cdi.ExtensionTests
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.158 s -- in ee.jakarta.tck.data.web.validation.ValidationTests
[ERROR] Tests run: 103, Failures: 0, Errors: 15, Skipped: 0

This example run is from the runner-web module of https://github.com/jakartaredhat/hibernate-data-tck

We need more complete example configurations for running sets of tests on a recent version of the surefire plugin.

Additional information

Provide a complete examples of the se-pom.xml and ee-pom.xml using 3.2.5 of surefire with examples of multiple groups and different standalone/core/web profiles.

starksm64 commented

It would be good to add a link to the tag expression syntax in the docs when discussing the groups and tags:
https://junit.org/junit5/docs/current/user-guide/#running-tests-tag-expressions