Part 2: CRS should fail if it key tests were skipped
jerstlouis opened this issue · 3 comments
Describe the bug
Prior to the OoM issue (#181) being resolved, the tests reported our Part 2: CRS implementation as successful, even though it had skipped most tests because of Out of Memory failure, such as bbox-crs (which was not yet implemented).
To Reproduce
Difficult now that the OoM issue has been resolved, but see https://cite.opengeospatial.org/teamengine/reports/jerstlouis4/s0002/html/ .
CRS was reported as passed with only a single successful test:
Verify Requirement Class: Precondition: requirement class http://www.opengis.net/spec/ogcapi-features-2/1.0/req/crs must be implemented
Verify Collections Collection Crs Identifier Of Crs Property: Implements A.1 Discovery, Abstract Test 1 (Requirement /req/crs/crs-uri, /req/crs/fc-md-crs-list A, /req/crs/fc-md-storageCrs, /req/crs/fc-md-crs-list-global), crs property in the collection objects in the path /collections; java.util.NoSuchElementException: No value present
Verify Collections Collection Crs Identifier Of Storage Crs: Implements A.1 Discovery, Abstract Test 1 (Requirement /req/crs/crs-uri, /req/crs/fc-md-crs-list A, /req/crs/fc-md-storageCrs, /req/crs/fc-md-crs-list-global), storageCrs property in the collection objects in the path /collections; java.util.NoSuchElementException: No value present
Verify Collections Path Collection Crs Property Contains Default Crs: Implements A.1 Discovery, Abstract Test 2 (Requirement /req/crs/fc-md-storageCrs-valid-value), storageCrs property in the collection objects in the path /collections; java.util.NoSuchElementException: No value present
Verify Collections Path Collection Crs Property Contains Default Crs: Implements A.1 Discovery, Abstract Test 2 (Requirement /req/crs/fc-md-crs-list B), crs property contains default crs in the collection objects in the path /collections; java.util.NoSuchElementException: No value present
Expected behavior
Conformance classes should fail if key tests were skipped.
Screenshots
Additional context
Originally reported in #181
Thank you for reporting.
We will discuss in the CITE team how the mechanism can be improved here.
@jerstlouis We are currently working on this issue. Do you have a public test API to reproduce this issue?
@dstenger Sorry, no, because our public API tries to conform to the requirements.
But this should be really easy to verify from code inspection, ensuring that the conformance class fails if essential tests were skipped.