interuss/monitoring

GetSystemVersion test scenario skipped despite system_identity provided as local resource

Closed this issue · 0 comments

Describe the bug
GetSystemVersion test scenario is getting skipped while running the uss_qualifier tests using f3548_self_contained configuration due to missing system_identity resource.
The test suite configuration f3548_21.yaml provides a local resource for system_identity, but it's getting ignored even though there is no resource provided in the resource pool. This is bug, as a local resource should be picked up if there is no resource provided in resource pool.

To reproduce
Steps to reproduce the behavior:

  1. Set up the test environment by running make start-locally and make start-uss-mocks
  2. Run uss_qualifier tests with configurations.dev.f3548_self_contained.
  3. Look at the console Output. You would see the following logs -
2024-02-21 05:00:15.616 | DEBUG   | monitoring.uss_qualifier.suites.definitions:__init__:166 - Ignoring local resource `system_identity` in favor of definition provided in `resources`
2024-02-21 05:00:15.616 | WARNING  | monitoring.uss_qualifier.suites.suite:__init__:270 - Skipping action 0 (ActionType.TestScenario scenarios.versioning.GetSystemVersions) because Test suite action to run ActionType.TestScenario scenarios.versioning.GetSystemVersions could not find required resource ID "system_identity" used to populate child resource ID "system_identity"
  1. Also check output/f3548/report.json, where it reports skipped_action for scenarios.versioning.GetSystemVersions.

Difference from expected behavior
Expected behavior is that if a resource is defined in local resources then it should not be ignored, if that same resource is not defined in the resource pool. However, here we see that there is no system_identity resource defined in f3548_self_contained.yaml, and still local resource for systems_identity defined in f3548_21.yaml is getting ignored. And, hence causing GetSystemVersions test scenarios to be skipped.

System on which behavior was encountered
OS: iOS, Linux

Codebase information
Output of git log -n 1:

commit 69e3adf (HEAD -> main, origin/main, origin/HEAD)
Author: Julien Perrochet julien.perrochet@orbitalize.com
Date: Tue Feb 20 14:54:30 2024 +0100

[uss_qualifier] create a SubscriptionValidator for scd subscriptions (#490)

subscription_validator

Output of git status:

[paste output of git status command here]

Additional comment -
Also happening in CI run. Report attached.
report.json