thoth-station/integration-tests

Provenance check tests pass when `Pipfile.lock` is modified

Closed this issue · 4 comments

mayaCostantini commented

Describe the bug
Integration tests pass successfully when the Pipfile.lock file is modified for the provenance_flask example in the test scenario for the thamos_provenance_check feature.

To Reproduce
Modify Pipfile.lock (example: remove the flask section within default in the JSON) and run tests for the thamos_provenance_check feature. Observe that tests are still green after the modification.

Expected behavior
Tests should fail.

mayaCostantini commented

/kind bug
/priority critical-urgent

fridex commented

Modify Pipfile.lock (example: remove the flask section within default in the JSON) and run tests for the thamos_provenance_check feature. Observe that tests are still green after the modification.

This will sadly not cause any provenance issue. The stack is no longer valid (users will not be able to run the application if they use such lock file), but the provenance is correct. To break provenance, one needs to have hashes that do we do not have recorded in our database. Or, use source of packages that we are not aware of (index that is not monitored).

  • 👍1
mayaCostantini commented

@fridex Thanks for the clarification, I am closing this issue then 👍🏻
/close

sesheta commented

@mayaCostantini: Closing this issue.

In response to this:

@fridex Thanks for the clarification, I am closing this issue then 👍🏻
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.