thoth-station/integration-tests

[5pt] Some integration-tests for aws&smaug are red or can be turned off

fridex opened this issue · 14 comments

fridex commented

Describe the bug

See the most recent report from the AWS&Smaug clusters. We should be able to reproduce the same quality of report for the AWS cluster as for the stage environment.

To Reproduce

See the most recent report for the AWS cluster.

For the test cases we do not want to run in AWS&Smaug clusters, we should set up behave tags.

fridex commented

Failing tests

  • Scenario: Browsing logs of container images builds during Amun inspection jobs/Then wait for inspection to finish successfully
  • Dependency Monkey tests
  • Container image analysis
  • Query for metadata for different indices for TensorFlow
  • Querying for the known Python Package Indexes on User API - most probably requires update of the database
    • missing https://pulp.operate-first.cloud/pypi/gym-donkeycar/simple
    • missing https://pulp.operate-first.cloud/pypi/hello-world/simple
    • missing https://pulp.operate-first.cloud/pypi/test/simple
  • schedule solver job
  • advise scenarios failing with HTTP 400
  • provenance scenarios failing with HTTP 400
fridex commented

/priority critical-urgent

Gregory-Pereira commented

/assign

mayaCostantini commented

/assign

codificat commented

/triage accepted

fridex commented

Based on the discussion we had with @harshad16, we will point integration tests to prod instead of smaug so results will differ. Let's wait for the report sent.

fridex commented

Related action item to be done first: thoth-station/thoth-application#2451

Gregory-Pereira commented

once the documentation pr gets merged, this will be no longer be blocked by thoth-station/thoth-application#2451.

  • 👍1
codificat commented

/sig devsecops

codificat commented

actually this is
/sig stack-guidance
to confirm the status of the various failing scenarios

codificat commented

/remove-sig devsecops

sesheta commented

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

harshad16 commented

/remove-lifecycle stale
/lifecycle frozen

KPostOffice commented

Running tests and looking into failing ones in stage. Running tests takes multiple hours, so this is slow going.