thoth-station/integration-tests

Schema validation failed for elyra-aidevsecops-tutorial test

fridex opened this issue · 8 comments

fridex commented

Describe the bug

Traceback (most recent call last):
  File "/opt/app-root/lib64/python3.8/site-packages/behave/model.py", line 1329, in run
    match.run(runner.context)
  File "/opt/app-root/lib64/python3.8/site-packages/behave/matchers.py", line 98, in run
    self.func(context, *args, **kwargs)
  File "features/steps/advise.py", line 251, in step_impl
    results = advise_using_config(
  File "/opt/app-root/lib64/python3.8/site-packages/thamos/lib.py", line 399, in advise_using_config
    return advise(
  File "/opt/app-root/lib64/python3.8/site-packages/thamos/lib.py", line 118, in wrapper
    result = func(api_client, *args, **kwargs)
  File "/opt/app-root/lib64/python3.8/site-packages/thamos/lib.py", line 591, in advise
    _wait_for_analysis(
  File "/opt/app-root/lib64/python3.8/site-packages/thamos/lib.py", line 211, in _wait_for_analysis
    response = status_func(analysis_id, verify_tls=verify_tls)
  File "/opt/app-root/lib64/python3.8/site-packages/thamos/lib.py", line 143, in is_analysis_ready
    raise ApiError(
thamos.exceptions.ApiError: Thoth Backend didn't respond with correct status code. Returned code - 500: {
  "detail": "[{'apiVersion': 'apps.openshift.io/v1', 'kind': 'DeploymentConfig', 'patch': {'op': 'add', 'path': '/spec/template/spec/containers/0/env/0', 'value': {'name': 'TF_ENABLE_ONEDNN_OPTS', 'value': '1'}}}] is not of type 'object'\n\nFailed validating 'type' in schema['properties']['result']['properties']['report']['properties']['products']['items']['properties']['advised_manifest_changes']['items']:\n    {'description': 'A change that was advised', 'type': 'object'}\n\nOn instance['result']['report']['products'][0]['advised_manifest_changes'][0]:\n    [{'apiVersion': 'apps.openshift.io/v1',\n      'kind': 'DeploymentConfig',\n      'patch': {'op': 'add',\n                'path': '/spec/template/spec/containers/0/env/0',\n                'value': {'name': 'TF_ENABLE_ONEDNN_OPTS', 'value': '1'}}}]",
  "status": 500,
  "title": "Response body does not conform to specification",
  "type": "about:blank"
}


Captured stdout:
{'host': 'stage.thoth-station.ninja', 'tls_verify': True, 'requirements_format': 'pipenv', 'overlays_dir': 'overlays', 'runtime_environments': [{'name': 'experiment', 'operating_system': {'name': 'ubi', 'version': '8'}, 'python_version': '3.8', 'recommendation_type': 'latest'}, {'name': 'download-dataset', 'operating_system': {'name': 'ubi', 'version': '8'}, 'python_version': '3.8', 'recommendation_type': 'latest'}, {'name': 'training', 'operating_system': {'name': 'ubi', 'version': '8'}, 'python_version': '3.8', 'recommendation_type': 'performance'}, {'name': 'inference', 'operating_system': {'name': 'ubi', 'version': '8'}, 'python_version': '3.8', 'recommendation_type': 'security'}, {'name': 'test-model', 'operating_system': {'name': 'ubi', 'version': '8'}, 'python_version': '3.8', 'recommendation_type': 'latest'}, {'name': 'nm-download', 'operating_system': {'name': 'ubi', 'version': '8'}, 'python_version': '3.8', 'recommendation_type': 'latest'}, {'name': 'pytorch-training', 'operating_system': {'name': 'ubi', 'version': '8'}, 'python_version': '3.8', 'recommendation_type': 'performance'}, {'name': 'nm-convert-onnx', 'operating_system': {'name': 'ubi', 'version': '8'}, 'python_version': '3.8', 'recommendation_type': 'latest'}, {'name': 'nm-sparseml', 'operating_system': {'name': 'ubi', 'version': '8'}, 'python_version': '3.8', 'recommendation_type': 'performance'}, {'name': 'nm-inference', 'operating_system': {'name': 'ubi', 'version': '8'}, 'python_version': '3.8', 'recommendation_type': 'performance'}, {'name': 'pytorch-inference', 'operating_system': {'name': 'ubi', 'version': '8'}, 'python_version': '3.8', 'recommendation_type': 'performance'}], 'managers': [{'name': 'thoth-advise', 'configuration': {'labels': ['bot']}}, {'name': 'info'}, {'name': 'version', 'configuration': {'maintainers': ['pacospace', 'sesheta', 'goern', 'fridex'], 'assignees': ['sesheta'], 'labels': ['bot'], 'changelog_file': True, 'changelog_smart': True}}]}

Captured logging:
INFO:thamos.lib:Using 'latest' recommendation type - see https://thoth-station.ninja/recommendation-types/
WARNING:thamos.lib:The user stack found in the lock file will not be supplied as requested
INFO:thamos.lib:Successfully submitted advise analysis 'adviser-220228195540-df53be0efe555398' to 'https://stage.thoth-station.ninja/api/v1'
ERROR:thamos.lib:Failed to obtain status from Thoth: Thoth Backend didn't respond with correct status code. Returned code - 500: {
  "detail": "[{'apiVersion': 'apps.openshift.io/v1', 'kind': 'DeploymentConfig', 'patch': {'op': 'add', 'path': '/spec/template/spec/containers/0/env/0', 'value': {'name': 'TF_ENABLE_ONEDNN_OPTS', 'value': '1'}}}] is not of type 'object'\n\nFailed validating 'type' in schema['properties']['result']['properties']['report']['properties']['products']['items']['properties']['advised_manifest_changes']['items']:\n    {'description': 'A change that was advised', 'type': 'object'}\n\nOn instance['result']['report']['products'][0]['advised_manifest_changes'][0]:\n    [{'apiVersion': 'apps.openshift.io/v1',\n      'kind': 'DeploymentConfig',\n      'patch': {'op': 'add',\n                'path': '/spec/template/spec/containers/0/env/0',\n                'value': {'name': 'TF_ENABLE_ONEDNN_OPTS', 'value': '1'}}}]",
  "status": 500,
  "title": "Response body does not conform to specification",
  "type": "about:blank"
}

WARNING:thamos.lib:Retrying in a few moments... (attempt 1/3)
ERROR:thamos.lib:Failed to obtain status from Thoth: Thoth Backend didn't respond with correct status code. Returned code - 500: {
  "detail": "[{'apiVersion': 'apps.openshift.io/v1', 'kind': 'DeploymentConfig', 'patch': {'op': 'add', 'path': '/spec/template/spec/containers/0/env/0', 'value': {'name': 'TF_ENABLE_ONEDNN_OPTS', 'value': '1'}}}] is not of type 'object'\n\nFailed validating 'type' in schema['properties']['result']['properties']['report']['properties']['products']['items']['properties']['advised_manifest_changes']['items']:\n    {'description': 'A change that was advised', 'type': 'object'}\n\nOn instance['result']['report']['products'][0]['advised_manifest_changes'][0]:\n    [{'apiVersion': 'apps.openshift.io/v1',\n      'kind': 'DeploymentConfig',\n      'patch': {'op': 'add',\n                'path': '/spec/template/spec/containers/0/env/0',\n                'value': {'name': 'TF_ENABLE_ONEDNN_OPTS', 'value': '1'}}}]",
  "status": 500,
  "title": "Response body does not conform to specification",
  "type": "about:blank"
}

WARNING:thamos.lib:Retrying in a few moments... (attempt 2/3)
ERROR:thamos.lib:Failed to obtain status from Thoth: Thoth Backend didn't respond with correct status code. Returned code - 500: {
  "detail": "[{'apiVersion': 'apps.openshift.io/v1', 'kind': 'DeploymentConfig', 'patch': {'op': 'add', 'path': '/spec/template/spec/containers/0/env/0', 'value': {'name': 'TF_ENABLE_ONEDNN_OPTS', 'value': '1'}}}] is not of type 'object'\n\nFailed validating 'type' in schema['properties']['result']['properties']['report']['properties']['products']['items']['properties']['advised_manifest_changes']['items']:\n    {'description': 'A change that was advised', 'type': 'object'}\n\nOn instance['result']['report']['products'][0]['advised_manifest_changes'][0]:\n    [{'apiVersion': 'apps.openshift.io/v1',\n      'kind': 'DeploymentConfig',\n      'patch': {'op': 'add',\n                'path': '/spec/template/spec/containers/0/env/0',\n                'value': {'name': 'TF_ENABLE_ONEDNN_OPTS', 'value': '1'}}}]",
  "status": 500,
  "title": "Response body does not conform to specification",
  "type": "about:blank"
}

To Reproduce
Steps to reproduce the behavior:

  1. See integration test report Integration tests update for ocp4-stage (2022-02-28 version 0.9.2)
fridex commented

/sig user-experience
/priority critical-urgent

mayaCostantini commented

This fix should normally solve the OpenAPI schema issue. It was integrated in the v0.34.7 of user-api, and stage already uses the v0.34.8. Should we still wait to see if the change is propagated?

fridex commented

I've just redeployed user-api in stage environment. It reports:

2022-03-01 11:04:27,146  23 INFO     thoth.user_api:64: This is User API v0.34.8+messaging.0.16.0.storages.0.70.0.common.0.35.0.python.0.16.8

Maybe worth checking if the scenario is still failing.

  • 👍1
fridex commented

BTW the last report from 12:06 PM CET has still this issue. Other test scenarios report this as well.

mayaCostantini commented

/assign

mayaCostantini commented

/lifecycle active

mayaCostantini commented

/close
The last integration tests report (Integration tests update for ocp4-stage (2022-03-01 version 0.9.2) at 1:03PM CEST) shows that the issue is solved for elyra-aidevsecops-tutorial.

sesheta commented

@mayaCostantini: Closing this issue.

In response to this:

/close
The last integration tests report (Integration tests update for ocp4-stage (2022-03-01 version 0.9.2) at 1:03PM CEST) shows that the issue is solved for elyra-aidevsecops-tutorial.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.