chaostoolkit-incubator/kubernetes-crd

CRD cannot create experiment pod

dmartin35 opened this issue · 1 comments

With the following CTK experiment CRO:

---
apiVersion: v1
kind: Secret
metadata:
  name: chaostoolkit-settings
  namespace: chaostoolkit-run
data:
  settings.yaml: *********
---
apiVersion: chaostoolkit.org/v1
kind: ChaosToolkitExperiment
metadata:
  name: my-chaos-exp
  namespace: chaostoolkit-crd
spec:
  namespace: chaostoolkit-run
  pod:
    image:
      name: chaosiq/chaostoolkit
    chaosArgs:
      - --verbose
      - run
      - https://console.chaosiq.dev/assets/experiments/71340a18-96fd-478a-bfdd-2f454bf429e1.json

The CRD is not able to create PODs for running the experiment, see CRD logs


[2020-03-23 13:31:38,521] kopf.objects         [INFO    ] Namespace 'chaostoolkit-run' already exists. Let's continue...
[2020-03-23 13:31:38,526] kopf.objects         [INFO    ] [chaostoolkit-crd/my-chaos-exp] chaostoolkit resources will be created in namespace 'chaostoolkit-run'
[2020-03-23 13:31:38,526] kopf.objects         [INFO    ] [chaostoolkit-crd/my-chaos-exp] Suffix for resource names will be '-ltva8'
[2020-03-23 13:31:38,570] kopf.objects         [INFO    ] [chaostoolkit-crd/my-chaos-exp] Created service account
[2020-03-23 13:31:38,672] kopf.objects         [INFO    ] [chaostoolkit-crd/my-chaos-exp] Created role
[2020-03-23 13:31:38,717] kopf.objects         [INFO    ] [chaostoolkit-crd/my-chaos-exp] Created rolebinding
[2020-03-23 13:31:38,870] kopf.objects         [INFO    ] Env config map named 'chaostoolkit-env'
[2020-03-23 13:31:38,874] kopf.objects         [INFO    ] Settings secret volume named 'chaostoolkit-settings'
[2020-03-23 13:31:38,875] kopf.objects         [INFO    ] Experiment config map named 'chaostoolkit-experiment'
[2020-03-23 13:31:38,875] kopf.objects         [INFO    ] Override default chaos command arguments: $ chaos --verbose run https://console.chaosiq.dev/assets/experiments/71340a18-96fd-478a-bfdd-2f454bf429e1.json
[2020-03-23 13:31:38,879] kopf.objects         [ERROR   ] [chaostoolkit-crd/my-chaos-exp] Handler 'create_chaos_experiment' failed with an exception. Will retry.
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/kopf/reactor/handling.py", line 291, in execute_handler_once
    lifecycle=lifecycle,  # just a default for the sub-handlers, not used directly.
  File "/usr/local/lib/python3.7/site-packages/kopf/reactor/handling.py", line 380, in invoke_handler
    **kwargs,
  File "/usr/local/lib/python3.7/site-packages/kopf/reactor/invocation.py", line 117, in invoke
    result = await fn(*args, **kwargs)  # type: ignore
  File "controller.py", line 71, in create_chaos_experiment
    pod_tpl = await create_pod(v1, cm, spec, ns, name_suffix)
  File "controller.py", line 268, in run
    return await loop.run_in_executor(executor, pfunc)
  File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "controller.py", line 474, in create_pod
    pod = api.create_namespaced_pod(body=tpl, namespace=ns)
  File "/usr/local/lib/python3.7/site-packages/kubernetes/client/apis/core_v1_api.py", line 6115, in create_namespaced_pod
    (data) = self.create_namespaced_pod_with_http_info(namespace, body, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/kubernetes/client/apis/core_v1_api.py", line 6206, in create_namespaced_pod_with_http_info
    collection_formats=collection_formats)
  File "/usr/local/lib/python3.7/site-packages/kubernetes/client/api_client.py", line 344, in call_api
    _return_http_data_only, collection_formats, _preload_content, _request_timeout)
  File "/usr/local/lib/python3.7/site-packages/kubernetes/client/api_client.py", line 178, in __call_api
    _request_timeout=_request_timeout)
  File "/usr/local/lib/python3.7/site-packages/kubernetes/client/api_client.py", line 387, in request
    body=body)
  File "/usr/local/lib/python3.7/site-packages/kubernetes/client/rest.py", line 266, in POST
    body=body)
  File "/usr/local/lib/python3.7/site-packages/kubernetes/client/rest.py", line 222, in request
    raise ApiException(http_resp=r)
kubernetes.client.rest.ApiException: (400)
Reason: Bad Request
HTTP response headers: HTTPHeaderDict({'Audit-Id': '7ccf57ba-353b-44da-988f-b83e66ee5074', 'Content-Type': 'application/json', 'Date': 'Mon, 23 Mar 2020 13:31:38 GMT', 'Content-Length': '478'})
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Pod in version \"v1\" cannot be handled as a Pod: v1.Pod.Spec: v1.PodSpec.Containers: []v1.Container: v1.Container.Image: ReadString: expects \" or n, but found {, error found in #10 byte of ...|\"image\": {\"name\": \"c|..., bigger context ...|\"containers\": [{\"name\": \"chaostoolkit\", \"image\": {\"name\": \"chaosiq/chaostoolkit\"}, \"imagePullPolicy\"|...","reason":"BadRequest","code":400}

the CRO was using bad template structure:

spec:
  namespace: chaostoolkit-run
  pod:
    image:
      name: chaosiq/chaostoolkit

path spec.pod.image.name is not valid, and shall be updated with value on spec.pod.image directly

closing issue as it's a bug of usage, not a bug of the CRD itself