Add shortcut to support `chaos verify`command

[dmartin35]

From now on, the chaos experiment template only support the chaos run command.

We can override this command, but it requires to pass the entire pod template, which is a bit complex just to verify instead of run.

I suggest we add at chaos experiment spec level an option to easily switch to verify mode runAsVerification: true

---
apiVersion: chaostoolkit.org/v1
kind: ChaosToolkitExperiment
metadata:
  name: my-chaos-verif
  namespace: chaostoolkit-crd
spec:
  namespace: chaostoolkit-run
  pod:
    image:
      name: chaosiq/chaostoolkit
    runAsVerification: true

[Lawouach]

Another approach is to rely on a different container image with a different entrypoint?

[dmartin35]

we already use another image different from the open source.
Our new image contains both the CTK and the ChaosIQ cloud plugin.

But even for this image, we could end up with users wanting to run both experiments and verifications.

• Do we want to force them to use another image ? or at minimal another tag ?
• Do we provide them an option to easily choose between run experiment and run verification as verification will still be loaded as either experiment path or experiment url

wdyt?

[dmartin35]

Btw i was already thinking of making two images/tags

• first, with CTK + chaosiq-cloud extension
• second, with CTK + a bunch/several extensions (maybe full list?)

It feels to me that creating another image shall be done when embedding different extensions,
rather than choosing for the command to run within that image

[Lawouach]

Maybe we don't specify the command in the entry point indeed then. So we could have the base image as follows:

ENTRYPOINT ["chaos"]

Then, rather than the runAsVerification we could have command: verify or command: run

That would substitute the command parameter of the pod declaration?

[dmartin35]

our current Dockerfile is only returning help

ENTRYPOINT ["/usr/local/bin/chaos"]
CMD ["--help"]

In the POD default template, the command is more complete with

        command:
        - "/bin/sh" 
        args:
        - "-c"
        - "/usr/local/bin/chaos run ${EXPERIMENT_PATH-$EXPERIMENT_URL} && exit $?"

But we can add the support for a command at ChaosExperiment resource level, that will be inserted on the POD command line arguments. is that what you had in mind ? or did you want to use that command as a full command replacement for the pod one ?

my initial intend was to be able to either via option or env variable to override run by verify

• "/usr/local/bin/chaos run|verify ${EXPERIMENT_PATH-$EXPERIMENT_URL} && exit $?"

We want to keep this internal stuff hidden as much as possible, to have clean and light CRO definition

[dmartin35]

For verification use case, do we keep the current CRO ChaosToolkitExperiment, and all experiment related wordings ?

or do we want to create another CRO ChaosToolkitVerification that will duplicate all concepts, but hides the complexity of internal commands ?
Our web interface and apis reflects both terms experiment/verifications, but the CRD does not (yet?)
is it not to overkill ? (as a verification only exists for our CIQ plugin)
is it another CRD ?
is it another kustomization ?

[Lawouach]

Remember that verification is not a chaostoolkit concept but a chaosiq one. So, this CRD must not make too much effort specifically on verification. It must be extensible to other usage of course.