ory/k8s

[CI] Test both migration patterns

Demonsthere opened this issue · 3 comments

Preflight checklist

Describe your problem

Currently we only test one set of values.yaml for applications. This causes us to test either initContainer or job automigration deployment patterns, and we can be (and have been) blind to changes that cause errors, like seen in #563. To fix that we need to create copies or layers of overrides file to test both cases

Describe your ideal solution

hacks/values/hydra.yaml -> Default overrides for the application
hacks/values/hydra-migration-job -> Subset of overrides to use specific migrations
hacks/values/hydra-migration-init -> Other subset

then test it using layers like `helm template hydra . -f values.yaml -f hydra-migration-job.yaml

Workarounds or alternatives

have a ton of files with minor differences or set overrides using --set?

Version

yes

Additional Context

No response

@Demonsthere You could use Helmfile with a values go template for this. That way you can have multiple setups to test without adding too much complexity.

Maybe we can work together on this to get a nice setup going?

Hello there :)
I was initially planning on using skaffold, to create multiple profiles and test them, but Helmfile is also a valid option.
The point would be to have a matrix CI definition, which runs multiple cases for each chart, and runs in parallel to each other.

Feel free to reach out to me in the opensource slack if you want to discuss it in real time or a call.
Cheers!