Possible to ignore only specific integration?
0x0013 opened this issue · 5 comments
Hi!
I know it is possible to annotate IngressRoute resource to make switchboard ignore it:
metadata:
annoations:
switchboard.borchero.com/ignore: "true"
But is it possible to only ignore a specific integration? Or only select specific integration(s) for a resource?
Example use case
We have several sites sharing the same main domain, for example:
- www.example.com
- api.example.com
- xyz.example.com
And we are using cert-manager managed wildcard certificate *.example.com. So the certificate resource is managed separately, and when creating an IngressRoute we would specify the appropriate secret, for example, star-example-com-tls. Because of this, we would not want switchboard to attempt to manage this and create a new certificate resource using star-example-com-tls secret, and if it did, cert-manager would probably fail operating on the resultant certificate resource, because the target secretName resource would already exist, tied to a different certificate resource.
At the same time, we would still like the external-dns resources to be created dynamically.
Of course, the whole cert-manager integration can be disabled, but there still might be other IngressRoute resources which are not covered by the wildcard certificate, and for which switchboard could manage the certificate resources. I would imagine that more similar needs would arise if in the future switchboard would add more possible integrations.
Is this something that you require right now or is it only something that might be useful in the future?
In general, I think that it should be discussed whether you need to "whitelist" or "blacklist" resources (i.e. actively include or exclude them from processing). In any case, I would think about introducing an annotation like <integration>.switchboard.borchero.com/enabled.
Yes, this is something we would use now. Although if it is not feasible in the short term, I will try to think/use some workaround in clusters where we require it - for example, not enabling the cert-manager integration, and falling back to Ingress instead of IngressRoute where it is required.
As for whitelist vs blacklist - my initial reaction would be to use a "disable" annotation and enable the integration by default. The reasoning being that if a user has chosen to globally enable the integration, they probably want to use the integration most times. But I am not convinced if that is always the best approach, especially for integrations like cert-manager where several resources (ingress/ingressRoute) might share a certificate.
Indeed, when cert-manager manages certificates for vanilla ingress resources, it requires one of the annotations to be present and will not pick them up by default, and the annotations also allowing more fine-grained configuration with them such as selecting the issuer.
However, external-dns, judging by example tutorials, seems to pick up ingresses by default, as long as the ingress source is configured and it fits the configured domain filter. For external-dns this might be the logical default because, unlike with cert-manager, your different distinct ingress resources will probably not share the same DNS name.
So maybe there is no clear-cut behavior that would be the best default for any (current and possibly future) switchboard integrations.
@0x0013 I feel like Switchboard should pick up IngressRoute resources by default as this simplifies the setup significantly.
On a different note, I had another idea how to solve this: what about limiting the annotation to switchboard.borchero.com/ignore and allow to specify a list of integrations (e.g. external-dns,cert-manager) or all to disable all integrations that are enabled in Switchboard?
I think that's a good idea, and a simple approach without introducing multiple annotations. Maybe the value to disable all integrations should stay true (or both true and all) to avoid breaking any existing resources if they already use it.
Though in the future you might still want to introduce additional annotations for more fine grained integration control (such as using different cert issuers), for simple ignoring of an integration, I think this is completely sufficient.
Finally implemented in v0.5.3 :)