Extend TektonConfig with spec.config.affinity inputs

[jsteenb2]

Feature request

We're actively deploying into restricted egress subnets. One thing we have to do now is update the tekton controller deployment resources with the affinity updates. What we want to do is add it into the tekton config alongside our tolerations. This would keep it concise, and remove a lot of burden of having to download and change the release.yaml resources. Looking at affinity assistant, and it doesn't look like it'll solve our needs. The following is our desired output:

apiVersion: operator.tekton.dev/v1alpha1
kind: TektonConfig
metadata:
  name: config
spec:
  targetNamespace: tekton-pipelines
  profile: all
  config:
    affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: kubernetes.io/os
                    operator: NotIn
                    values:
                      - windows
                  - key: node-role.kubernetes.io/restricted-egress
                    operator: In
                    values:
                      - ''
    tolerations:
      - effect: NoSchedule
        key: node-role.kubernetes.io/restricted-egress
        operator: Exists
    priorityClassName: system-cluster-critical

Use case

We're usign tekton for a build pipeline, and we need to run that pipeline in a restricted egress subnet. Atm, it requires a lot of updates to the raw yaml releases, which is burdensome and error prone.

[jkandasa]

Hi @jsteenb2 thanks to reach us. from v0.68.0 we have add new field called options, with options you can achieve this. https://tekton.dev/docs/operator/tektonconfig/#additional-fields-as-options

apiVersion: operator.tekton.dev/v1alpha1
kind: TektonConfig
metadata:
  name: config
spec:
  pipeline:
    options:
      deployments:
        tekton-pipelines-controller:
          spec:
            template:
              spec:
                affinity:
                  nodeAffinity:
                    requiredDuringSchedulingIgnoredDuringExecution:
                      nodeSelectorTerms:
                        - matchExpressions:
                          - key: kubernetes.io/os
                            operator: NotIn
                            values:
                              - windows
                          - key: node-role.kubernetes.io/restricted-egress
                            operator: In
                            values:
                              - ''
                nodeSelector:
                tolerations:
                  - effect: NoSchedule
                    key: node-role.kubernetes.io/restricted-egress
                    operator: Exists
                topologySpreadConstraints:

However the priorityClassName NOT in the options. I will include that. thanks!

[jsteenb2]

ohhh interesting, thank you! I'll give this a shot today 👍

[jsteenb2]

would be great to see an example of what the pod_template woudl look like integrating into this. The options field was opaque in its structure to me. I never would have guessed I could have done what you outlined above. Extending the docs here would be immensely helpful 🙏

[jsteenb2]

@jkandasa , is there a way to set the affinity and tolerations for all deployments? Or do we have to list them all out separately in the options? I'm using yaml references, but it feels suboptimal having to map this to each individually.

[jkandasa]

@jsteenb2 you have to add the entry to each deployment separately in the options.

[jsteenb2]

gotcha, that's less than ideal, but workable on our end