All Nginx ingress paths are included in each canary ingress

[sepehrjavid]

Hi all,
I have an issue when using multiple paths under the same hostname in nginx ingress.

How to create the issue:

app1.yaml

apiVersion: argoproj.io/v1alpha1  # Changed from apps/v1
kind: Rollout                     # Changed from Deployment
metadata:
  labels:
    app: app1
  name: app1
  namespace: default
spec:
  replicas: 2
  selector:
    matchLabels:
      app: app1
  template:
    metadata:
      labels:
        app: app1
    spec:
      containers:
        image: myimage:latest
        name: app1
        ports:
        - containerPort: 8080
          protocol: TCP
  # Rollout specific conf
  revisionHistoryLimit: 5
  strategy:
    canary:
      stableService: app1-stable
      canaryService: app1-canary
      trafficRouting:
        nginx:
          # Either stableIngress or stableIngresses must be configured, but not both.
          stableIngress: apps-stable
          # annotationPrefix: nginx.ingress.kubernetes.io # optional
          additionalIngressAnnotations:
            canary-by-header: X-Canary
            canary-by-header-value: "true"
      # immediately before the promotion
      stableMetadata:
        labels:
          role: active
      canaryMetadata:
        labels:
          role: preview
      steps:
      - setCanaryScale:
          replicas: 1  
      - pause: {}
      - setWeight: 20
      - pause: {}
      - setWeight: 80
      - pause: {}
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: app1
  name: app1-stable
  namespace: default
spec:
  ports:
  - name: web
    port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: app1
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: app1
  name: app1-canary
  namespace: default
spec:
  ports:
  - name: web
    port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: app1

app1.yaml

apiVersion: argoproj.io/v1alpha1  # Changed from apps/v1
kind: Rollout                     # Changed from Deployment
metadata:
  labels:
    app: app2
  name: app2
  namespace: default
spec:
  replicas: 2
  selector:
    matchLabels:
      app: app2
  template:
    metadata:
      labels:
        app: app2
    spec:
      containers:
        image: myimage:latest
        name: app2
        ports:
        - containerPort: 8080
          protocol: TCP
  # Rollout specific conf
  revisionHistoryLimit: 5
  strategy:
    canary:
      stableService: app2-stable
      canaryService: app2-canary
      trafficRouting:
        nginx:
          # Either stableIngress or stableIngresses must be configured, but not both.
          stableIngress: apps-stable
          # annotationPrefix: nginx.ingress.kubernetes.io # optional
          additionalIngressAnnotations:
            canary-by-header: X-Canary
            canary-by-header-value: "true"
      # immediately before the promotion
      stableMetadata:
        labels:
          role: active
      canaryMetadata:
        labels:
          role: preview
      steps:
      - setCanaryScale:
          replicas: 1  
      - pause: {}
      - setWeight: 20
      - pause: {}
      - setWeight: 80
      - pause: {}
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: app2
  name: app2-stable
  namespace: default
spec:
  ports:
  - name: web
    port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: app2
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: app2
  name: app2-canary
  namespace: default
spec:
  ports:
  - name: web
    port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: app2

apps-stable.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/rewrite-target: /$1/$3
  name: apps-stable
spec:
  rules:
  - host: bug.focalpay.app
    http:
      paths:
      - backend:
          service:
            name: app1-stable
            port: 
              number: 8080
        path: /(app1)(/|$)(.*)
        pathType: ImplementationSpecific
      - backend:
          service:
            name: app2-stable
            port: 
              number: 8080
        path: /(app2)(/|$)(.*)
        pathType: ImplementationSpecific

In this scenario, argo rollout creates two ingresses containing both paths in both ingresses:

app1-apps-stable-canary:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: X-Canary
    nginx.ingress.kubernetes.io/canary-by-header-value: "true"
    nginx.ingress.kubernetes.io/canary-weight: "0"
  name: app1-apps-stable-canary
spec:
  rules:
  - host: bug.focalpay.app
    http:
      paths:
      - backend:
          service:
            name: app1-canary
            port: 
              number: 8080
        path: /(app1)(/|$)(.*)
        pathType: ImplementationSpecific
      - backend:
          service:
            name: app2-stable
            port: 
              number: 8080
        path: /(app2)(/|$)(.*)
        pathType: ImplementationSpecific

app2-apps-stable-canary:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: X-Canary
    nginx.ingress.kubernetes.io/canary-by-header-value: "true"
    nginx.ingress.kubernetes.io/canary-weight: "0"
  name: app2-apps-stable-canary
spec:
  rules:
  - host: bug.focalpay.app
    http:
      paths:
      - backend:
          service:
            name: app1-stable
            port: 
              number: 8080
        path: /(app1)(/|$)(.*)
        pathType: ImplementationSpecific
      - backend:
          service:
            name: app2-canary
            port: 
              number: 8080
        path: /(app2)(/|$)(.*)
        pathType: ImplementationSpecific

My expected canary ingress config is:

app1-apps-stable-canary:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: X-Canary
    nginx.ingress.kubernetes.io/canary-by-header-value: "true"
    nginx.ingress.kubernetes.io/canary-weight: "0"
  name: app1-apps-stable-canary
spec:
  rules:
  - host: bug.focalpay.app
    http:
      paths:
      - backend:
          service:
            name: app1-canary
            port: 
              number: 8080
        path: /(app1)(/|$)(.*)
        pathType: ImplementationSpecific

app2-apps-stable-canary:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: X-Canary
    nginx.ingress.kubernetes.io/canary-by-header-value: "true"
    nginx.ingress.kubernetes.io/canary-weight: "0"
  name: app2-apps-stable-canary
spec:
  rules:
  - host: bug.focalpay.app
    http:
      paths:
      - backend:
          service:
            name: app2-canary
            port: 
              number: 8080
        path: /(app2)(/|$)(.*)
        pathType: ImplementationSpecific

Can someone please explain if I'm missing something or if at all is possible to get my expected canary ingress?
Thank you in advance!

[sepehrjavid]

Update: There was no issue and there was a misunderstanding on my side.