cashapp/cmmc

CMMC fails merging `mapRoles` and `mapUsers` at the same time

bakayolo opened this issue · 4 comments

bakayolo commented

I have a config map with mapRoles and mapUsers define that I want CMMC to merge in the aws-auth configmap. Unfortunately, it only merge the mapUsers and ignore the mapRoles.

Here is my configmap CMMC will grab

apiVersion: v1
data:
  mapRoles: |
    - "groups":
      - "system:masters"
      "rolearn": "arn:aws:iam::261357321482:role/shuffle-labs-atlantis-dev-ecs_task_execution"
      "username": "atlantis"
  mapUsers: |
    - "groups":
      - "system:masters"
      "userarn": "arn:aws:iam::261357321482:user/ben"
      "username": "ben"
kind: ConfigMap
metadata:
  annotations:
    config.cmmc.k8s.cash.app/watched-by-merge-source: kube-system/aws-auth-map-users
  labels:
    cmmc.k8s.cash.app/merge: aws-auth-map
  name: aws-auth-mapping-cmmc
  namespace: kube-system

Here is the merged aws-auth configmap

apiVersion: v1
data:
  mapRoles: |
    - groups:
      - system:bootstrappers
      - system:nodes
      rolearn: arn:aws:iam::261357321482:role/shuffle-labs-private_eks-dev-eks-node
      username: system:node:{{EC2PrivateDNSName}}
  mapUsers: |
    - "groups":
      - "system:masters"
      "userarn": "arn:aws:iam::261357321482:user/ben"
      "username": "ben"
kind: ConfigMap
metadata:
  annotations:
    config.cmmc.k8s.cash.app/managed-by-merge-target: kube-system/kube-system-aws-auth
  name: aws-auth
  namespace: kube-system
bakayolo commented 
$ k get mergesource    ⎈ shuffle-labs-private_eks-dev/kube-system
NAME                 READY   STATUS
aws-auth-map-roles   True    Data from 1 ConfigMap(s) accumulated.
aws-auth-map-users   True    Data from 1 ConfigMap(s) accumulated.

$ k get mergetarget    ⎈ shuffle-labs-private_eks-dev/kube-system
NAME                   TARGET                 READY   STATUS                         VALIDATION
kube-system-aws-auth   kube-system/aws-auth   True    Target ConfigMap up to date.   2 MergeSources reporting valid data.
bakayolo commented

I can see an error in the logs but I think it's unrelated

2022-01-09T16:01:50.847Z	ERROR	controller-runtime.manager.controller.mergetarget	Reconciler error	{"reconciler group": "config.cmmc.k8s.cash.app", "reconciler kind": "MergeTarget", "name": "kube-system-aws-auth", "namespace": "kube-system", "error": "failed updating MergeTarget Status: failed updating initial status: Operation cannot be fulfilled on mergetargets.config.cmmc.k8s.cash.app \"kube-system-aws-auth\": the object has been modified; please apply your changes to the latest version and try again", "errorVerbose": "Operation cannot be fulfilled on mergetargets.config.cmmc.k8s.cash.app \"kube-system-aws-auth\": the object has been modified; please apply your changes to the latest version and try again\nfailed updating initial status\ngithub.com/cashapp/cmmc/controllers.(*MergeTargetReconciler).updateDataStatus\n\t/workspace/controllers/mergetarget_controller.go:176\ngithub.com/cashapp/cmmc/controllers.(*MergeTargetReconciler).reconcileMergeTarget\n\t/workspace/controllers/mergetarget_controller.go:131\ngithub.com/cashapp/cmmc/controllers.(*MergeTargetReconciler).Reconcile\n\t/workspace/controllers/mergetarget_controller.go:117\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:298\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:99\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1371\nfailed updating MergeTarget Status\ngithub.com/cashapp/cmmc/controllers.(*MergeTargetReconciler).reconcileMergeTarget\n\t/workspace/controllers/mergetarget_controller.go:132\ngithub.com/cashapp/cmmc/controllers.(*MergeTargetReconciler).Reconcile\n\t/workspace/controllers/mergetarget_controller.go:117\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:298\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:99\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1371\ngithub.com/cashapp/cmmc/controllers.(*MergeTargetReconciler).Reconcile\n\t/workspace/controllers/mergetarget_controller.go:118\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:298\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:99\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1371"}
github.com/go-logr/zapr.(*zapLogger).Error
	/go/pkg/mod/github.com/go-logr/zapr@v0.2.0/zapr.go:132
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:302
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:216
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1
	/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:185
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1
	/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:155
k8s.io/apimachinery/pkg/util/wait.BackoffUntil
	/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:156
k8s.io/apimachinery/pkg/util/wait.JitterUntil
	/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:133
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext
	/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:185
k8s.io/apimachinery/pkg/util/wait.UntilWithContext
	/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:99
bakayolo commented

Found it -> https://github.com/cashapp/cmmc/blob/main/controllers/mergesource_controller.go#L247-L255
It's actually by design :( CMMC does not support multiple mergesource

I am not sure why it would be a weird thing to double merge though. It would be incorrect but it would due to an misconfig unrelated to CMMC (duplicate merge source)

bakayolo commented

Fix here -> #9