giantswarm/aws-operator

Access denied errors in k8s-controller-manager log (Incorrect iam access policies)

r7vme opened this issue · 2 comments

r7vme commented

I found lots of errors like this in pretty fresh guest cluster in AWS.

    1 service_controller.go:749] Failed to process service. Retrying in 5m0s: Error getting LB for service sam/xxx-sizes-service-develop: AccessDenied: User: arn:aws:sts::xxx:assumed-
, request id: 8d48abfb-cdc9-11e7-bd7e-f11d0092dd77 
    1 service_controller.go:749] Failed to process service. Retrying in 5m0s: Error getting LB for service sam/xxx-article-service-develop: AccessDenied: User: arn:aws:sts::xxx:assume
, request id: 8d49965c-cdc9-11e7-bd7e-f11d0092dd77
    1 service_controller.go:749] Failed to process service. Retrying in 5m0s: Error getting LB for service default/kubernetes: AccessDenied: User: arn:aws:sts::xxx:assumed-role/uonr0-mas
, request id: 8d4a329e-cdc9-11e7-bd7e-f11d0092dd77
    1 service_controller.go:749] Failed to process service. Retrying in 5m0s: Error getting LB for service kube-system/kube-dns: AccessDenied: User: arn:aws:sts::xxx:assumed-role/uonr0-m
, request id: 8d4af5f0-cdc9-11e7-bd7e-f11d0092dd77
    1 service_controller.go:749] Failed to process service. Retrying in 5m0s: Error getting LB for service kube-system/default-http-backend: AccessDenied: User: arn:aws:sts::xxx:assumed-
, request id: 8d4be051-cdc9-11e7-bd7e-f11d0092dd77 
    1 service_controller.go:749] Failed to process service. Retrying in 5m0s: Error getting LB for service kube-system/nginx-ingress-controller: AccessDenied: User: arn:aws:sts::xxx:assu
, request id: 8d4ccab2-cdc9-11e7-bd7e-f11d0092dd77
    1 service_controller.go:749] Failed to process service. Retrying in 5m0s: Error getting LB for service sam/xxx-replenishment-backend-develop: AccessDenied: User: arn:aws:sts::xxx:
, request id: 8d4db513-cdc9-11e7-bd7e-f11d0092dd77
    1 service_controller.go:749] Failed to process service. Retrying in 5m0s: Error getting LB for service sam/xxx-device-service-sam-6-provide-a-local-dev-environment: AccessDenied: User: arn
, request id: 8d4e9f74-cdc9-11e7-bd7e-f11d0092dd77
puja108 commented

This is related to a bug upstream, where services that are not of type loadbalancer still trigger a read action on ELBs. I reported this (or rather added to an issue) but there was no prio on this. With recent changed in our clusters where we allow all ELB actions in our policies (we enabled services of type loadbalancer) we shouldn't see this anymore.

@fgimenez was doing the changes last week

I would check with the updated clusters and then close this.

  • 👍1
r7vme commented

Great ,please close if not actual.