UnauthorizedOperation: You are not authorized to perform this operation

[nkhine]

I have setup a new k8s cluster and all works well, in that I can create pods, i have setup helm and installed mongo, cluster is working fine.

The issue I am having is that, when I try to create a service, i get
UnauthorizedOperation: You are not authorized to perform this operation

Here is the full error:

➜ tack git:(master) ✗ kubectl get svc                                                                                                                    (git)-[master] 
NAME           CLUSTER-IP   EXTERNAL-IP   PORT(S)        AGE
kubernetes     10.3.0.1     <none>        443/TCP        21m
test-service   10.3.0.81    <pending>     80:31902/TCP   4s                                                                                               
➜  tack git:(master) ✗ kubectl describe svc  test-service                                                                                                 (git)-[master] 
Name: test-service
Namespace: default
Labels: <none>
Annotations: service.beta.kubernetes.io/aws-load-balancer-backend-protocol=http
Selector: pod=app,track=feature
Type: LoadBalancer
IP: 10.3.0.81
Port: http 80/TCP
NodePort: http 31902/TCP
Endpoints: <none>
Session Affinity: None
Events:
  FirstSeen LastSeen Count From SubObjectPath Type Reason Message
  --------- -------- ----- ---- ------------- -------- ------ -------
  25s 25s 1 service-controller Warning CreatingLoadBalancerFailed Error creating load balancer (will retry): Failed to create load balancer for service default/test-service: UnauthorizedOperation: You are not authorized to perform this operation.
 status code: 403, request id: 6469cad8-a935-4723-ad8b-1a528f529a52
  20s 20s 1 service-controller Warning CreatingLoadBalancerFailed Error creating load balancer (will retry): Failed to create load balancer for service default/test-service: UnauthorizedOperation: You are not authorized to perform this operation.
 status code: 403, request id: d1e242e0-0138-4d29-899e-fea82059839b
  25s 10s 3 service-controller Normal CreatingLoadBalancer Creating load balancer
  10s 10s 1 service-controller Warning CreatingLoadBalancerFailed Error creating load balancer (will retry): Failed to create load balancer for service default/test-service: UnauthorizedOperation: You are not authorized to perform this operation.
 status code: 403, request id: a6150a36-f8b0-4474-bd8c-1d9ded529149

on CloudTrail there are errors, showing the user,

{
    "eventVersion": "1.05",
    "userIdentity": {
        "type": "AssumedRole",
        "principalId": "xxxxx:i-0dxxxx",
        "arn": "arn:aws:sts::xxxx:assumed-role/kz8s-master-test/i-0dxxx",
        "accountId": "xxxx",
        "accessKeyId": "xxxx",
        "sessionContext": {
            "attributes": {
                "mfaAuthenticated": "false",
                "creationDate": "2017-06-26T18:37:09Z"
            },
            "sessionIssuer": {
                "type": "Role",
                "principalId": "xxxxx",
                "arn": "arn:aws:iam::xxxxx:role/kz8s-master-test",
                "accountId": "xxxxxx",
                "userName": "kz8s-master-test"
            }
        }
    },
    "eventTime": "2017-06-26T18:57:26Z",
    "eventSource": "ec2.amazonaws.com",
    "eventName": "CreateSecurityGroup",
    "awsRegion": "us-west-2",
    "sourceIPAddress": "xx",
    "userAgent": "aws-sdk-go/1.6.10 (go1.7.5; linux; amd64)",
    "errorCode": "Client.UnauthorizedOperation",
    "errorMessage": "You are not authorized to perform this operation.",
    "requestParameters": {
        "groupName": "k8s-elb-a96f417",
        "groupDescription": "Security group for Kubernetes ELB a96f417 (default/trint-app-feature)",
        "vpcId": "vpc-7xxxx"
    },
    "responseElements": null,
    "requestID": "f9e68a7c-a06e-4011-9e58-b79d4dc0b93b",
    "eventID": "2d420c73-7966-4109-82de-fd8ee684e567",
    "eventType": "AwsApiCall",
}

the policy is as follows:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "s3:Get*"
      ],
      "Effect": "Allow",
      "Resource": [ "arn:aws:s3:::kz8s-pki-test-1111-us-west-2/*" ]
    },
    {
      "Action": [
        "ec2:AttachVolume",
        "ec2:CreateTags",
        "ec2:CreateVolume",
        "ec2:DeleteVolume",
        "ec2:Describe*",
        "ec2:DescribeInstances",
        "ec2:DetachVolume",
        "elasticloadbalancing:*"
        ],
      "Effect": "Allow",
      "Resource": [ "*" ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "ecr:GetAuthorizationToken",
        "ecr:BatchCheckLayerAvailability",
        "ecr:GetDownloadUrlForLayer",
        "ecr:GetRepositoryPolicy",
        "ecr:DescribeRepositories",
        "ecr:ListImages",
        "ecr:BatchGetImage"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "autoscaling:DescribeAutoScalingGroups",
        "autoscaling:DescribeAutoScalingInstances",
        "autoscaling:SetDesiredCapacity",
        "autoscaling:TerminateInstanceInAutoScalingGroup"
      ],
      "Resource": "*"
    }
  ]
}

what am i missing?

[tomfotherby]

This PR will fix your issue: #184 , I'm not sure why it's not being merged because it's a critical fix.

[nkhine]

@tomfotherby thanks this worked perfectly.