stripe/veneur

S3 plugin - bucket is not in region

ruilapa opened this issue · 0 comments

Created bucket in region us-west-2 "devops-veneur-test2" with inline policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::<<LOCAL_ACCOUNT_ID>>:root"
            },
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::devops-veneur-test2",
                "arn:aws:s3:::devops-veneur-test2/*"
            ]
        }
    ]
}

Created IAM user with inline policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": [
                "arn:aws:s3:::devops-veneur-test2/*",
                "arn:aws:s3:::devops-veneur-test2"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets",
                "s3:HeadBucket"
            ],
            "Resource": "*"
        }
    ]
}

Created Access Key / Secret Access Key for above user.

Configured veneur /root/.aws/credentials with above credentials
I am able to

aws s3 cp /etc/veneur.cfg s3://devops-veneur-test2/ --region us-west-2

Configured veneur.cfg with:

aws_access_key_id: "<<ACCESS_KEY>>"
aws_secret_access_key: "<<SECRET_ACCESS_KEY>>"
aws_region: "us-west-2"
aws_s3_bucket: "devops-veneur-test2"

Veneur logs:

time="2018-12-03T14:40:19Z" level=info msg="Set mutex profile fraction" MutexProfileFraction=0 previousMutexProfileFraction=0
time="2018-12-03T14:40:19Z" level=info msg="Set block profile rate (nanoseconds)" BlockProfileRate=0
time="2018-12-03T14:40:19Z" level=info msg="Preparing workers" number=32
time="2018-12-03T14:40:19Z" level=info msg="Successfully created AWS session"
time="2018-12-03T14:40:19Z" level=info msg="S3 archives are enabled"
time="2018-12-03T14:40:19Z" level=info msg="Starting server" version=cdcfb315b61057a0ee5c8bb3fd3b3bca0a69059b
time="2018-12-03T14:40:19Z" level=info msg="Starting span workers" n=0
time="2018-12-03T14:40:19Z" level=info msg="Starting span sink" sink=metric_extraction
time="2018-12-03T14:40:19Z" level=info msg="Listening on UDP address" address="0.0.0.0:8125" listeners=4 protocol=statsd
time="2018-12-03T14:40:19Z" level=info msg="Tracing sockets are not configured - not reading trace socket"
time="2018-12-03T14:40:19Z" level=info msg="Starting gRPC server" address="0.0.0.0:8128"
time="2018-12-03T14:40:19Z" level=info msg="Starting Event worker"
time="2018-12-03T14:40:19Z" level=info msg="HTTP server listening" address="0.0.0.0:8127"
time="2018-12-03T14:40:29Z" level=error msg="Error posting to s3" error="BucketRegionError: incorrect region, the bucket is not in 'us-west-2' region\n\tstatus code: 301, request id: , host id: " metrics=2
time="2018-12-03T14:40:39Z" level=error msg="Error posting to s3" error="BucketRegionError: incorrect region, the bucket is not in 'us-west-2' region\n\tstatus code: 301, request id: , host id: " metrics=37
time="2018-12-03T14:40:49Z" level=error msg="Error posting to s3" error="BucketRegionError: incorrect region, the bucket is not in 'us-west-2' region\n\tstatus code: 301, request id: , host id: " metrics=37