aboisvert/s3cp

S3 Endpoints

rjocoleman opened this issue · 2 comments

aboisvert commented

Make sense ... but slightly different than the endpoint configuration.

What I'm thinking is the following,

  1. Use S3CP_ENDPOINT if set, "us-east-1" as default.
  2. Check bucket location_constraint (this will use the endpoint from #1).
    If location is restricted, switch to the necessary region. Otherwise,
    continue with endpoint set in #1.

Does that fit what you had in mind?

On Sun, May 19, 2013 at 12:59 AM, Robert Coleman
notifications@github.comwrote:

I spotted this:
https://github.com/aboisvert/s3cp/blob/master/lib/s3cp/utils.rb#L47

FYI:
aws/s3 has got a method to get this programatically:
https://github.com/aws/aws-sdk-ruby/blob/master/lib/aws/s3/bucket.rb#L257

It maps to this:
http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region
http://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGETlocation.html

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/12
.

rjocoleman commented
  • As I read it S3CP_ENDPOINT is a total override, it specifies the whole endpoint in Amazons jargon, and S3CP uses that if set..
    With bucket location/LocationConstraint support I can't imagine a use case for this but that could be me being naive 😊 I expect this is for power users with very specific requirements. If this ENV variable is set don't move on to the next step.
  • I expect the bucket location API call should serve most non us-east-1 users by automatically determining their endpoint and enable s3cp to work without configuration past the key, id and bucket.
    You don't need to query a specific endpoint for this, just the generic s3 API host = BucketName.s3.amazonaws.com
    I'd be hitting this via aws/s3 and building the endpoint from it's result. As I see it EU and US Standard are the outliers - the rest of the endpoints can be generated without much manipulation.