s5cmd ls --region argument, and implied lists respect the cp argument
jordan-jack-schneider opened this issue · 0 comments
If you're doing a copy from one region to another with --no-clobber
there are a lot of failure modes with region resolution. The resolution order of ls is
Region detection
While executing the commands, s5cmd detects the region according to the following order of priority:
--source-region or --destination-region flags of cp command.
AWS_REGION environment variable.
Region section of AWS profile.
Auto detection from bucket region (via HeadBucket API call).
us-east-1 as default region.
So this is actually impossible if the region section of your aws profile is set (which you might want as a default). Without the config, it relied on HeadBucket calls, which can sometimes cause performance problems when making a lot of copies with s5cmd run
.
Ideally, s5cmd would take in a --region
argument, and s5cmd cp $SRC $DST --destination-region $REGION --no-clobber
would pass the --destination-region
flag to the internal list needed by --no-clobber
.