403 on Sync
Closed this issue · 3 comments
Hi there! Big fan of this project. I've been having issues with syncing since I put in a bucket policy to whitelist my CDN's IP addresses. Since then, I've been able to sync locally but not using Travis (with identical environment variables). Very strange.
My bucket policy looks like this:
{
"Version": "2008-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Deny",
"Principal": {
"AWS": "*"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::ashfurrow.com/*",
"Condition": {
"NotIpAddress": {
"aws:SourceIp": [
"..."
]
}
}
},
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::243135559143:user/travis"
},
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::ashfurrow.com/*",
"arn:aws:s3:::ashfurrow.com"
]
}
]
}I added an additional statement based on #63 but it's still not working. I also have the following IAM policy attached to the travis user:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::ashfurrow.com",
"arn:aws:s3:::ashfurrow.com/*"
]
}
]
}Removing the statement banning all but certain IP addresses fixes the issue locally and on Travis. I'm using middleman-s3_sync 3.3.10, running bundle exec middleman s3_sync --bucket=ashfurrow.com and with the following config:
activate :s3_sync do |s3_sync|
s3_sync.bucket = 'ashfurrow.com'
s3_sync.aws_access_key_id = ENV['SITE_AWS_KEY']
s3_sync.aws_secret_access_key = ENV['SITE_AWS_SECRET']
endI am admittedly out of my element here, and I know that this relates to AWS config as much as this gem. I'd appreciate any advice you have to offer. If it's simply a matter of removing the whitelist of IP addresses, that's fine – at this point, I just really want to know why it's not working.
Here is the stack trace I'm getting:
bundle exec middleman s3_sync --bucket=ashfurrow.com
s3_sync Let's see if there's work to be done...
Processing sitemap |Time: 00:00:03 | ========================== | Time: 00:00:03
Processing remote files |Time: 00:00:00 | ===================== | Time: 00:00:00
/home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/excon-0.49.0/lib/excon/middlewares/expects.rb:6:in `response_call': Expected(200) <=> Actual(403 Forbidden) (Excon::Errors::Forbidden)
excon.error.response
:body => ""
:cookies => [
]
:headers => {
"Content-Type" => "application/xml"
"Date" => "Sat, 14 May 2016 15:22:44 GMT"
"Server" => "AmazonS3"
"Transfer-Encoding" => "chunked"
"x-amz-id-2" => "r7/jqVb5kNJVwV5sgqqMXK8h834LqWufSqqYQdkTtQ3Lo6fOkNFLv9SNq2YWIdhNB8zbdrIDmOI="
"x-amz-request-id" => "1F390364BB2ADADA"
}
:host => "s3.amazonaws.com"
:local_address => "172.17.0.154"
:local_port => 49369
:path => "/ashfurrow.com/blog/.DS_Store"
:port => 443
:reason_phrase => "Forbidden"
:remote_ip => "54.231.114.186"
:status => 403
:status_line => "HTTP/1.1 403 Forbidden\r\n"
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/excon-0.49.0/lib/excon/middlewares/response_parser.rb:8:in `response_call'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/excon-0.49.0/lib/excon/connection.rb:389:in `response'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/excon-0.49.0/lib/excon/connection.rb:253:in `request'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/excon-0.49.0/lib/excon/middlewares/idempotent.rb:26:in `error_call'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/excon-0.49.0/lib/excon/middlewares/base.rb:10:in `error_call'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/excon-0.49.0/lib/excon/middlewares/base.rb:10:in `error_call'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/excon-0.49.0/lib/excon/connection.rb:273:in `rescue in request'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/excon-0.49.0/lib/excon/connection.rb:221:in `request'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/excon-0.49.0/lib/excon/middlewares/idempotent.rb:26:in `error_call'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/excon-0.49.0/lib/excon/middlewares/base.rb:10:in `error_call'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/excon-0.49.0/lib/excon/middlewares/base.rb:10:in `error_call'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/excon-0.49.0/lib/excon/connection.rb:273:in `rescue in request'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/excon-0.49.0/lib/excon/connection.rb:221:in `request'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/excon-0.49.0/lib/excon/middlewares/idempotent.rb:26:in `error_call'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/excon-0.49.0/lib/excon/middlewares/base.rb:10:in `error_call'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/excon-0.49.0/lib/excon/middlewares/base.rb:10:in `error_call'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/excon-0.49.0/lib/excon/connection.rb:273:in `rescue in request'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/excon-0.49.0/lib/excon/connection.rb:221:in `request'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/fog-core-1.39.0/lib/fog/core/connection.rb:81:in `request'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/fog-xml-0.1.2/lib/fog/xml/connection.rb:9:in `request'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/fog-aws-0.9.2/lib/fog/aws/storage.rb:611:in `_request'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/fog-aws-0.9.2/lib/fog/aws/storage.rb:606:in `request'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/fog-aws-0.9.2/lib/fog/aws/requests/storage/head_object.rb:41:in `head_object'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/fog-aws-0.9.2/lib/fog/aws/models/storage/files.rb:101:in `head'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/middleman-s3_sync-3.3.10/lib/middleman/s3_sync/resource.rb:22:in `full_s3_resource'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/middleman-s3_sync-3.3.10/lib/middleman/s3_sync/resource.rb:177:in `redirect?'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/middleman-s3_sync-3.3.10/lib/middleman/s3_sync/resource.rb:159:in `status'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/middleman-s3_sync-3.3.10/lib/middleman/s3_sync.rb:176:in `tap'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/middleman-s3_sync-3.3.10/lib/middleman/s3_sync.rb:176:in `block in work_to_be_done?'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/parallel-1.6.1/lib/parallel.rb:410:in `call'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/parallel-1.6.1/lib/parallel.rb:410:in `call_with_index'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/parallel-1.6.1/lib/parallel.rb:288:in `block (2 levels) in work_in_threads'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/parallel-1.6.1/lib/parallel.rb:419:in `with_instrumentation'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/parallel-1.6.1/lib/parallel.rb:287:in `block in work_in_threads'
from /home/travis/build/ashfurrow/blog/vendor/bundle/ruby/2.1.0/gems/parallel-1.6.1/lib/parallel.rb:183:in `block (2 levels) in in_threads'
rake aborted!
Command failed with status (1): [bundle exec middleman s3_sync --bucket=ash...]
/home/travis/build/ashfurrow/blog/Rakefile:14:in `block (2 levels) in <top (required)>'
/home/travis/build/ashfurrow/blog/Rakefile:60:in `block (2 levels) in <top (required)>'
/home/travis/build/ashfurrow/blog/Rakefile:82:in `block (2 levels) in <top (required)>'
Tasks: TOP => deploy:production
(See full trace by running task with --trace)IAM policy DENY policies take precedence over the allow policies. In this case, you are denying access to any principals outside of the approved IP range. I'm guessing that the IP range(s) contain the IP address for your office or location.
The IAM Policy Evaluation Logic document goes over this in details.
I believe that there is a solution. You might be able to specify a principal to exclude from the policy using the NotPrincipal element in your policy. This element is documented here. That could look like:
{
"Version": "2008-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Deny",
"Principal": {
"AWS": "*"
},
"NotPrincipal": {
"AWS": "arn:aws:iam::243135559143:user/travis"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::ashfurrow.com/*",
"Condition": {
"NotIpAddress": {
"aws:SourceIp": [
"..."
]
}
}
}Let me know if this works as I haven't used that element before and I am quite curious to see whether it helps.
I'll have to borrow some of your CDN hooks since I am also exposing my site through Cloudflare as well.
Awesome! That worked, I didn't know about NotPrincipal, thank you very much. I had to remove Principle from the policy, Amazon insisted I couldn't have both, but it worked.
Here is the PR I added fetching Cloudflare IPs and updating the bucket policy. Thanks again for the quick reply and assistance!