Jets Deploy is not successful.
Closed this issue · 7 comments
Checklist
- Upgrade Jets: Are you using the latest version of Jets? This allows Jets to fix issues fast.
- Reproducibility: Are you reporting a bug others will be able to reproduce and not asking a question. If you're unsure or want to ask a question, do so on https://community.boltops.com
- Code sample: Have you put together a code sample to reproduce the issue and make it available? Code samples help speed up fixes dramatically. If it's an easily reproducible issue, then code samples are not needed. If you're unsure, please include a code sample.
My Environment
Software | Version |
---|---|
Operating System | MacOS Ventura 13.4.1 |
Jets | 6.0.3 |
Ruby | 3.3.1 |
Rails | 7.1.3.3 |
Expected Behaviour
Successful deployment.
Current Behavior
Error after building and pushing container image.
Step-by-step reproduction instructions
The following is an excerpt from the error log at that time.
113c74938a2f: Layer already exists
c13df57104fa: Layer already exists
2bd1a2222589: Layer already exists
a0446a1a49e6: Pushed
b7922889b265: Pushed
9c7852261ccf: Pushed
87cfc986bc55: Pushed
v1-2024-06-01T07-21-29Z: digest: sha256:143aafd303be09243b4e3be02371f36da6c8f2b266374c9d831f30113df21622 size: 3669
Building CloudFormation templates
bundler: failed to load command: jets-remote (/codebuild/output/src2987860223/src/wrapper/vendor/bundle/ruby/3.2.0/bin/jets-remote)
/codebuild/output/src2987860223/src/wrapper/vendor/bundle/ruby/3.2.0/bundler/gems/jets-76ce3d3b3501/lib/jets/cfn/iam/policy.rb:66:in `all_actions_colon_star': undefined method `include?' for nil:NilClass (NoMethodError)
action.include?(":") ? action : "#{action}:*"
^^^^^^^^^
from /codebuild/output/src2987860223/src/wrapper/vendor/bundle/ruby/3.2.0/bundler/gems/jets-76ce3d3b3501/lib/jets/cfn/iam/policy.rb:88:in `statement_from_hash'
from /codebuild/output/src2987860223/src/wrapper/vendor/bundle/ruby/3.2.0/bundler/gems/jets-76ce3d3b3501/lib/jets/cfn/iam/policy.rb:59:in `block in statement_from_array'
from /codebuild/output/src2987860223/src/wrapper/vendor/bundle/ruby/3.2.0/bundler/gems/jets-76ce3d3b3501/lib/jets/cfn/iam/policy.rb:55:in `map'
from /codebuild/output/src2987860223/src/wrapper/vendor/bundle/ruby/3.2.0/bundler/gems/jets-76ce3d3b3501/lib/jets/cfn/iam/policy.rb:55:in `statement_from_array'
from /codebuild/output/src2987860223/src/wrapper/vendor/bundle/ruby/3.2.0/bundler/gems/jets-76ce3d3b3501/lib/jets/cfn/iam/policy.rb:37:in `standardize'
from /codebuild/output/src2987860223/src/wrapper/vendor/bundle/ruby/3.2.0/bundler/gems/jets-remote-7acc981690be/lib/jets/remote/cfn/resource/iam/application_role.rb:108:in `custom_policy'
from /codebuild/output/src2987860223/src/wrapper/vendor/bundle/ruby/3.2.0/bundler/gems/jets-remote-7acc981690be/lib/jets/remote/cfn/resource/iam/application_role.rb:59:in `computed_policies'
from /codebuild/output/src2987860223/src/wrapper/vendor/bundle/ruby/3.2.0/bundler/gems/jets-remote-7acc981690be/lib/jets/remote/cfn/resource/iam/class_role.rb:19:in `computed_policies'
from /codebuild/output/src2987860223/src/wrapper/vendor/bundle/ruby/3.2.0/bundler/gems/jets-remote-7acc981690be/lib/jets/remote/cfn/resource/iam/application_role.rb:50:in `policies'
from /codebuild/output/src2987860223/src/wrapper/vendor/bundle/ruby/3.2.0/bundler/gems/jets-remote-7acc981690be/lib/jets/remote/cfn/resource/iam/separate_policy.rb:50:in `policy_document'
from /codebuild/output/src2987860223/src/wrapper/vendor/bundle/ruby/3.2.0/bundler/gems/jets-remote-7acc981690be/lib/jets/remote/cfn/resource/iam/separate_policy.rb:38:in `definition'
Code Sample
just "new"ed Rails App with API mode.
Solution Suggestion
nothing.
Bummer.
Killing 2 birds with one stone. Created a video demo with an Rails API built from scratch and deployed it to AWS Lambda with Jets: https://learn.boltops.com/courses/ruby-on-jets-6-0-guide/lessons/rails-api-on-aws-lambda-with-jets
Interested in what's going on here. Will follow up.
Thanks for making the explanatory video.
I'll check it out later and let you know how it goes.
I tried again, following the video you created.
This time the type of error that occurs seems to have changed.
Below is the error log from that time.
$ AWS_PROFILE=******* AWS_REGION=ap-northeast-1 bundle exec jets deploy
f913b9a38ea5: Pushed
a5de6de934c2: Pushed
3c6ec9c9f94a: Pushed
v1-2024-06-03T18-11-40Z: digest: sha256:863ff69410cef6df7b8d748f5fda81fdbd948cc7ec68c7cc5a47f3f770fc9386 size: 3460
Building CloudFormation templates
Built CloudFormation templates at /tmp/jets/******-web-api-dev/templates
#<Thread:0x0000ffff9d525d58 /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/bundler/gems/jets-remote-7550aa744771/lib/jets/remote/package/assets/upload.rb:25 run> terminated with exception (report_on_exception is true):
/codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-core-3.196.1/lib/seahorse/client/plugins/raise_response_errors.rb:17:in `call': Access Denied (Aws::S3::Errors::AccessDenied)
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-s3-1.151.0/lib/aws-sdk-s3/plugins/sse_cpk.rb:24:in `call'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-s3-1.151.0/lib/aws-sdk-s3/plugins/dualstack.rb:21:in `call'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-s3-1.151.0/lib/aws-sdk-s3/plugins/accelerate.rb:43:in `call'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-core-3.196.1/lib/aws-sdk-core/plugins/checksum_algorithm.rb:111:in `call'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-core-3.196.1/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:16:in `call'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-core-3.196.1/lib/aws-sdk-core/plugins/invocation_id.rb:16:in `call'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-core-3.196.1/lib/aws-sdk-core/plugins/idempotency_token.rb:19:in `call'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-core-3.196.1/lib/aws-sdk-core/plugins/param_converter.rb:26:in `call'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-core-3.196.1/lib/seahorse/client/plugins/request_callback.rb:89:in `call'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-core-3.196.1/lib/aws-sdk-core/plugins/response_paging.rb:12:in `call'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-core-3.196.1/lib/seahorse/client/plugins/response_target.rb:24:in `call'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-core-3.196.1/lib/seahorse/client/request.rb:72:in `send_request'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-s3-1.151.0/lib/aws-sdk-s3/client.rb:15616:in `put_object'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-s3-1.151.0/lib/aws-sdk-s3/file_uploader.rb:66:in `block in put_object'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-s3-1.151.0/lib/aws-sdk-s3/file_uploader.rb:55:in `block in open_file'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-s3-1.151.0/lib/aws-sdk-s3/file_uploader.rb:55:in `open'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-s3-1.151.0/lib/aws-sdk-s3/file_uploader.rb:55:in `open_file'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-s3-1.151.0/lib/aws-sdk-s3/file_uploader.rb:65:in `put_object'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-s3-1.151.0/lib/aws-sdk-s3/file_uploader.rb:46:in `block in upload'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-core-3.196.1/lib/aws-sdk-core/plugins/user_agent.rb:29:in `feature'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-s3-1.151.0/lib/aws-sdk-s3/file_uploader.rb:40:in `upload'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-s3-1.151.0/lib/aws-sdk-s3/customizations/object.rb:477:in `block in upload_file'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-core-3.196.1/lib/aws-sdk-core/plugins/user_agent.rb:29:in `feature'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-s3-1.151.0/lib/aws-sdk-s3/customizations/object.rb:476:in `upload_file'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/bundler/gems/jets-remote-7550aa744771/lib/jets/remote/package/assets/upload.rb:40:in `upload_to_s3'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/bundler/gems/jets-remote-7550aa744771/lib/jets/remote/package/assets/upload.rb:26:in `block (3 levels) in upload_folder'
bundler: failed to load command: jets-remote (/codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/bin/jets-remote)
/codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-core-3.196.1/lib/seahorse/client/plugins/raise_response_errors.rb:17:in `call': Access Denied (Aws::S3::Errors::AccessDenied)
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-s3-1.151.0/lib/aws-sdk-s3/plugins/sse_cpk.rb:24:in `call'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-s3-1.151.0/lib/aws-sdk-s3/plugins/dualstack.rb:21:in `call'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-s3-1.151.0/lib/aws-sdk-s3/plugins/accelerate.rb:43:in `call'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-core-3.196.1/lib/aws-sdk-core/plugins/checksum_algorithm.rb:111:in `call'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-core-3.196.1/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:16:in `call'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-core-3.196.1/lib/aws-sdk-core/plugins/invocation_id.rb:16:in `call'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-core-3.196.1/lib/aws-sdk-core/plugins/idempotency_token.rb:19:in `call'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-core-3.196.1/lib/aws-sdk-core/plugins/param_converter.rb:26:in `call'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-core-3.196.1/lib/seahorse/client/plugins/request_callback.rb:89:in `call'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-core-3.196.1/lib/aws-sdk-core/plugins/response_paging.rb:12:in `call'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-core-3.196.1/lib/seahorse/client/plugins/response_target.rb:24:in `call'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-core-3.196.1/lib/seahorse/client/request.rb:72:in `send_request'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-s3-1.151.0/lib/aws-sdk-s3/client.rb:15616:in `put_object'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-s3-1.151.0/lib/aws-sdk-s3/file_uploader.rb:66:in `block in put_object'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-s3-1.151.0/lib/aws-sdk-s3/file_uploader.rb:55:in `block in open_file'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-s3-1.151.0/lib/aws-sdk-s3/file_uploader.rb:55:in `open'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-s3-1.151.0/lib/aws-sdk-s3/file_uploader.rb:55:in `open_file'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-s3-1.151.0/lib/aws-sdk-s3/file_uploader.rb:65:in `put_object'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-s3-1.151.0/lib/aws-sdk-s3/file_uploader.rb:46:in `block in upload'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-core-3.196.1/lib/aws-sdk-core/plugins/user_agent.rb:29:in `feature'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-s3-1.151.0/lib/aws-sdk-s3/file_uploader.rb:40:in `upload'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-s3-1.151.0/lib/aws-sdk-s3/customizations/object.rb:477:in `block in upload_file'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-core-3.196.1/lib/aws-sdk-core/plugins/user_agent.rb:29:in `feature'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/gems/aws-sdk-s3-1.151.0/lib/aws-sdk-s3/customizations/object.rb:476:in `upload_file'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/bundler/gems/jets-remote-7550aa744771/lib/jets/remote/package/assets/upload.rb:40:in `upload_to_s3'
from /codebuild/output/src3576152674/src/wrapper/vendor/bundle/ruby/3.2.0/bundler/gems/jets-remote-7550aa744771/lib/jets/remote/package/assets/upload.rb:26:in `block (3 levels) in upload_folder'
Error running command: exit status 1
[Container] 2024/06/03 18:11:51.991075 Command did not exit successfully ./jets-go deploy exit status 1
There was no difference between executing the command inside the container and executing the command directly on the host.
And, I have granted AdministratorAccess to my AWS user.
I wish I could provide more information, but I really don't have anything else.
RE: I wish I could provide more information, but I really don't have anything else.
All good. It does help a little bit.
- Thanks for sending over the
config/jets
files. Well, it looks like the vanilla-generated files. So that's not it. - I also deployed to
AWS_REGION=ap-northeast-1
, the same region as you. Of course, it deployed successfully. 🥺
RE: I have granted AdministratorAccess to my AWS user.
There's a difference between the AWS IAM user you use locally vs the AWS Codebuild Remote Runner IAM Role. Docs: https://docs.rubyonjets.com/docs/iam/deploy/vs/ Jets creates and manages the IAM Role associated with the CodeBuild Remote Runner, and your config/jets
settings did not override them. So believe that looks good.
- This seems to be an issue specific to your AWS account.
- This is a guess and a shot-in-the-dark, but your AWS account might have an AWS Service Control Policy at the Organization level.
- Handling multiple birds with one stone again. Added these docs: https://docs.rubyonjets.com/docs/iam/deploy/remote/#aws-service-control-policies
I'm guessing it's an AWS SCP that's resulting in the Access Denied (Aws::S3::Errors::AccessDenied)
error. I was kind of able to mimic the error by creating an SCP for one of my accounts. I created a deny policy that denies all s3. It mimic the error, albeit it errors earlier in the process. It's might be because my mimic SCPs is not like yours and even more restrictive. Again, it's a guess.
Maybe check if your AWS account has SCPs.
I tested it in a clean environment with no SCPs, and the deploy was successful.
I will let you know if I find any other problems again.
Thanks for your help.