Is there a way to ignore files or directories during jets deploy
Closed this issue · 7 comments
We have some sub directories in our main jets app that are quite large and don't need to be deployed. Is there the equivalent of a .jetsignore file or something where we can specify directories/files that shouldn't be deployed.
Sure. Currently, anything in .gitignore or .dockerignore is not included in the deployment package.
So can add the files not wanted in the deployment package to one of those. 👍
Thanks a bunch. One issue I am still having is it seems the automatically generated /vendor gems directory is including test and development mode gems, and is thus blowing through the lambda size limit upon deployment. I thought gems got stored in the gem layer?
Actually I think I know what the problem is. This is my current error:
=> rsync -a --links /tmp/jets/arist/stage/code/vendor/gems/ruby/2.7.0/ /tmp/jets/arist/stage/opt/ruby/gems/2.7.0/
=> Replacing compiled gems with AWS Lambda Linux compiled versions: /tmp/jets/arist/stage/opt
Checking projects gems for binary serverlessgems...
Over the Lambda size limit of 250MB
Please reduce the size of your code.
Sizes:
Code: 162.9MB - /tmp/jets/arist/stage/code
Gem Layer: 111.5MB - /tmp/jets/arist/stage/opt
Total Package: 274.4MB
Over limit by: 24.4MB
Note that upon manual inspection, the size of my /tmp/jets/arist/stage/code directory is under 4 MB, however if you also include the vendor directory that is automatically created, du -sh reports the size of my code directory as 162.9 MB.
@tongueroo I think your code here https://github.com/boltops-tools/jets/blob/35bb5a40148074ae03f03b85b16e63707df90970/lib/jets/builders/code_size.rb#L22-L26 that calculates that the size of code directory + size of gem layer is too big is not taking into account the symlinks in the vendor directory, so the gem layer is effectively being counted twice. In reality jets should be able to deploy this just fine, but it is aborting with an over limit error. Thoughts?
I would think an easy fix for this would be to calculate deployment size and possibly throw the over limit error before creating the symlinks, or to make code_size.rb skip symlinks when calculating total size.
Update: I am only experiencing this issue in Github Actions / ubuntu-latest. Running locally in ubuntu 20.04, it works just fine. Maybe something about the Github Actions container environment messes up rsync -a --links
Yeah I can now confirm -- for whatever reason the du command used by jets deploy (and every combination of du command line args I've been able to put together) will include the symlinked vendor/gems/ruby/2.7.0 directory when calculating the size of the /tmp/jets/arist/stage/code directory when inside GitHub Actions (as well as on several of my colleagues' MacOS machines), but will exclude it (correctly) on other systems/environments. I've also manually inspected the created directories in GitHub Actions after failure and I can confirm that the symlink is set up correctly and is identical to the one I have when I run jets deploy locally. So this is some kind of environment-specific bug with du and might have something to do with IO in docker.
I think I will submit a pull request for jets that provides a more portable compute_size implementation in pure ruby that explicitly doesn't follow symlinks.