FormidableLabs/serverless-jetpack

Question: Packaging configuration seems to be ignored

DanEscudero opened this issue · 3 comments

DanEscudero commented

Hi!

I'm facing some trouble when generating the package, and I think files are not being included correctly.

Setup

I'm using serverless@3.28.1, and serverless-jetpack@0.11.1
I have this serverless.yml:

service: jetpack-test

provider:
  name: aws
  runtime: nodejs14.x

plugins:
  - serverless-jetpack

functions:
  myFunction:
    name: 'myFunction'
    handler: main.handler
    timeout: 5

    jetpack:
      preInclude:
        - 'files/function-pre-include.txt'
        - '!files/function-pre-exclude.txt'
    
    package:
      include:
        - 'files/function-include.txt'
        - '!files/function-exclude.txt'

custom:
  jetpack:
    preInclude:
      - '!**'
      - 'files/service-pre-include.txt'
      - '!files/service-pre-exclude.txt'

package:
  include:
    - 'files/package-include.txt'
    - '!files/package-exclude.txt'

And my file structure is pretty straight-forward:

serverless.yml
package.json
node_modules
main.js
files
|__ function-pre-include.txt
|__ function-pre-exclude.txt
|__ function-include.txt
|__ function-exclude.txt
|__ service-pre-include.txt
|__ service-pre-exclude.txt
|__ package-include.txt
|__ package-exclude.txt

Excpected:

The expected should be a zip file with all *include.txt files, along with node_modules production dependencies, like so:

node_modules
files
|__ function-pre-include.txt
|__ function-include.txt
|__ service-pre-include.txt
|__ package-include.txt

Actual Result:

When running yarn serverless package, the zip file contains only this files:

node_modules
files
|__ service-pre-include-file.txt
|__ package-include-file.txt

Report

When generating report, this is what I'm getting:

# Jetpack Bundle Report

## jetpack-test.zip

- Path: /home/daniel/layer-test/.serverless/jetpack-test.zip
- Mode: dependency
- Roots: (None)

### Tracing: Configuration
# Ignores (`0`):
# Allowed Missing (`0`):

### Patterns: Include

# Automatically added
- '**'
# Jetpack (`3`): `custom.jetpack.preInclude` + `function.{NAME}.jetpack.preInclude`
- '!**'
- 'files/service-pre-include.txt'
- '!files/service-pre-exclude.txt'
# Jetpack (`4`): dependency filtering mode additions
- '!node_modules/**'
- '!node_modules/**'
- 'node_modules/is-number/**'
- '!node_modules/is-number/node_modules/**'
# Jetpack (`0`): trace mode additions

# Serverless (`2`): `package.include` + `function.{NAME}.package.include` + internal extras
- 'files/package-include.txt'
- '!files/package-exclude.txt'

### Patterns: Exclude

# Serverless (`8`): `package.exclude` + `function.{NAME}.exclude` + internal extras
- '.git/**'
- '.gitignore'
- '.DS_Store'
- '.serverless/**'
- '.serverless_plugins/**'
- 'npm-debug.log*'
- 'yarn-error.log*'
- '/home/daniel/layer-test/.serverless_plugins'

### Files (`6`): Included

- files/package-include.txt
- files/service-pre-include.txt
- node_modules/is-number/LICENSE
- node_modules/is-number/README.md
- node_modules/is-number/index.js
- node_modules/is-number/package.json

### Files (`0`): Excluded
### Tracing Dynamic Misses (`0` files): Sources
### Tracing Dynamic Resolved (`0` files): Sources
### Tracing Dynamic Misses (`0` packages): Dependencies
### Tracing Dynamic Resolved (`0` packages): Dependencies
### Collapsed (`0`): Sources
### Collapsed (`0`): Dependencies

I also conducted another test, but this time removing the - !**/** line from custom.jetpack.preInclude; This time, I got in the output package all files in files folder, except for service-pre-exclude and package-exclude.

This way, both of this tests show that function.{NAME}.package.include and function.{NAME}.jetpack.preInclude are being ignored; So, the question is: What am I missing? Is this a configuration issue? Or did I misunderstand the documentation?

Here, I'm leaving a repo where you can clone and test this very same repository that I used and reproduce this issue:
https://github.com/DanEscudero/sls-jetpack-issue-reproduction

Thanks in advance!

ryan-roemer commented

So, the tricky part is for service packaging in SLS (with or without jetpack) function.NAME.include is completely ignored if you are not packaging individually: true. For example:

diff --git a/serverless.yml b/serverless.yml
index bd741b3..b14d5dd 100644
--- a/serverless.yml
+++ b/serverless.yml
@@ -4,9 +4,6 @@ provider:
   name: aws
   runtime: nodejs14.x
 
-plugins:
-  - serverless-jetpack
-
 functions:
   myFunction:
     name: 'myFunction'
@@ -16,7 +13,7 @@ functions:
       preInclude:
         - 'files/function-pre-include.txt'
         - '!files/function-pre-exclude.txt'
-    
+
     package:
       include:
         - 'files/function-include.txt'
@@ -31,5 +28,6 @@ custom:
 
 package:
   include:
+    - '!**'
     - 'files/package-include.txt'
     - '!files/package-exclude.txt'

Where I've disabled jetpack, added a service level package.includeof !** at the start (to negate the initially added ** SLS starts with). The result is that the service level files/package-include.txt is included, but the function.myFunction.pcakge.include'ed file files/function-include.txt is not.

We end up with only one file in this case:

$ yarn sls package && zipinfo -1 .serverless/jetpack-test.zip
files/package-include.txt
ryan-roemer commented

Turning now to Jetpack, jetpack does the same thing -- function.NAME.jetpack.preInclude only is applicable if individually: true. So let's package that function individually instead:

diff --git a/serverless.yml b/serverless.yml
index bd741b3..84a02a2 100644
--- a/serverless.yml
+++ b/serverless.yml
@@ -16,8 +16,9 @@ functions:
       preInclude:
         - 'files/function-pre-include.txt'
         - '!files/function-pre-exclude.txt'
-    
+
     package:
+      individually: true
       include:
         - 'files/function-include.txt'
         - '!files/function-exclude.txt'
@@ -25,11 +26,11 @@ functions:
 custom:
   jetpack:
     preInclude:
-      - '!**/**'
+      - '!**'
       - 'files/service-pre-include.txt'
       - '!files/service-pre-exclude.txt'

The resulting report (and zip file if you info it) now show all four of your expected include files:

$ yarn sls jetpack package --report
# ...
### Files (`8`): Included

- files/function-include.txt
- files/function-pre-include.txt
- files/package-include.txt
- files/service-pre-include.txt
- node_modules/is-number/LICENSE
- node_modules/is-number/README.md
- node_modules/is-number/index.js
- node_modules/is-number/package.json

I would characterize this issue as: (1) expected behavior, that (2) is not well-documented either in SLS or in this plugin. Accordingly I've opened #353 to better document this.

Hopefully this helps you through your issue. I wanted to further thank you for taking the time to open up a full error reproduction repository -- it makes diagnosis and discussion of nuanced issues like these actually tractable!

DanEscudero commented

Thanks!
I added the individually: true in the package session, and it worked well!