Testlio/lambda-tools

[:new:] Add support for static assets

Closed this issue ยท 3 comments

Type:
๐Ÿ†•

Description:
Currently there is no way for Lambda functions to bundle static assets, which may be necessary for various template files etc.

Motivation:
At the moment, only the function code is packaged into the ZIP file, which means it is not possible to have Lambda functions use a file from disk (for example a template file). We should try to circumvent this by creating an option or a way for the deployment script to bundle not only code, but also arbitrary files.

Affected features:

  • lambda deploy
  • lambda deploy-single

Proposed implementation:

  • One way to implement this is to have a "manifest.json" file in each Lambdas directory. This file could then contain information about the files that need to be bundled along, with paths relative to the Lambda directory. Something along the lines of { "include": { "path/to/have/in/lambda": "path/relative/to/local/lambda/directory/file.json" } }
  • Alternative is to have a special folder under the Lambda directory, which is always packaged up and included, sort of like an "assets" or "static" type of approach.
  • The benefit of the first approach is the chance of reusing files, however, the benefit of the second is the simplicity
  • In either case, this will need to adjust the bundling logic in lib/deploy/bundle-lambdas-step.js

Testing strategy:

  • Create a set of Lambdas (a small service), some of which want files, others that don't.
  • Have the Lambda functions log out the contents of their "home directory" on AWS
  • Validate that the files that needed to be included, are included, and no extra files are added

Example usage:
Script usage would not change, as this functionality should be per Lambda function

The first approach will likely be the chosen one, as it allows better control over defining the paths that the files will end up getting in the final Lambda function.

Two problems remain:

  1. Where to store this path mapping, one candidate would be cf.json, as it could contain another key in there (sibling to Properties) that can be picked out as Properties currently is for customising the Lambda configuration.
  2. How to handle file paths locally in lambda run and lambda execute. If we choose an approach where the files in the final bundle can have arbitrary paths, then the local running environment would need to somehow sandbox the files to appear in the correct spots as well. Otherwise testing static files would only work in AWS.

One option would be something like https://github.com/tschaub/mock-fs, where during Lambda execution fs module is mocked out with the files from the bundle.

Just to cover all bases, I also tried out brfs, which would in theory work, however the problem is that all fs references are resolved statically, meaning that if there is a Lambda function that does indeed want to use fs dynamically, it wouldn't be able to do so.

More and more it seems that the correct approach is to utilise the cf.json file to have a mapping of files that need to be included in the bundle, then this mapping can be utilised both when bundling the Lambda function as well as when running the Lambda function locally (via something like mock-fs in the execution wrapper).