python-lambda-layer-builder

Creates an AWS Lambda Layers structure that is optimized for: Lambda Layer directory structure, compiled library compatibility, and minimal file size.

This repo was created to address these issues:

Many methods of creating Lambda zip files for Python functions don't work for Lambda Layers
- This is due to the fact Lambda Layers require specific library paths within the zip, where regular Lambda zips don't
Compiled dependencies must be created in an environment that matches the Lambda runtime
Reduce size of the layer by removing unnecessary libraries and files

Note: This script requires Docker and uses a container to mimic the Lambda environment.

Features

Builds either a zip file or a raw directory strucutre (e.g. if you want to use frameworks like Serverless for packaging) containing Python dependencies and places the libraries into the proper directory structure for lambda layers
Ensures compiled libraries are compatible with Lambda environment by using the lambci/lambda Docker container that mimics the lambda runtime environment
Optimized the zip size by removing .pyc files and unnecessary libraries
Allows specifying lambda supported python versions: 2.7, 3.6, 3.7 and 3.8
Automatically searches for requirements.txt file in several locations:
- same directory as script
- parent directory or script (useful when used as submodule)
- function sub-directory of the parent directory

Installation

This function can be cloned for standalone use, into a parent repo or added as a submodule.

Clone for standalone use or within a repo:

# If installing into an exisiting repo, navigate to repo dir
git clone --depth 1 https://github.com/tobilg/python-lambda-layer-builder _build_layer

Alternatively, add as a submodule:

cd {repo root}
git submodule add https://github.com/tobilg/python-lambda-layer-builder _build_layer
# Update submodule
git submodule update --init --recursive --remote

Usage

$ ./build.sh -h
AWS Lambda Layer Builder for Python Libraries

Usage: build.sh [-p PYTHON_VER] [-n NAME] [-f] [-r] [-h] [-v]
  -p PYTHON_VER : Python version to use: 2.7, 3.6, 3.7, 3.8 (default 3.7)
  -n NAME       : Name of the layer
  -f REQ_PATH	  : Path to requirements file
  -r            : Raw mode, don't zip layer contents
  -d            : Don't install Python dependencies
  -s            : Don't strip .so files
  -h            : Help
  -v            : Display build.sh version

Run the builder with the command ./build.sh
- or _build_layer/build.sh if installed in sub-dir
It uses the first requirements.txt file found in these locations (in order):
- Input on the command line using the -f switch
- Same directory as script
- Parent directory of script (useful when used as submodule)
- Function sub-directory of the parent directory (useful when used as submodule)
Optionally specify Python Version
- -p PYTHON_VER - specifies the Python version: 2.7, 3.6, 3.7, 3.8 (default 3.7)

Custom cleaning logic

You can edit the _clean.sh file if you want to add custom cleaning logic for the build of the Lambda layer. The above part of the file must stay intact:

#!/usr/bin/env bash
# Change to working directory
cd $1
# ----- DON'T CHANGE THE ABOVE -----

# Cleaning statements
# ----- CHANGE HERE -----
rm test.xt

The _make.sh script will then execute the commands after the Python packages have been installed.

Uninstall

If installed as submodule and need to remove

# Remove the submodule entry from .git/config
git submodule deinit -f $submodulepath
# Remove the submodule directory from the superproject's .git/modules directory
rm -rf .git/modules/$submodulepath
# Remove the entry in .gitmodules and remove the submodule directory located at path/to/submodule
git rm -f $submodulepath
# remove entry in submodules file
git config -f .git/config --remove-section submodule.$submodulepath