Lightweight docker image for running, testing, packaging and deploying python-based AWS lambda functions
- Docker: python-lambda
- GitHub: python-lambda-docker
- Based on the python-lambda library by nficano
I needed a Docker-based environment in which to host AWS python lambda functions for the purpose of testing and building them... python-lambda works well under virtualenv for development, but build and deployment automation require a clean and reproducible environment to operate in. Our CI system already supported Docker as a containerization system, so it was the obvious choice.
In order to use this, you will have your project derive its own Dockerfile based on a base python-lambda image corresponding to which version of Python you wish to run.
An example of a usable project can be found in the example/ directory. The lambda function in service.py takes a JSON input file like the one provided in event.json and returns an ASCII-art version of the text described in it. The Dockerfile derives from python-lambda as a base image, and loading the example directory contents into the image at the path /lambda, then installs dependencies from the requirements.txt file.
To build a docker image from the provided example/ called example-lambda-image with the ASCII-art lambda function in it, run:
$ cd example/
$ docker build --tag example-lambda-image .
Any time you make changes to the example project, you'll need to re-run the docker build command above, or you can investigate using docker volumes to sync local filesystem changes into the container.
The example Dockerfile uses a :latest docker tag in the FROM line, which is currently the same as :3.6, but if you wish to use a different Python version you can change this. Supported Python versions are 2.7 and 3.6. To use Python version 2.7 change the first line of the example Dockerfile to:
FROM kilna/python-lambda:2.7
You will also need to change the following line in config.yaml
runtime: python2.7
If you want to execute the lambda function against the event.json input file:
$ docker run example-lambda-image lambda invoke
_ _ _ _ _ _ _
| || |___| | |___ __ __ _____ _ _| |__| | |
| __ / -_) | / _ \_ \ V V / _ \ '_| / _` |_|
|_||_\___|_|_\___( ) \_/\_/\___/_| |_\__,_(_)
|/
If you would like to build the lambda function (which will package the function and all its dependencies), run:
$ docker run example-lambda-image lambda build
The result will be a zip file created in the /lambda/dist directory inside the container.
If you would like to build and test the lambda function and gather up the results, you can run:
$ docker run example-lambda-image lambda_build_tar | tar -x -v
build.log
test.log
dist/
dist/2017-09-01-003647-example-lambda.zip
Behind the scenes, what this does is:
- Removes any log and dist files from prior runs
- Executes 'lambda build' and stores the log in /lambda/build.log in the container
- If present and executable, runs /lambda/run_tests and stores the log in /lambda/test.log in the container
- Tars the log files, and the contents of the dist directory in /lambda on the container and pipes it to standard output
- Untars the contents bundled up within the container, and extracts them into your current directory
You can deploy your lambda function to Amazon's infrastructure... you'll need to add AWS credentials into the config.yaml file. Alternately, if your local AWS CLI environment is working, you can add a .aws/ directory (you can cp ~/.aws) into the example/ directory), then re-build your image. Once AWS is working within your container, you can then run the following to deploy your function to Amazon:
$ docker run example-lambda-image lambda deploy