Generate code and manage infrastructure for receiving webhooks with AWS API Gateway and pushing to SQS via Lambda.
Webhooks are great, and many projects and services are now offering them as a notification option. But sometimes it makes more sense to have the messages in a queue that can absorb changes in rate and de-couple the sending service from a potentially slow or unavailable backend.
webhook2lambda2sqs generates code for an AWS Lambda function to receive webhook content via API Gateway and push it to an SQS queue, where it can be consumed as needed. It supports multiple endpoints via unique URL paths (API Gateway resources), where content sent to each endpoint is pushed to one or more SQS queues.
In addition, webhook2lambda2sqs includes a wrapper around HashiCorp Terraform to automate creation and maintenance of all or part of the infrastructure required to operate this (the API Gateway and its configuration, the Lambda function, the function's IAM role, etc.). If TerraForm isn't a viable option for you to manage infrastructure with, you can use the generated configuration (which maps quite closely to AWS API parameters) as a guide for manual management.
There are also helper commands to view the Lambda Function and API Gateway logs, send a test message, and view the queue contents.
For full documentation, see: http://webhook2lambda2sqs.readthedocs.io/en/latest/
- An Amazon AWS account to run this all in (note - it will probably be cheap, but not free)
- Python 2.7+ (currently tested with 2.7, 3.3, 3.4, 3.5). Note that AWS Lambda currently only supports python 2.7 as an execution environment, but you're welcome to use other versions on the machine where you run this project.
- Python VirtualEnv and
pip(recommended installation method; your OS/distribution should have packages for these) - HashiCorp Terraform >= 0.6.16 to manage the AWS infrastructure, if desired. Terraform is written in Go, and distributed as a static binary.
- Lambda Function code generation
- Terraform configuration generation
- Management of infrastructure via Terraform
- AWS-related helpers for inspecting queues and logs, querying information, and enabling metrics/logging/rate limiting on the API Gateway.
For full documentation, see: http://webhook2lambda2sqs.readthedocs.io/en/latest/
Bug reports and feature requests are happily accepted via the GitHub Issue Tracker. Pull requests are welcome. Issues that don't have an accompanying pull request will be worked on as my time and priority allows.
This program is licensed under the GNU Affero General Public License, version 3.0 or later ("AGPLv3"). The AGPLv3 includes a clause that source code must be made available to anyone using the program over a network.