Async generator-based HTTP Proxy

About

Features

Uses multiple asynchronous functions in parallel where is possible
Small memory footprint, handles well small and big payloads using generators
Fast, totally asynchronous
Headers sent are preserved
Status code is passed through

To Do

Use FastAPI to have an autogenerated OpenAPI documentation
More unit tests mocking Redis and HTTP requests
Only supports POST verb, it possible to adding more
Optional, use poetry with pyproject.toml (PEP 518)

Developing and/or running locally

First, create a .env file

cp .env.example .env

Then create a virtual env and install the dependencies

python3 -m venv venv
source venv/bin/activate
pip install --requirement requirements/development.txt

Run with docker-compose

Redis is running on a tmpfs; each time that starts, it starts with a fresh database

make run

Run tests

make tests

Run linters, type-checking, auto-formaters, and other tools

make vet

Making a request

Make a HTTP POST request with some header

curl -X POST -H "Authorization: Bearer mytoken" http://127.0.0.1:3000/anything

It will return all headers plus the JWT header, which was sent to upstream

{
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "*/*", 
    "Accept-Encoding": "gzip, deflate", 
    "Authorization": "Bearer mytoken", 
    "Connection": "keep-alive", 
    "Content-Length": "0", 
    "Host": "127.0.0.1:3000", 
    "User-Agent": "curl/7.68.0", 
    "X-My-Jwt": "eyJhbGciOiJIUzUxMiIsImlhdCI6MTY2NTY3NTIxNSwianRpIjoiY2NhOWJkZDg0ODFlNGI4ZGE5Y2QzNDA5YWY5MzVmODEiLCJ0eXAiOiJKV1QifQ.eyJ1c2VyIjoidXNlcm5hbWUiLCJkYXRlIjoiMjAyMi0xMC0xM1QxNTozMzozNS4zOTc0NzErMDA6MDAifQ.zxiKtZekBex0q5HeTDe5YQQ41dO5rh4-nJutvKUJ0LI3s2Nx1ZGPiRd1eSjtu5xo0dmQMSYxqW5vjgIaWd6_Bg"
  }, 
  "json": null, 
  "method": "POST", 
  "origin": "172.18.0.4", 
  "url": "http://127.0.0.1:3000/anything"
}

Get status

curl http://127.0.0.1:3000/status

Deployment

Create a project on Google Cloud and run make deploy

skhaz/async-generator-based-http-proxy