/aiolimit

An outbound request rate limiter

Primary LanguagePythonGNU Affero General Public License v3.0AGPL-3.0

aiolimit

An outbound request rate limiter

Purpose

When limiting outbound requests so as to not trigger penalties in a remote system, the remote clock's opinion is the only one which matters. This library allows you to dispatch your requests so that from the perspective of the remote system, regardless of implementation, you cannot have exceeded the remote rate quota.

Installation

python -m pip install -e git+https://github.com/hmusgrave/aiolimit.git#egg=aiolimit

Examples

import asyncio, aiolimit

async def some_remote_call():
    return 1

rps = 1500
max_items_in_window = 100
window_width_seconds = max_items_in_window / rps

async def main():
    limiter = aiolimit.Limiter(window_width_seconds, max_items_in_window)
    for _ in range(1000):
        result = await limiter.run(some_remote_call())

print(asyncio.run(main()))

Status

Contributions welcome (from 22 Feb 2021 through roughly 31 March 2021 no changes will be made, feel free to fork).

Test suite catches:

  • Some obvious misimplementations
  • Bugs related to anything fine-grained like clock resolution

Overhead suitable for:

  • Typical http requests
  • Typical rpc requests
  • High-performance networking
  • Low-level socket handling (depends)

Limiter types:

  • Sliding window (at most n requests in any window of w seconds)
  • Leaky bucket (using the sliding window limiter against a remote leaky bucket algorithm results in marginally lower average throughput than could otherwise be achieved)
  • Dynamic parameter discovery (for any kind of limiter)

Model incorporates:

  • Clock drift
  • Clock resolution
  • Local vs Remote clocks
  • Request latency
  • Relativity

Safe for:

  • Single Thread
  • Multiple Processes

Works on:

  • Asyncio
  • Multiple event loops