reddit/baseplate.py

Rethink max_concurrency and active_requests

Closed this issue · 1 comments

These values are based on the number of live connections rather than actual active requests. In many cases, those connections are idle. We need to be limiting and tracking concurrency by actual requests instead.

Once server timeouts are in place, should we just drop max_concurrency? I don't think it's ever right to reject the inbound connections (that sort of overload should be handled at the autoscaler and service mesh) and the timeout will be a more effective defense mechanism for the server.