Rethink max_concurrency and active_requests
Closed this issue · 1 comments
spladug commented
These values are based on the number of live connections rather than actual active requests. In many cases, those connections are idle. We need to be limiting and tracking concurrency by actual requests instead.
spladug commented
Once server timeouts are in place, should we just drop max_concurrency
? I don't think it's ever right to reject the inbound connections (that sort of overload should be handled at the autoscaler and service mesh) and the timeout will be a more effective defense mechanism for the server.