Always get this error: HTTPClientError.getConnectionFromPoolTimeout
iWECon opened this issue · 4 comments
Describe the bug
I started some timing tasks to grab the content of some third-party interfaces.
Timed tasks are performed every 4 minutes.
But I get this error all the time:
Whenever my task lasts for a period of time (uncertain length, sometimes a week, sometimes a few hours), I receive such an error message.
Whenever this error occurs, I can only recover by restarting the server :(
Anyone has any ideas?
Here is my codes (sensitive information has been cleaned up):
Environment
- Vapor 4 Latest Version
- Ubuntu 20.04
@iWECon How many tasks do you have going on at once? How long, on average, does this task take to complete?
I'm asking this because the description of the error says:
/// Aquiring a HTTP connection from the connection pool timed out.
///
/// This can have multiple reasons:
/// - A connection could not be created within the timout period.
/// - Tasks are not processed fast enough on the existing connections, to process all waiters in time
public static let getConnectionFromPoolTimeout = HTTPClientError(code: .getConnectionFromPoolTimeout)
Thus, if the connection pool is completely exhausted, but tasks continue to execute and do not release connections, and new connections are prohibited from being created due to exceeding the limit, then we may encounter just such an error — new tasks get queued and want to get connections from the pool, but all connections are occupied by previous long-lived tasks, so new tasks are canceled due to timeout.
This parameter — app.http.client.configuration.connectionPool — specifies the number of connections in the pool. The default value is 8. Maybe this is too low for your needs?
P.S. Approximately 95% of the issues in this repository whose authors label them bug aren't bugs at all. Yours, apparently, too. Maybe it’s not worth making this label before the true state of affairs becomes clear? This seems a little premature, don't you think?
@mkll
The task 4 minutes run once and it will spend time about 1 minute.
This error will cause all my other services to lose the ability to create new threads from the thread pool at the same time.
That is to say, after the error, the thread pool was occupied until I restarted the server.
Logically, this error should be some error that causes the old thread not to be released correctly. (I think)
@iWECon Vapor initially allocates 64 threads in the thread pool unless you change it at startup. Not sure if the thread pool is relevant to your problem.
Have you changed the value of the parameter I wrote about above? Did this change anything?
Figure out what's going on, put debug prints in tasks, for example, try enabling the debug logging level, etc. No one will debug your code for you.