WeTransfer/sqewer

Configurable AWS SQS open and read timeout

roysab opened this issue · 0 comments

The AWS SQS client by default uses the following timeouts:

  • http_open_timeout: 15 seconds
  • http_read_timeout: 60 seconds
  • http_idle_timeout: 5 seconds

For job submit Sqewer uses a retriable request, so we could set a shorter open timeout, as it will retry a number of times. Especially when SQS messages are submitted within the handling of a web request with a 30 second timeout, it would be nice to be able to configure the timeout and number of retries so that it will fit within that 30 seconds request timeout, so that an error will be logged in Appsignal.

Here's a link to an example where this appears to happen that a SQS message submit takes 15 seconds, likely hitting the AWS SQS Client open timeout after 15 seconds and then quickly succeeding on the retry:
https://wetransfer.atlassian.net/browse/AB-29?focusedCommentId=16417

This means that if it times out on the first and second attempt, that the request would already have exceeded the 30 seconds and be aborted.

So configuring a shorter timeout would allow more than 2 attempts, and result in faster response times.