tus/tus-node-server

Concurrent uploads to S3 fails

Closed this issue · 3 comments

josephrony277 commented

I'm trying to upload files from different clients to S3 concurrently. Most of the concurrent uploads finish without issues, but for a few uploads, the patch request returns a 408-time-out error after 3 mins. Node js server application has thrown this error below

{
"code": "ECONNRESET",
"name": "Error",
"message": "aborted",
"stack": "Error: aborted\n at connResetException (node:internal/errors:720:14)\n at abortIncoming (node:_http_server:766:17)\n at socketOnClose (node:_http_server:760:3)\n at Socket.emit (node:events:529:35)\n at Socket.emit (node:domain:489:12)\n at TCP. (node:net:350:12)"
}

I tried a fix for the issue #409 and even with the latest version, the issue still remains.

Even when closing the closing the client app during upload, the same error is thrown in the server application.

Fyi,

  • Chunk size configured in the client - 50MB
  • S3 part size configured in the node js app - 50MB
  • Node js app is running as fargate service
Murderlon commented

Hi, can you try not setting the chunk size on the client, you never want to set this it's always worse unless you are forced to.

I would also considering lowering the S3 part size (for instance 10MB or 20MB), which doesn't have anything to do with the client chunk size and they don't have to be in sync.

Can you see if the issue persists then? Of course, the tus server should work with all settings but I I'm trying to see if it may be a connection error to S3 because of your settings.

mitjap commented

@josephrony277 Do you use Amazon S3 or any other provider?

Murderlon commented

Fairly certain this was fixed in @tus/s3-store@1.0.1 or @tus/s3-store@1.1.0. We can reopen if someone can reproduce this.