amazon-archives/aws-sdk-core-ruby

Progress callback for S3Object

Closed this issue · 3 comments

amw commented

(Recreating issue aws/aws-sdk-ruby#648 from v1 of the library)

Hi, I would like to indicate progress of a large file upload to my users. Currently it is not supported with the SDK. If I wanted to use a single-part upload only (I don't) I could use the block form of S3Object#write and report how many bytes I've fed to the SDK, but that's not an actual upload - at that point the data is still waiting to be sent. For multipart upload there's not even that.

What do you think about adding a callback to that method? Something like:

progress = -> bytes { ... }
object.write file: path, progress_callback: progress

I think this is a good idea, and would like to see something like this as part of the SDK. There are some interesting challenges. This would need to work for simple PUT requests as well as multipart uploads, including those that use client-side encryption. Adding this as a feature of the file transfer manager should be possible.

For single put requests, this would require wrapping the io object that is streamed as the HTTP response body so that it could emit as bytes are read from it. Without this, you wouldn't receive notifications until the upload was complete.

Let me chew on this and I'll try to update this issue with a more formalized proposal.

This is added to our public backlog in the primary repo. Closing this issue as the old repository will be closed soon.

Hi there, as I'm integrating an S3 target in rest-ftp-daemon, I was wondering if upload progress was available through a callback method or passing a block, now ?
Any news about this feature ?