Can I stream a large object from S3?
Closed this issue · 2 comments
Is there a way to do this? I'm not seeing anything in the docs, so just wondering if it is possible.
@ryanwinchester I think it's possible, but it would require to keep track of each section. There is a range
parameter that you can use in https://hexdocs.pm/aws/AWS.S3.html#get_object/22, which makes this possible. I didn't play with it, but I will try and get back to you.
Also I think that it's possible to add more docs related to the parameters, because here you can find a more detailed explanation: https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html
@ryanwinchester I made this small PoC to demonstrate how this would work using Stream.resource/3
: https://github.com/philss/aws-s3-stream-download-poc/blob/main/lib/download_manager.ex#L22
I'm closing the issue. Feel free to reopen if you need to :)