Documentation question: Does Nginx leverage multipart downloads?
Closed this issue · 4 comments
This is mostly a question around the documentation of this module. With this plugin, is Nginx leveraging multi-part downloads for higher download speeds?
No it does not.
Multipart download is a way of downloading parts of a file i.e. support for the Range header. Doing so in itself does not speed things up. There are 3rd party solutions that claim to speed up the download by making parallel calls. This works in practise (the speeding up aspect) if there is a per connection rate limit imposed by say your ISP but you have not saturated your bandwidth.
Ok -- it would be good to document this. I'm curious, have you done testing to show that the multipart downloads don't help that much in the real world?
Also, I'm curious what you think causes Nginx to be so fast at downloading from S3 in that case? We've done testing and when we use the apt-transport-s3 package to host a 3GB file in S3, we rarely see better than 1.8MB/sec downloads. When we put Nginx in front of it with the ngx_aws_auth plugin, we see it start out at 1.8MB/sec, but quickly accelerate to 20+MB/sec.
Additional question (that should be documented) -- does this module provide any way to check some kind of cache-expiration-timeout in S3? I know we can have Nginx cache in-memory (or disk) the files it proxies back to a client .. but Im wondering if we can determine that cache-time based on S3 telling Nginx whether or not the file has been changed since it last downloaded it. Any thoughts there?
nginx will honour caching policies exposed by the upstream (s3 in this case) unless you explicitly override. So the default behaviour is to honour the S3 policy. I don't think nginx has support of "if modified since" semantics as yet.
A lot of your questions are well suited for the nginx mailing lists. I suggest you pose those questions there for better answers