mlhpdx/s3-upload-stream

Question about part length

Closed this issue · 2 comments

Hi,

Thanks a lot for the example, this solved me a big issue.
I just have a question regarding the calculation of the part length in StartNewPart method:
_metadata.PartLength = Math.Min(MAX_PART_LENGTH, Math.Max(_metadata.PartLength, (_metadata.PartCount / 2 + 1) * MIN_PART_LENGTH));

_metadata.PartLength = Math.Min(MAX_PART_LENGTH, Math.Max(_metadata.PartLength, (_metadata.PartCount / 2 + 1) * MIN_PART_LENGTH));

What is the logic behind _metadata.PartCount / 2 + 1?

Good question. Effectively this make the part size grow as more parts are added. To make the code flexible, the initial part size is small, which works for small to medium size files. This code increases the part size so that upload of larger files also works well (there is a hard limit of 10,000 parts in S3 so the part size needs to be large to upload very large files).

It's a bit of a weak approach, but simple and I didn't want to distract from the main point of the library here by adding a "strategy pattern" to control it, which would be the best answer.

Make sense?

Thank you so much for the quick answer.