antespi/s3md5

Automatically calculate the chunk size from the etag

Closed this issue · 6 comments

I am not entirely sure this is correct but it worked to calculate the chunk size for my files:

chunks=$(echo $etag | cut -d'-' -f 2); filesize=$(du -b $file | cut -f 1); echo "($filesize / (1024 * 1024)) / $chunks" + 1 | bc

So with that maybe your script can add an option like s3md5 -etag $etag filename so you don't need to provide the chunk size.

I'll try to make the time to do a PR.

This could be a good feature, then script won't return etag, but true if etag is equal or false if not.
Thanks, waiting for your PR ;)

I have most of it done, will finish it on Monday :-)

--David

I have a PR coming, I changed a few things while trying this out. However, the way I was calculating the chunk size is incorrect, there is no real way of knowing what is the chunk size used when uploading. All I am doing now is to indicate what are valid values for the chunk size in case you don't know.

I created the pull request but didn't get referenced here. #2

I've merge your PR. Thanks for your improvements.
I've just tested it in my servers and it's running very well.

Not a problem, I will probably make some more updates to it since I want to ship it to our customers for them to be able to verify what they download from us.

Thanks for a cool script and for being open to collaborating :-)