allow specifying the content length for resumable uploads
benjaminp opened this issue · 2 comments
Is your feature request related to a problem? Please describe.
The resumable upload API allows specifying a final length of the object if known in a HTTP header. There is no way to pass this value into the Java GCS API for resumable uploads, though.
Describe the solution you'd like
A Storage.writer method overload that takes the final content length.
In general we recommend folks use checksums for validation of all intended bytes reaching gcs rather than relying on the number of bytes. Using a checksum like crc32c will ensure the bytes are received in the correct sequence and match with what GCS receives.
For an example of how the crc32c precondition is provided to an upload, you can take a look at one of our integration tests that verify correct checksum plumbing and handling
To compute a crc32c checksum you can use Guava's crc32c HashFunction https://guava.dev/releases/33.1.0-jre/api/docs/com/google/common/hash/Hashing.html#crc32c()
I agree that a checksum of the content is preferable when available. However, sometimes you are proxying a file from a client that gives you the size upfront but not a checksum in which case GCS verifying the size is better than nothing.