googleapis/storage-testbench

grpc: object upload issues found by java retry conformance tests

BenWhitehead opened this issue · 0 comments

While updating the java retry conformance suite to also run for transport: GRPC I ran into two oddities in the testbench retry_test integration with grpc.

storage.object.create instructions do not apply to StartResumableWrite

Java treats all resumable upload PUTs as idempotent. Because the instruction isn't being processed until WriteObject, java still retries. This is inconsistent with JSONs handling, which does apply instructions to starting a resumable upload.

WriteObject does not finalize a resumable session sometimes

When testbench receives a WriteObjectRequest with finish_write: true without any checksummed_data it incorrectly response with an incremental response rather than the resources.

  • >>> message sent
  • <<< message received
>>> {
  upload_id: "ef1d103dc03cc220587514ab957cb56e3d9e80af394af3b5d05f8ba97602a94f"
  checksummed_data {
    content: "AAAAAAAAAAAAAAAAAAAA<snip (262144)>"
  }
}
<<< {
  persisted_size: 262144
}
## resumable session left open for further appends

## EOF signaled in app, finalize resumable session
>>> {
  upload_id: "ef1d103dc03cc220587514ab957cb56e3d9e80af394af3b5d05f8ba97602a94f"
  write_offset: 262144
  finish_write: true
}
<<< {
  persisted_size: 262144
}

The checksummed_data in WriteObjectRequest is in the oneof data declaration. Because it is declared in the oneof it is implicitly optional and need to be present in the message.

AIP-146 touches on this briefly

Because the individual fields in the oneof have different keys, a developer can programmatically determine which (if any) of the fields is populated.

Testbench will need to be updated to account for this.