Unable to initiate resumable uploads from java sdk: Cloud storage v2 API.
nithinks21 opened this issue · 4 comments
PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.
Hi There,
I am trying to initiate a resumable upload, from the java sdk using the following code. But getting an error on startResumableWrite request.
Kindly, let me know if I am missing any mandatory params in my request. If so, can you guide me in adding them in the request.
It would be great if you can point me to any usage examples, other than the java source doc(as I have already referred to it).
Here's the code:
`
public static void main(String[] args) throws Exception {
Map<String, String> m = new java.util.HashMap<>();
m.put("type", "service_account");
m.put("project_id", "**-development-******");
m.put("private_key_id", "**********************************");
m.put("private_key", "-----BEGIN PRIVATE KEY-----\n****************************\n-----END PRIVATE KEY-----\n");
m.put("client_email", "*******@**-development-******.iam.gserviceaccount.com");
m.put("client_id", "************************");
m.put("auth_uri", "https://accounts.google.com/o/oauth2/auth");
m.put("token_uri", "https://oauth2.googleapis.com/token");
m.put("auth_provider_x509_cert_url", "https://www.googleapis.com/oauth2/v1/certs");
m.put("client_x509_cert_url", "https://www.googleapis.com/robot/v1/metadata/x509/*******%40**-development-********.iam.gserviceaccount.com");
m.put("universe_domain", "googleapis.com");
byte[] configBytes = JsonUtils.toBytes(m);
InputStream is = new ByteArrayInputStream(configBytes);
GoogleCredentials credentials = GoogleCredentials.fromStream(is);
com.google.storage.v2.StorageSettings storageSettings = StorageSettings.newBuilder()
.setCredentialsProvider(FixedCredentialsProvider.create(credentials)).build();
com.google.storage.v2.StorageClient storageClientV2 = StorageClient.create(storageSettings);
var startResumableWriteRequest = StartResumableWriteRequest.newBuilder()
.setWriteObjectSpec(WriteObjectSpec.newBuilder().build())
.setCommonObjectRequestParams(CommonObjectRequestParams.newBuilder().build())
.setObjectChecksums(ObjectChecksums.newBuilder().build()).build();
var startResumableWriteResponse = storageClientV2.startResumableWrite(startResumableWriteRequest);
}
`
binary: gapic-google-cloud-storage-v2-2.26.0.jar
Class: com.google.storage.v2.StorageClient
API: startResumableWrite
Issue:
`
Exception in thread "main" com.google.api.gax.rpc.InvalidArgumentException: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: An x-goog-request-params request metadata property must be provided for this request.
at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:92)
at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:41)
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:86)
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:66)
at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:97)
at com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:84)
at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1127)
at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31)
at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1286)
at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1055)
at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:807)
at io.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:578)
at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:548)
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at com.google.api.gax.grpc.ChannelPool$ReleasingClientCall$1.onClose(ChannelPool.java:541)
at io.grpc.internal.DelayedClientCall$DelayedListener$3.run(DelayedClientCall.java:489)
at io.grpc.internal.DelayedClientCall$DelayedListener.delayOrExecute(DelayedClientCall.java:453)
at io.grpc.internal.DelayedClientCall$DelayedListener.onClose(DelayedClientCall.java:486)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:567)
at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:71)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:735)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:716)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Suppressed: com.google.api.gax.rpc.AsyncTaskException: Asynchronous task failed
at com.google.api.gax.rpc.ApiExceptions.callAndTranslateApiException(ApiExceptions.java:57)
at com.google.api.gax.rpc.UnaryCallable.call(UnaryCallable.java:112)
at com.google.storage.v2.StorageClient.startResumableWrite(StorageClient.java:3047)
at aone.libs.vault.GcsTest.main(GcsTest.java:49)
Caused by: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: An x-goog-request-params request metadata property must be provided for this request.
at io.grpc.Status.asRuntimeException(Status.java:539)
... 17 more
`
Thanks in Advance,
Nithin
Very sorry for the late response on this!
It looks like you're trying to use the generated library. That library isn't user-friendly and we don't recommend it for anyone. We should make this clearer on our docs, and we will follow up on that.
Instead we recommend using the Storage SDK.
To get the equivalent of what you're using here, you should use the Storage.writer method, which will start a resumable upload
@JesseLovelace,
My requirement is to generate a presigned URL, to perform resumable uploads.
I understand from the GCP support team that the java SDK doesn't support it.
Alternatively, I tried to use the ReST API, to initiate a resumable upload, but there is an issue in obtaining an access token from Google Credentials instance.
I had opened a ticket for this issue --> #2310
Access token is a mandatory field required to use these ReST APIs.
[1].http://cloud/storage/docs/performing-resumable-uploads#resume-upload
[2].https://cloud.google.com/storage/docs/performing-resumable-uploads#initiate-session
[3].https://cloud.google.com/storage/docs/performing-resumable-uploads#resume-upload
Hi @nithinks21,
You can create a resumable upload signed URL and use it with the Java Storage SDK; here's an example, https://cloud.google.com/java/docs/reference/google-cloud-storage/latest/com.google.cloud.storage.Storage#com_google_cloud_storage_Storage_writer_java_net_URL_
Does this help address your requirement?
Closing this this issue out. Please re-open if the suggestion does not solve your problem.