Checking for object existence in GCS emits an error log

[renchap]

Environment details

• OS: macOS
• Ruby version: 3.1.1
• Gem name and version: google-apis-core version 0.7.0, google-apis-storage_v1 version 0.18.0, google-cloud-storage version 1.38.0

Steps to reproduce

1. create a file object for a non-existing file in an existing bucket
2. call .exists? on this GCS file
3. an HTTP error is displayed on the terminal output: [2022-08-14T22:22:53.471526 #35151] ERROR -- : Error - #<Google::Apis::ClientError: notFound: No such object: …

Code example

storage = Google::Cloud::Storage.new

bucket = storage.bucket("my_bucket", skip_lookup: true)
file = bucket.file("nonexisting")

assert !file.exists?

Sample output:

E, [2022-08-14T22:22:53.471526 #35151] ERROR -- : Error - #<Google::Apis::ClientError: notFound: No such object: shrine-gcs-test-renchap-shrine-gcs-test2/076b75da175f580d5a76c430dfe911a4 status_code: 404
header: #<HTTP::Message::Headers:0x0000000106daad70 @http_version="1.1", @body_size=0, @chunked=false, @request_method="GET",
@request_uri=#<Addressable::URI:0xa8c URI:https://storage.googleapis.com/storage/v1/b/shrine-gcs-test-renchap-shrine-gcs-test2/o/076b75da175f580d5a76c430dfe911a4?>, @request_query=nil, @request_absolute_uri=nil, @status_code=404, @reason_phrase="Not Found", @body_type=nil, @body_charset=nil, @body_date=nil, @body_encoding=#<Encoding:UTF-8>, @is_request=false,
@header_item=[["X-GUploader-UploadID", "ADPycdv98U_Q_jheQz6qK9W9RKq5htzTS5n4edmpsQm8IKwhWJflCa865Ls05QWW2SafZcPY9tWaYauTBFEjNZWilQlnLuAP7BNQ"], ["Content-Type", "application/json; charset=UTF-8"], ["Date", "Sun, 14 Aug 2022 20:22:53 GMT"], ["Vary", "Origin"], ["Vary", "X-Origin"], ["Cache-Control", "no-cache, no-store, max-age=0, must-revalidate"], ["Expires", "Mon, 01 Jan 1990 00:00:00 GMT"], ["Pragma", "no-cache"], ["Content-Length", "353"], ["Server", "UploadServer"], ["Alt-Svc", "h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000,h3-Q050=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000,quic=\":443\"; ma=2592000; v=\"46,43\""]], @dumped=false> body: "{\n  \"error\": {\n    \"code\": 404,\n    \"message\": \"No such object: shrine-gcs-test-renchap-shrine-gcs-test2/076b75da175f580d5a76c430dfe911a4\",\n    \"errors\": [\n      {\n        \"message\": \"No such object: shrine-gcs-test-renchap-shrine-gcs-test2/076b75da175f580d5a76c430dfe911a4\",\n        \"domain\": \"global\",\n        \"reason\": \"notFound\"\n      }\n    ]\n  }\n}\n">

I am not expecting an error to be displayed here. The file is expected to not exist, this log is not useful as it is not a real error but the expected behaviour.

Source

This issue started to appear with google-apis-core version 0.7.0 , probably introduced by #10542.
It does not occur when I force google-apis-core to 0.6.0.

[bajajneha27]

Hi @renchap
Thanks for reaching out to us.
I actually get the error message when I try to fetch a non-existing file i.e. file = bucket.file("nonexisting") with this code.

Also, if I try to fetch a non-existing file, it ultimately returns nil . So the variable file is nil and hence the exists? method won't work.
I'm confused with the statement create a file object for a non-existing file in an existing bucket . Can you please elaborate if you actually created the file or not?

[renchap]

I'm confused with the statement create a file object for a non-existing file in an existing bucket . Can you please elaborate if you actually created the file or not?

The object in GCS is not created. This is what the code I used above does:

bucket = storage.bucket("my_bucket", skip_lookup: true)
file = bucket.file("nonexisting") # here there is no `gs://my_bucket/nonexisting` object in GCS

[bajajneha27]

Thanks for clarifying. I think this is a bug, thank you for bringing this to our notice. I'm trying to reproduce this and let me get back on this.

[frankyn]

Hi @bajajneha27 what is the status of this issue?

[bajajneha27]

Hi @frankyn and @renchap ,
Sorry for the delay. I'm still not able to reproduce this.

Even with 0.6.0 version of google-apis-core library, I get nil for a non-existing file.

file = bucket.file "nonexisting" #nil

@renchap Do you expect that the object gets created when you call above code. Would you be able to share the code snippet of what exactly you expect here?

[renchap]

@bajajneha27 I am not expecting the object to be created by the call. The behaviour of this call is correct, it returns true / false depending on the object existence.

BUT the issue here is that an error is logged for each of these calls when the object does not exists, which is flooding our logger as we often check for existence of non-existing files.

Before 0.7.0, no logs were emitted when calling bucket.file("nonexisting").exists? on an object that does not exists in GCS.

[bajajneha27]

Thanks for clarifying, @renchap
We figured that it's because of this PR. We're planning to revert it back for now. Once that's released, you won't see those noisy logs. Sorry for this!

[bajajneha27]

Gem with bug fix is released. The issue should be resolved now. I'll go ahead and close this issue. Please feel free to reopen it if issue still persists.