404 error when reading a blob from a bucket

[ceteri]

Here's an object in GCS that's public gs://kglab-tutorial/foaf.rdf with the URL https://storage.googleapis.com/kglab-tutorial/foaf.rdf

I can access that via a browser, and also simply using curl:

!curl https://storage.googleapis.com/kglab-tutorial/foaf.rdf

which returns the contents correctly.

However, when I try to instantiate a Pathy object:

import pathy

path = pathy.Pathy("gs://kglab-tutorial/foaf.rdf")

with path.open() as f:
    print(f.read())

this throws a ClientError:

---------------------------------------------------------------------------
ClientError                               Traceback (most recent call last)
<ipython-input-14-d8941c1e5e8e> in <module>
      3 path = pathy.Pathy("gs://kglab-tutorial/foaf.rdf")
      4 
----> 5 with path.open() as f:
      6     print(f.read())

/opt/anaconda3/lib/python3.7/site-packages/pathy/base.py in open(self, mode, buffering, encoding, errors, newline)
    643             encoding=encoding,
    644             errors=errors,
--> 645             newline=newline,
    646         )
    647 

/opt/anaconda3/lib/python3.7/site-packages/pathy/base.py in open(self, path, mode, buffering, encoding, errors, newline)
    380             encoding=encoding,
    381             errors=errors,
--> 382             newline=newline,
    383         )
    384 

/opt/anaconda3/lib/python3.7/site-packages/pathy/file.py in open(self, path, mode, buffering, encoding, errors, newline)
    124     ) -> StreamableType:
    125         if self.lookup_bucket(path) is None:
--> 126             raise ClientError(message=f'bucket "{path.root}" does not exist', code=404)
    127 
    128         full_path = self.full_path(path)

ClientError: (404) bucket "kglab-tutorial" does not exist

Is there any other configuration which I've missed?

This with both pathy-0.3.4 and pathy-0.3.3
using Py 3.7.4 on macOS

Thank you -

[justindujardin]

Hi @ceteri, sorry for the slow response. I tried out your snippet, and it works for me. If I had to guess, I'd say that you've probably got a credentials problem with google's cloud SDK.

Is it possible you're having the same trouble as in #43 ?

I'd love to add better error handling, if you can help me understand what's failing for you specifically, and confirm you have valid credentials. Can you run gsutil cp gs://kglab-tutorial/foaf.rdf ./ without error?

[justindujardin]

I'm going to close this issue for now. If you still have problems feel free to reopen it.