Authentication error is not bubbling up
Closed this issue · 13 comments
If a user isn't authenticated to the Google Cloud API, then running any fissfc command would result in the following error, which doesn't inform the authentication issue. For example:
$ fissfc monitor -p nci-mnoble-bi-org -w dev
Traceback (most recent call last):
File "/xchip/tcga/Tools/gdac/gdops_1.6.0/lib/python2.7/site-packages/firecloud/fiss.py", line 1579, in __pretty_print_fc_exception
e = json.loads(e.args[0])
IndexError: tuple index out of range
ResponseNotReady:To reproduce the issue, delete the following file so the user isn't authenticated anymore:
~/.config/gcloud/application_default_credentials.json
Then run:
fissfc monitor -p nci-mnoble-bi-org -w dev
Thank you! However, when I remove my credentials and issue that command I receive the following stack trace
fissfc monitor -p nci-mnoble-bi-org -w dev Traceback (most recent call last): File "/usr/local/lib/python2.7/site-packages/firecloud/fiss.py", line 1577, in __pretty_print_fc_exception e = json.loads(e.args[0]) File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 339, in loads return _default_decoder.decode(s) ValueError: No JSON object could be decoded ApplicationDefaultCredentialsError: The Application Default Credentials are not available. They are available if running in Google Compute Engine. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
which (although still uglier than I'd hoped the _pretty_print function would be) does indicate the source of the error. My FISS version is 0.16.4 ... what's yours?
FWIW: I just pushed to master an update to fiss.py to make this case even prettier at the CLI ... now, as elsewhere in the code, the "ugly" stack trace will only be burped to the screen when --verbose flag has been specified ... so, when it's not --verbose then all you should see is
ApplicationDefaultCredentialsError: The Application Default Credentials are not available. They are available if running in Google Compute Engine. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
I was able to reproduce this issue on cga3, and the FISS version is 0.16.4 as well. Did you run it on cga3?
No, I was using my laptop (Python 2.7.12)
On CGA3 I see the same result as I did on my laptop (posted 2 hours ago, before pushing "even prettier" commit to master, which has not been deployed yet on CGA3 et al servers):
`/home/unix/mnoble(cga03) fissfc monitor -p nci-mnoble-bi-org -w dev
Traceback (most recent call last):
File "/xchip/tcga/Tools/gdac/gdops_1.6.0/lib/python2.7/site-packages/firecloud/fiss.py", line 1579, in __pretty_print_fc_exception
e = json.loads(e.args[0])
File "/xchip/tcga/Tools/python_builds/python__2.7.12/lib/python2.7/json/init.py", line 339, in loads
return _default_decoder.decode(s)
ValueError: No JSON object could be decoded
ApplicationDefaultCredentialsError: The Application Default Credentials are not available. They are available if running in Google Compute Engine. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.`
path to trace.log: /home/unix/vicky/trace.log
(command used: strace -f 2>trace.log fissfc monitor -p nci-mnoble-bi-org -w dev)
I can't read that file :)
Would you please adjust your umask setting (at login) to 0002?
Mine was set to 002. The problem was the permission on my root directory. I fixed it. Can you try again?
Got it, and was finally able to reproduce
I've done some digging here, and FISS is telling you absolutely everything it knows about the error state. You can verify this by inspecting
/tcga/Tools/python_builds/python__2.7.12/lib/python2.7/httplib.py
(e.g. on cga3 server) and observing that the exception class is just a stub
class ResponseNotReady(ImproperConnectionState):
pass
with no message, etc. So the exception name is all that FISS can report when it catches and indicates the exception on the CLI.
In other words, the error is bubbling up as much as it can be bubbled. And we have a different, larger question to consider: is FISS employing the appropriate strategy for credentialing? If so, then when we delete the application default credentials file then this is the best error we're going to get.
David, I've added you to this issue so you can weigh in on the credentialing. As you can see from my last comment, I've put to rest the suspicion that the exception is not being exposed at the top level to the user--it is, but happens to not be very informative.