hikmahealth/hikma-health-backend

Unable to run application in dev environment - lack of GCP creds

Opened this issue · 3 comments

tanjinP commented

Expected Behavior

Run the application on my local with no errors

Current Behavior

Application throws exception when trying to start the application:

app_1  | [2020-05-07 04:25:04 +0000] [12] [ERROR] Exception in worker process
app_1  | Traceback (most recent call last):
app_1  |   File "/usr/local/lib/python3.7/dist-packages/gunicorn/arbiter.py", line 583, in spawn_worker
app_1  |     worker.init_process()
app_1  |   File "/usr/local/lib/python3.7/dist-packages/gunicorn/workers/base.py", line 129, in init_process
app_1  |     self.load_wsgi()
app_1  |   File "/usr/local/lib/python3.7/dist-packages/gunicorn/workers/base.py", line 138, in load_wsgi
app_1  |     self.wsgi = self.app.wsgi()
app_1  |   File "/usr/local/lib/python3.7/dist-packages/gunicorn/app/base.py", line 67, in wsgi
app_1  |     self.callable = self.load()
app_1  |   File "/usr/local/lib/python3.7/dist-packages/gunicorn/app/wsgiapp.py", line 52, in load
app_1  |     return self.load_wsgiapp()
app_1  |   File "/usr/local/lib/python3.7/dist-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
app_1  |     return util.import_app(self.app_uri)
app_1  |   File "/usr/local/lib/python3.7/dist-packages/gunicorn/util.py", line 350, in import_app
app_1  |     __import__(module)
app_1  |   File "/app/app.py", line 4, in <module>
app_1  |     from photos.photos_api import photos_api
app_1  |   File "/app/photos/photos_api.py", line 5, in <module>
app_1  |     from photos.photo_storage import store_photo, retrieve_photo
app_1  |   File "/app/photos/photo_storage.py", line 6, in <module>
app_1  |     storage_client = storage.Client()
app_1  |   File "/usr/local/lib/python3.7/dist-packages/google/cloud/storage/client.py", line 110, in __init__
app_1  |     project=project, credentials=credentials, _http=_http
app_1  |   File "/usr/local/lib/python3.7/dist-packages/google/cloud/client.py", line 226, in __init__
app_1  |     _ClientProjectMixin.__init__(self, project=project)
app_1  |   File "/usr/local/lib/python3.7/dist-packages/google/cloud/client.py", line 178, in __init__
app_1  |     project = self._determine_default(project)
app_1  |   File "/usr/local/lib/python3.7/dist-packages/google/cloud/client.py", line 193, in _determine_default
app_1  |     return _determine_default_project(project)
app_1  |   File "/usr/local/lib/python3.7/dist-packages/google/cloud/_helpers.py", line 186, in _determine_default_project
app_1  |     _, project = google.auth.default()
app_1  |   File "/usr/local/lib/python3.7/dist-packages/google/auth/_default.py", line 308, in default
app_1  |     credentials, project_id = checker()
app_1  |   File "/usr/local/lib/python3.7/dist-packages/google/auth/_default.py", line 166, in _get_explicit_environ_credentials
app_1  |     os.environ[environment_vars.CREDENTIALS]
app_1  |   File "/usr/local/lib/python3.7/dist-packages/google/auth/_default.py", line 92, in _load_credentials_from_file
app_1  |     "File {} was not found.".format(filename)
app_1  | google.auth.exceptions.DefaultCredentialsError: File  was not found.
app_1  | [2020-05-07 04:25:04 +0000] [12] [INFO] Worker exiting (pid: 12)
app_1  | [2020-05-07 04:25:04 +0000] [9] [INFO] Shutting down: Master
app_1  | [2020-05-07 04:25:04 +0000] [9] [INFO] Reason: Worker failed to boot.
hikma-health-backend_app_1 exited with code 3

Possible Solution

A couple of suggestions:

  • we can instantiate the Storage client outside the initialization process
  • we can mock the Storage client in the case GCP creds are not available
    • not too familiar with the intricacies of app so not sure how much this would block in terms of local development
  • we mount GCP creds onto the environment (as suggested here)

Steps to Reproduce (for bugs)

  • run docker-compose up after checking out the repo (master)

Context

Your Environment

  • Version used: Latest commit
  • Operating System and version (desktop or mobile):
    • macOS Catalina v 10.15.4
    • Docker version 19.03.8, build afacb8b
sbrother commented

Thanks for the super detailed report. I think the best solution here is to provide a local storage backend that doesn't require GCP creds. Happy to accept PRs, or I can do it when I get a chance next week.

  • 👍1
tanjinP commented

@sbrother thanks for the quick response.

I would like to take a stab at it if you don't mind (see it as an opportunity to get more familiar with Flask).

I'm thinking an if else block on the instantiation of the client based on the presence of the environment variable (GOOGLE_APPLICATION_CREDENTIALS) and to follow a similar pattern in usage (only see this necessary in the method store_photo).

The PR would be an opportunity to continue the discussion.

sbrother commented

That sounds great. I think it would be best to add another config variable called STORAGE_BACKEND here: https://github.com/hikmahealth/hikma-health-backend/blob/master/app/config.py#L16 and have it set to 'local' and dev or 'gcs' in prod. We shouldn't be uploading/downloading photos from GCS during development anyway, so it would be nice to have a local backend even if GOOGLE_APPLICATION_CREDENTIALS happened to be set.

Definitely happy to review your PR when it comes - thanks!

  • 👍1