google-api-core was not found on production
Closed this issue · 11 comments
I got following error when I accessed to the end point of flask handler.
2017-11-05T08:30:31.471Z - error: exec(bytecode, module.__dict__)
File "site-packages/google/cloud/storage/__init__.py", line 39, in <module>
File "/Users/seijik/.pyenv/versions/venv27/lib/python2.7/site-packages/PyInstaller/loader/pyimod03_importers.py", line 389, in load_module
exec(bytecode, module.__dict__)
File "site-packages/google/cloud/storage/bucket.py", line 24, in <module>
File "/Users/seijik/.pyenv/versions/venv27/lib/python2.7/site-packages/PyInstaller/loader/pyimod03_importers.py", line 389, in load_module
exec(bytecode, module.__dict__)
File "site-packages/google/api_core/__init__.py", line 23, in <module>
File "site-packages/pkg_resources/__init__.py", line 562, in get_distribution
File "site-packages/pkg_resources/__init__.py", line 436, in get_provider
File "site-packages/pkg_resources/__init__.py", line 981, in require
File "site-packages/pkg_resources/__init__.py", line 867, in resolve
pkg_resources.DistributionNotFound: The 'google-api-core' distribution was not found and is required by the application
Failed to execute script main
Build command is following.
py-cloud-fn video http -p -f main.py && cd cloudfn/target && gcloud beta functions deploy video --trigger-http --stage-bucket mybucket && cd ../..
Once I added following line to cloudfn/hooks/hook-google.cloud.storage.py
It was fixed on emulator. However I still have same error on production.
datas += copy_metadata('google-api-core')
Do you have any idea? I guess some google lib changed dependency but not certain at the moment.
I'm using following google libraries
google-auth==1.0.1
google-cloud==0.27.0
google-cloud-bigquery==0.28.0
google-cloud-storage==1.6.0
macOS Sierra version 10.12.6
Python 2.7.10
Thanks!
I'm experiencing a similar problem (both under Py 2.7 & 3.5), but with the object storage access (using the storage example as base). I'm still figuring out if it's my own fault, but I've been trying for a full day. (Disclaimer: Python is not my main language).
2.7
D example_bucket 175933831454062 2017-11-27 17:53:11.306 Function execution started
E example_bucket 175933831454062 2017-11-27 17:53:14.676 Traceback (most recent call last):
File "function.py", line 3, in <module>
File "/tmp/pip-build-9fSBYr/pyinstaller/PyInstaller/loader/pyimod03_importers.py", line 389, in load_module
File "site-packages/google/cloud/storage/__init__.py", line 37, in <module>
File "/tmp/pip-build-9fSBYr/pyinstaller/PyInstaller/loader/pyimod03_importers.py", line 389, in load_module
File "site-packages/google/cloud/storage/batch.py", line 30, in <module>
File "/tmp/pip-build-9fSBYr/pyinstaller/PyInstaller/loader/pyimod03_importers.py", line 389, in load_module
File "site-packages/google/cloud/exceptions.py", line 24, in <module>
File "/tmp/pip-build-9fSBYr/pyinstaller/PyInstaller/loader/pyimod03_importers.py", line 389, in load_module
File "site-packages/google/api_core/__init__.py", line 23, in <module>
File "site-packages/pkg_resources/__init__.py", line 564, in get_distribution
File "site-packages/pkg_resources/__init__.py", line 436, in get_provider
File "site-packages/pkg_resources/__init__.py", line 984, in require
File "site-packages/pkg_resources/__init__.py", line 870, in resolve
pkg_resources.DistributionNotFound: The 'google-api-core' distribution was not found and is required by the application
Failed to execute script function
E example_bucket 175933831454062 2017-11-27 17:53:14.699 Error: Function failed
at /user_code/index.js:93:14
D example_bucket 175933831454062 2017-11-27 17:53:14.706 Function execution took 3401 ms, finished with status: 'error'
3.5
[...]
E example_bucket 175931674995184 2017-11-27 17:27:45.152 Failed to execute script function
Traceback (most recent call last):
File "function.py", line 3, in <module>
File "<frozen importlib._bootstrap>", line 969, in _find_and_load
File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 664, in _load_unlocked
File "<frozen importlib._bootstrap>", line 634, in _load_backward_compatible
File "/app/cloudfn/pip-cache-3.5/lib/python3.5/site-packages/PyInstaller/loader/pyimod03_importers.py", line 389, in load_module
File "site-packages/google/cloud/storage/__init__.py", line 37, in <module>
File "<frozen importlib._bootstrap>", line 969, in _find_and_load
File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 664, in _load_unlocked
File "<frozen importlib._bootstrap>", line 634, in _load_backward_compatible
File "/app/cloudfn/pip-cache-3.5/lib/python3.5/site-packages/PyInstaller/loader/pyimod03_importers.py", line 389, in load_module
File "site-packages/google/cloud/storage/batch.py", line 30, in <module>
File "<frozen importlib._bootstrap>", line 969, in _find_and_load
File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 664, in _load_unlocked
File "<frozen importlib._bootstrap>", line 634, in _load_backward_compatible
File "/app/cloudfn/pip-cache-3.5/lib/python3.5/site-packages/PyInstaller/loader/pyimod03_importers.py", line 389, in load_module
File "site-packages/google/cloud/exceptions.py", line 24, in <module>
File "<frozen importlib._bootstrap>", line 969, in _find_and_load
File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 664, in _load_unlocked
File "<frozen importlib._bootstrap>", line 634, in _load_backward_compatible
File "/app/cloudfn/pip-cache-3.5/lib/python3.5/site-packages/PyInstaller/loader/pyimod03_importers.py", line 389, in load_module
File "site-packages/google/api_core/__init__.py", line 23, in <module>
File "site-packages/pkg_resources/__init__.py", line 564, in get_distribution
File "site-packages/pkg_resources/__init__.py", line 436, in get_provider
File "site-packages/pkg_resources/__init__.py", line 984, in require
File "site-packages/pkg_resources/__init__.py", line 870, in resolve
pkg_resources.DistributionNotFound: The 'google-api-core' distribution was not found and is required by the application
E example_bucket 175931674995184 2017-11-27 17:27:45.168 Error: Function failed
at /user_code/index.js:93:14
at process._tickDomainCallback (internal/process/next_tick.js:135:7)
D example_bucket 175931674995184 2017-11-27 17:27:45.173 Function execution took 1916 ms, finished with status: 'error'
@seijik42 : Are you still having the issue or have you found the cause?
A new version has been uploaded to PyPI (see #26) and perhaps helps you solve it. Can you test it under 3.5? I would like to understand better if it's the same problem and we can work together or if I have to submit a separate issue.
@pataquets Not yet found the cause. Actually I was away from this for a while since this might be fixed by some library updates.
Thank you for suggestion. I will try when I have time, probably this weekend.
Great. Let's keep this updated on progress to add efforts.
@MartinSahlen : Any advice on this?
BTW, I tried the following:
- Deleted the
/usr/local/lib/python2.7/dist-packages/cloudfn/cli.pyc. - Added an extra
--hidden-importclause forgoogle-api-coretobuild()function in file/usr/local/lib/python2.7/dist-packages/cloudfn/cli.py. - Tested again for 2.7 and 3.5.
No success. :(
I will look into this. Sorry for the lag, I am very time constrained at the moment.
@pataquets
Sorry for late, I could spend some time for this yesterday. I think I could fix this issue.
What I did is:
- Update to latest master
- Added
datas += copy_metadata('google-api-core')to all relevant hook files - Added "google-api-core" to requirements.txt
Earlier, I changed hook files in wrong place. I needed to change files under cloudfn cache folder.
I think this will be fixed once master branch is fixed.
I'm not sure (1) is required.
I didn't have time to create PR for this but should be strait forward.
Now I'm facing another error message but I think it is separate and my specific issue.
[Edited]
I'm also trying to use the google-cloud-storage api, and still getting the same error:
- Added datas += copy_metadata('google-api-core') to hooks/hook-google.cloud.storage.py
- Added google-api-core to requirements.txt
Anything else we should try out?
Thanks!
Marcelo
Not to spam this issue, but I am getting this as well. Any insight would be greatly appreciate when you have time!
Traceback (most recent call last):
File "handler.py", line 3, in <module>
File "/tmp/pip-build-7Dv_nn/pyinstaller/PyInstaller/loader/pyimod03_importers.py", line 389, in load_module
File "site-packages/google/cloud/storage/__init__.py", line 37, in <module>
File "/tmp/pip-build-7Dv_nn/pyinstaller/PyInstaller/loader/pyimod03_importers.py", line 389, in load_module
File "site-packages/google/cloud/storage/batch.py", line 30, in <module>
File "/tmp/pip-build-7Dv_nn/pyinstaller/PyInstaller/loader/pyimod03_importers.py", line 389, in load_module
File "site-packages/google/cloud/exceptions.py", line 24, in <module>
File "/tmp/pip-build-7Dv_nn/pyinstaller/PyInstaller/loader/pyimod03_importers.py", line 389, in load_module
File "site-packages/google/api_core/__init__.py", line 23, in <module>
File "site-packages/pkg_resources/__init__.py", line 564, in get_distribution
File "site-packages/pkg_resources/__init__.py", line 436, in get_provider
File "site-packages/pkg_resources/__init__.py", line 984, in require
File "site-packages/pkg_resources/__init__.py", line 870, in resolve
pkg_resources.DistributionNotFound: The 'google-api-core' distribution was not found and is required by the application
Failed to execute script handler
Hey all, it seems the issue might be fixed if I add google-api-core to the hooks file for all google cloud libs?
hey @MartinSahlen I tried that (with only one google lib, added google-api-core to the hook file and to requirements) still the same error ...
Thanks
Gents, I'm experiencing the same problem, even with this fix. Any suggestions, @MartinSahlen ?
Thanks a lot