Problem with ee Authenticate
Vincent-Nossek opened this issue · 21 comments
Environment Information
I run the following code under google Colab on Chrome navigator.
Description
I was trying to run a Colab notebook, that worked without issues previously one day ago, and I can not authentificate as I get the following error when I click on the "Generate token"
I also tried to run pip install pygis but had the same issue.
A similar issue has been reported very recently under the "Discussions" tab from another user who have the same error but run the code through JupyterLab.
What I Did
import ee
ee.Authenticate()
I can confirm that ee.Authenticate() is broken for local Jupyter. It takes about 10 seconds to open the login page in the browser, which is significantly slower than previous versions. After that, it throws the invalid request error. Authentication can't proceed. The same procesure works fine on Colab.
Since earthengine-api v0.1.383, users need to specify the project id for ee.Initialize(), which is an extra barrier for ee Python users. Most Python users only have one cloud project (the default cloud project). It would be great if ee.Initialize() can use the default cloud project under the user' account so that users don't have to specify the project id explicitly. It makes more sense to specify a project id only when a user has multiple cloud projects. Otherwise, the breaks most exsiting ee Python code.
ee.Initialize(project='my-project-id')Hi @giswqs
Thank you for fixing this issue. We encountered the same issue in our project and used your solution, ee.Authenticate(auth_mode='localhost') to fix it. It seems that specifying the project id to initialize with the code ee.Initialize(project='my-project-id') only needs to be used for remote environments like google colab, but not local environments. For our project the code snippet below worked without having to specify a project id. I'm not sure exactly why this is the case.
ee.Authenticate(auth_mode='localhost')
ee.Initialize()This bug has been fixed. Run pip install -U geemap and restart the kernel to take effect.
Running the following code should authenticate and initialize ee automatically. No need to run ee.Authenticate() and ee.Initialize() explictly.
import geemap
m = geemap.Map()
m@jdbcode @naschmitz @schwehr Note that ee.Authenticate(auth_mode='notebook') does not work any more. It needs to be fixed in the earthengine-api package.
Thank you so much!
@jdbcode @naschmitz @schwehr Note that ee.Authenticate(auth_mode='notebook') does not work any more. It needs to be fixed in the earthengine-api package.
Noted, we're looking, thanks for notifying!
For those that are trying to ee.Authenticate() directly in a local Jupyter notebook (as opposed to allowing geemap to attempt it indirectly: m = geemap.Map()), use ee.Authenticate(auth_mode='localhost') for now, despite the auth docs currently saying that auth_mode='notebook' should work.
Also note that ee.Authenticate() now takes about 10 seconds before openning up the browser. It used to be very fast, less than one second.
Nice! Thanks also for adding the if for Colab!
However, I think the need for a project in initialization is also causing problems.
Testing from Colab:
(I made sure that it is not accessing stored credentials - deleted them temporarily)
!pip install -U geemap
import geemap
m = geemap.Map()
m
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
[<ipython-input-11-ef5c07d70876>](https://localhost:8080/#) in <cell line: 1>()
----> 1 m = geemap.Map()
2 m
[/usr/local/lib/python3.10/dist-packages/geemap/geemap.py](https://localhost:8080/#) in __init__(self, **kwargs)
155 # Default reducer to use
156 if kwargs["ee_initialize"]:
--> 157 self.roi_reducer = ee.Reducer.mean()
158 self.roi_reducer_scale = None
159
AttributeError: module 'ee' has no attribute 'Reducer'
I don't think it's initializing successfully and not showing the init error in stack trace.
Issue tracker for auth_mode='notebook' not working: https://issuetracker.google.com/319451094
I can confirm that the colab auth is broken for geemap. Looking into it now.
Here is the workaround right now. I think the requirement for specifying a cloud project explicitly is a barrier for Python users. Most ee Python users probably only has one cloud project under their account, with default name like ee-<user-id>. It would be easier if ee.Initialize() user the default project for initialization. In this way, users don't have to login the Code Editor or Google Cloud to check the project id. If there are multiple projects or if ee API is not enabled for the default project, it then makes sense to throw the error message.
import geemap
geemap.ee_initialize(project='<your-project-id>')
m = geemap.Map()
mThe requirement for specifying a project-id will break all python examples in the documentation. Colab users will have to do one of the following to authenticate and initialize a session. It would be much easier if ee.Initialize() just uses the user's default project rather than having to specify it explictly. In this way, it will not break existing code.
import ee
import geemap
ee.Authenticate(auto_mode='colab')
ee.Iinitialize(project='<your-project-id>')
m = geemap.Map()
mimport geemap
geemap.ee_initialize(project='<your-project-id>')
m = geemap.Map()
mhttps://developers.google.com/earth-engine/guides/image_visualization#colab-python
@jdbcode Do you know what is the Python eqivalent to !gcloud projects list? If the user only has one cloud project, it can be used as the default project for ee initialize. Otherwise, print out the list of project ids and alert users to choose a proper project id from the list to initialize. This is only need for Colab.
The auth_mode='notebook' stills works for Colab, but not for local jupyter.
However, auth_mode='colab' fails if the project id is not specified. Personally, I would prefer auth_mode='notebook' because users don't have to leave Colab to go to the Code Editor or Google Cloud to find out the project id.
ee.Authenticate(auth_mode='notebook') is working again for local Jupyter. It will allow you to select a project, which means you don't need to specify one in call to ee.Initialize().
https://issuetracker.google.com/319451094
If you are running locally though you might do auth and project setting through the command line once to save the credentials and then you only need to call ee.Initialize() (without specifying project) or create a map in geemap for each new EE session. Read more here: https://developers.google.com/earth-engine/guides/auth#python_and_command_line
earthengine authenticate
earthengine set_project YOUR-EE-PROJECT
The requirement for specifying a project-id will break all python examples in the documentation. Colab users will have to do one of the following to authenticate and initialize a session. It would be much easier if
ee.Initialize()just uses the user's default project rather than having to specify it explictly. In this way, it will not break existing code.
@giswqs - I agree, for a large group of users, having to specify a project in ee.Initialize(project='YOUR-EE-PROJECT') is serious friction to getting going. With ee.Authenticate(auth_mode='notebook') that should not be a problem because they can select the project in the Auth UI and not need it when calling ee.Initialize(). Cloud platform and projects are confusing, but they are here to stay, so I'm a little reluctant to hide them too much.
What I'd really like are for people to be able to access the Colab secrets through an API that allows a person to write for example earth engine credentials with the default project as an attribute. (so people don't have to run it in the command line locally and then copy/paste into colab) - then they should only have to ee.Initialize() I think. We've heard from the Colab team that something like this might be possible, but need them to make changes, which could take some time.