geemap.ee_export_image() Report: "An error occurred while downloading." Without any instructions.
Pluto-Nemo opened this issue · 8 comments
Pluto-Nemo commented
Environment Information
System: Win10, Python: 3.8.15, geemap: 0.17.3
Please run the following code on your computer and share the output with us so that we can better debug your issue:
Source Code:
import os
import pandas as pd
import time
os.environ['HTTP_PROXY'] = 'http://127.0.0.1:7890'
os.environ['HTTPS_PROXY'] = 'http://127.0.0.1:7890'
import ee
import geemap
ee.Initialize()
SA = ee.FeatureCollection("projects/ee-2402214215/assets/srtp/SA")
vis_params = {'color': '000000ff', 'width': 2, 'lineType': 'solid', 'fillColor': 'ffffff44'}
Map.addLayer(SA, vis_params, "SA")
bands = ['sur_refl_b01', 'sur_refl_b02', 'sur_refl_b03', 'sur_refl_b04', 'sur_refl_b05', 'sur_refl_b06',
'sur_refl_b07', 'QC_500m', 'state_1km']
date1 = ee.Date("2015-01-01")
date2 = ee.Date("2015-01-05")
ModisCollection = ee.ImageCollection("MODIS/006/MOD09GA") \
.filterDate(date1, date2) \
.filterBounds(SA) \
.select(bands)
BitMask_Cloud = 1 << 10
def cloudfree(image):
qc = image.select('QC_500m')
state = image.select('state_1km')
cloud = state.bitwiseAnd(BitMask_Cloud)
# quality = qc.bitwiseAnd()
mask = cloud.eq(0)
return image.updateMask(mask)
ModisCollection = ModisCollection.map(cloudfree)
ModisList = ModisCollection.toList(ModisCollection.size().getInfo())
for i in range(ModisCollection.size().getInfo()):
image = ee.Image(ModisList.get(i))
# print(image)
image = image.clipToCollection(SA)
geemap.ee_export_image(image, filename=image.get('system:index').getInfo()+".tif", region=SA.geometry(), scale=500, crs='EPSG:4326', format='GeoTIFF', file_per_band=False)Description
The code above is used to download the Modis image without the cloud and the image has been clipped by the shapefile I uploaded to google drive.
I encountered a strange problem while using the ee_export_image() function. I've searched the issues to find out how to resolve my problem. But I cannot find the same situation. The area of the image is actually a small one.
What I Did
What I did is just run the code above.
It is the crash report without any instruction that confused me.
Output & Report
Generating URL ...
Downloading data from https://earthengine.googleapis.com/v1alpha/projects/earthengine-legacy/thumbnails/7957bc8a2fad79b4f53aafbdc696912b-1885dfcdf56be18f85d05caeefdab48c:getPixels
Please wait ...
An error occurred while downloading.
Generating URL ...
Downloading data from https://earthengine.googleapis.com/v1alpha/projects/earthengine-legacy/thumbnails/7212ba8a1e8310bd08a1c0c455c29306-380bb050bd2f628848ddd241b3304b35:getPixels
Please wait ...
An error occurred while downloading.
Generating URL ...
Downloading data from https://earthengine.googleapis.com/v1alpha/projects/earthengine-legacy/thumbnails/9ef45b85efb00e7f9a064290eefd2e75-6471592e569c75f4deb8a9ba87903101:getPixels
Please wait ...
An error occurred while downloading.
Generating URL ...
Downloading data from https://earthengine.googleapis.com/v1alpha/projects/earthengine-legacy/thumbnails/d1387bb5064e88eb6e5d88a6bf9c5a12-44268affb24903a195fdd726f496a61e:getPixels
Please wait ...
An error occurred while downloading.
giswqs commented
Rerun your code and click one of the hyperlinks in the output. It will show you what error it is.
Pluto-Nemo commented
Thanks so much for your reply!
Report:
{
"error": {
"code": 401,
"message": "Invalid token: 1885dfcdf56be18f85d05caeefdab48c",
"status": "UNAUTHENTICATED"
}
}
Excuse me but as I know the token is available. So how can I solve it?
Pluto-Nemo commented
Additionally, I also found some different report like this:
{
"error": {
"code": 400,
"message": "User memory limit exceeded.",
"status": "INVALID_ARGUMENT"
}
}
OR
---------------------------------------------------------------------------
RemoteDisconnected Traceback (most recent call last)
File D:\Anaconda\envs\GEE\lib\site-packages\urllib3\connectionpool.py:703, in HTTPConnectionPool.urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
702 # Make the request on the httplib connection object.
--> 703 httplib_response = self._make_request(
704 conn,
705 method,
706 url,
707 timeout=timeout_obj,
708 body=body,
709 headers=headers,
710 chunked=chunked,
711 )
713 # If we're going to release the connection in ``finally:``, then
714 # the response doesn't need to know about the connection. Otherwise
715 # it will also try to release it and we'll have a double-release
716 # mess.
File D:\Anaconda\envs\GEE\lib\site-packages\urllib3\connectionpool.py:449, in HTTPConnectionPool._make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
445 except BaseException as e:
446 # Remove the TypeError from the exception chain in
447 # Python 3 (including for exceptions like SystemExit).
448 # Otherwise it looks like a bug in the code.
--> 449 six.raise_from(e, None)
450 except (SocketTimeout, BaseSSLError, SocketError) as e:
File <string>:3, in raise_from(value, from_value)
File D:\Anaconda\envs\GEE\lib\site-packages\urllib3\connectionpool.py:444, in HTTPConnectionPool._make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
443 try:
--> 444 httplib_response = conn.getresponse()
445 except BaseException as e:
446 # Remove the TypeError from the exception chain in
447 # Python 3 (including for exceptions like SystemExit).
448 # Otherwise it looks like a bug in the code.
File D:\Anaconda\envs\GEE\lib\http\client.py:1348, in HTTPConnection.getresponse(self)
1347 try:
-> 1348 response.begin()
1349 except ConnectionError:
File D:\Anaconda\envs\GEE\lib\http\client.py:316, in HTTPResponse.begin(self)
315 while True:
--> 316 version, status, reason = self._read_status()
317 if status != CONTINUE:
File D:\Anaconda\envs\GEE\lib\http\client.py:285, in HTTPResponse._read_status(self)
282 if not line:
283 # Presumably, the server closed the connection before
284 # sending a valid response.
--> 285 raise RemoteDisconnected("Remote end closed connection without"
286 " response")
287 try:
RemoteDisconnected: Remote end closed connection without response
During handling of the above exception, another exception occurred:
MaxRetryError Traceback (most recent call last)
File D:\Anaconda\envs\GEE\lib\site-packages\requests\adapters.py:489, in HTTPAdapter.send(self, request, stream, timeout, verify, cert, proxies)
488 if not chunked:
--> 489 resp = conn.urlopen(
490 method=request.method,
491 url=url,
492 body=request.body,
493 headers=request.headers,
494 redirect=False,
495 assert_same_host=False,
496 preload_content=False,
497 decode_content=False,
498 retries=self.max_retries,
499 timeout=timeout,
500 )
502 # Send the request.
503 else:
File D:\Anaconda\envs\GEE\lib\site-packages\urllib3\connectionpool.py:787, in HTTPConnectionPool.urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
785 e = ProtocolError("Connection aborted.", e)
--> 787 retries = retries.increment(
788 method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
789 )
790 retries.sleep()
File D:\Anaconda\envs\GEE\lib\site-packages\urllib3\util\retry.py:592, in Retry.increment(self, method, url, response, error, _pool, _stacktrace)
591 if new_retry.is_exhausted():
--> 592 raise MaxRetryError(_pool, url, error or ResponseError(cause))
594 log.debug("Incremented Retry for (url='%s'): %r", url, new_retry)
MaxRetryError: HTTPSConnectionPool(host='earthengine.googleapis.com', port=443): Max retries exceeded with url: /v1alpha/projects/earthengine-legacy/thumbnails/9ef45b85efb00e7f9a064290eefd2e75-cd35af05535e039d8dc056000e7c83b6:getPixels (Caused by ProxyError('Cannot connect to proxy.', RemoteDisconnected('Remote end closed connection without response')))
During handling of the above exception, another exception occurred:
ProxyError Traceback (most recent call last)
File D:\Anaconda\envs\GEE\lib\site-packages\geemap\common.py:2260, in ee_export_image(ee_object, filename, scale, crs, crs_transform, region, dimensions, file_per_band, format, unmask_value, timeout, proxies)
2259 print(f"Downloading data from {url}\nPlease wait ...")
-> 2260 r = requests.get(url, stream=True, timeout=timeout, proxies=proxies)
2262 if r.status_code != 200:
File D:\Anaconda\envs\GEE\lib\site-packages\requests\api.py:73, in get(url, params, **kwargs)
63 r"""Sends a GET request.
64
65 :param url: URL for the new :class:`Request` object.
(...)
70 :rtype: requests.Response
71 """
---> 73 return request("get", url, params=params, **kwargs)
File D:\Anaconda\envs\GEE\lib\site-packages\requests\api.py:59, in request(method, url, **kwargs)
58 with sessions.Session() as session:
---> 59 return session.request(method=method, url=url, **kwargs)
File D:\Anaconda\envs\GEE\lib\site-packages\requests\sessions.py:587, in Session.request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
586 send_kwargs.update(settings)
--> 587 resp = self.send(prep, **send_kwargs)
589 return resp
File D:\Anaconda\envs\GEE\lib\site-packages\requests\sessions.py:701, in Session.send(self, request, **kwargs)
700 # Send the request
--> 701 r = adapter.send(request, **kwargs)
703 # Total elapsed time of the request (approximately)
File D:\Anaconda\envs\GEE\lib\site-packages\requests\adapters.py:559, in HTTPAdapter.send(self, request, stream, timeout, verify, cert, proxies)
558 if isinstance(e.reason, _ProxyError):
--> 559 raise ProxyError(e, request=request)
561 if isinstance(e.reason, _SSLError):
562 # This branch is for urllib3 v1.22 and later.
ProxyError: HTTPSConnectionPool(host='earthengine.googleapis.com', port=443): Max retries exceeded with url: /v1alpha/projects/earthengine-legacy/thumbnails/9ef45b85efb00e7f9a064290eefd2e75-cd35af05535e039d8dc056000e7c83b6:getPixels (Caused by ProxyError('Cannot connect to proxy.', RemoteDisconnected('Remote end closed connection without response')))
During handling of the above exception, another exception occurred:
UnboundLocalError Traceback (most recent call last)
Cell In[2], line 40
38 # print(image)
39 image = image.clipToCollection(SA)
---> 40 geemap.ee_export_image(image, filename=image.get('system:index').getInfo()+".tif", region=SA.geometry(), scale=500, crs='EPSG:4326', format='GeoTIFF', file_per_band=False)
File D:\Anaconda\envs\GEE\lib\site-packages\geemap\common.py:2272, in ee_export_image(ee_object, filename, scale, crs, crs_transform, region, dimensions, file_per_band, format, unmask_value, timeout, proxies)
2270 except Exception as e:
2271 print("An error occurred while downloading.")
-> 2272 print(r.json()["error"]["message"])
2273 return
2275 try:
UnboundLocalError: local variable 'r' referenced before assignment
It is quite comlicated that I cannot work out a single problem. So please help me .....
giswqs commented
The error message indicates that it is your network problem. Under the hood, geemap.ee_export_image() wraps the ee.Image.getDownloadURL() function. If ee.Image.getDownloadURL() does not work, geemap.ee_export_image() won't work either. I would encourage you to use ee.Image.getDownloadURL() directly to download data. This is not really a geemap problem.
https://developers.google.com/earth-engine/apidocs/ee-image-getdownloadurl
Pluto-Nemo commented
Thanks soooo much! Thanks for your advice, I've solved all the problems above!
janjaappape commented
Hi @giswqs ,
I've stumbled upon a very similar problem. The main issue is the size of my Images - which are relatively large. I've tried to create a workaround by incrementally increassing the resolution untill it actually downloads. However, this is a very slow process. Any thoughts on improving this?
ndvi = ndvi.clip(roi).unmask()
ndvi_scale = 250
dres = 50
for i in np.arange(10,100,10):
try:
ndvi.getDownloadURL(dict(scale=ndvi_scale))
geemap.ee_export_image(ndvi, filename=fname, scale=ndvi_scale, region=roi, file_per_band=False)
break
except:
ndvi_scale = ndvi_scale + dres
print("Scale too small for current input area, increasing resolution to ", ndvi_scale, " m")```
Where ndvi is an average MODIS NDVI ee.Image and roi is a simple square geomtry to bound the Image.
giswqs commented
Try the geemap.download_ee_image() function
janjaappape commented
Great, I had to install geedim first, but saved my day! Thanks a lot!