matplotlib/basemap

arcgisimage - urllib2.HTTPError: HTTP Error 400: Bad Request

mwlock opened this issue · 8 comments

mwlock commented

Using the provided example :

from mpl_toolkits.basemap import Basemap
import matplotlib.pyplot as plt

map = Basemap(llcrnrlon=3.75,llcrnrlat=39.75,urcrnrlon=4.35,urcrnrlat=40.15, epsg=5520)
#http://server.arcgisonline.com/arcgis/rest/services

map.arcgisimage(service='ESRI_Imagery_World_2D', xpixels = 1500, verbose= True)
plt.show()

I get the following error:

http://server.arcgisonline.com/ArcGIS/rest/services/ESRI_Imagery_World_2D/MapServer/export?bbox=89626124.1385,252193030.113,92533661.1526,254772111.328&bboxSR=5520&imageSR=5520&size=1500,1330&dpi=96&format=png32&transparent=true&f=image
Traceback (most recent call last):
  File "plot_map.py", line 73, in <module>
    map.arcgisimage(service='ESRI_Imagery_World_2D', xpixels = 1500, verbose= True)
  File "/home/mwlock2/.local/lib/python2.7/site-packages/mpl_toolkits/basemap/__init__.py", line 4308, in arcgisimage
    img = Image.open(urlopen(basemap_url))
  File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 467, in error
    result = self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 654, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python2.7/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 473, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 400: Bad Request
PowerBroker2 commented

I'm getting a very similar error, too

molinav commented

There seems to be some problem with this concrete ESRI REST API service ESRI_Imagery_World_2D, it might be that ESRI has removed it:
https://www.arcgis.com/home/item.html?id=39758ebd4b9642b79e7a2f54d8871556

Quoted from the link above:

This map is in Extended Support and is no longer updated. Esri recommends that you use World_Imagery instead (ArcGIS 9.3 or higher is required).

The alternative service that they recommend is the World_Imagery service:
https://www.arcgis.com/home/item.html?id=10df2279f9684e4a9f6a7f08febac2a9

This other service seems to work with an example link that I was trying directly on the browser:

https://server.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer/export?bbox=138.4,34.9,139.4,36.0&bboxSR=4269&imageSR=4269&size=2000,2200&dpi=96&format=png32&f=image

image

But if I try with the link of your example and replace ESRI_Imagery_World_2D with World_Imagery, I just get a blank image:

http://server.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer/export?bbox=89626124.1385,252193030.113,92533661.1526,254772111.328&bboxSR=5520&imageSR=5520&size=1500,1330&dpi=96&format=png32&transparent=true&f=image

image

molinav commented

I think that my World_Imagery examples are not behaving properly because the URL attributes bboxSR and imageSR are not set correctly.

molinav commented

If I try to use simple cylindrical coordinates (EPSG=4326), the code snippet works:

from mpl_toolkits.basemap import Basemap
import matplotlib.pyplot as plt

bmap = Basemap(llcrnrlon=3.75, llcrnrlat=39.75, urcrnrlon=4.35, urcrnrlat=40.15, epsg=4326)
bmap.arcgisimage(service='World_Imagery', xpixels=1500, verbose= True)
plt.show()

menorca

So there are at least two problems:

  • One problem is the map service itself, it seems that ESRI_Imagery_World_2D is defunct since May 2022.
  • When trying to use World_Imagery as replacement for ESRI_Imagery_World_2D, there are issues with the EPSG values defined in the URL attributes (exact problem still not identified).
molinav commented

@matthew-william-lock I have found that the following block is causing the problems with the EPSG 5520 coordinates:

basemap/packages/basemap/src/mpl_toolkits/basemap/__init__.py

Lines 4287 to 4289 in f370fd3

if self.projection != 'cyl':
xmin = (180./np.pi)*xmin; xmax = (180./np.pi)*xmax
ymin = (180./np.pi)*ymin; ymax = (180./np.pi)*ymax

I do not have any idea why this if-block is there (it looks like a radian-to-degree conversion), but this recalculation of the coordinates xmin, xmax, ymin and ymax is overwriting the correct transformed coordinates with wrong values. So I am getting a blank image because in fact I am requesting to the REST API one region outside the map.

For the record, if I comment this if-block in my installed basemap library and run the snippet with the new World_Imagery map:

from mpl_toolkits.basemap import Basemap
import matplotlib.pyplot as plt

bmap = Basemap(llcrnrlon=3.75, llcrnrlat=39.75, urcrnrlon=4.35, urcrnrlat=40.15, epsg=5520)
bmap.arcgisimage(service='World_Imagery', xpixels = 1500, verbose= True)
plt.show()

then the verbose output that I get is the following link (note the change in the bbox values):

http://server.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer/export?bbox=1564270.9620172689,4401598.726055895,1615017.0560379555,4446612.184939481&bboxSR=5520&imageSR=5520&size=1500,1330&dpi=96&format=png32&transparent=true&f=image

and the following image (i.e. what you were expecting from the example):

menorca

I would need to inspect why this if-block was added in the past, I have no idea.

WeatherGod commented

A good place to look to find out what exactly things are in which situation is the __init__() mega-function:

basemap/packages/basemap/src/mpl_toolkits/basemap/__init__.py

Line 592 in f370fd3

def __init__(self, llcrnrlon=None, llcrnrlat=None,
. It is complicated, with lots of if-branching, but this would tell you what units those attributes should be in when the Basemap object is initialized. My suspicion is that there was a time when the lat/lon bounds would be recorded in radians except for cylindrical projection, but then a future change in the init made them always in degrees.

molinav commented

@matthew-william-lock May I ask you to check if the patch works for you? You would need to install basemap using the temporary bugfix-546 branch first:

python -m pip install https://github.com/matplotlib/basemap/archive/refs/heads/bugfix-546.zip#subdirectory=packages/basemap

After that, you can try the updated Python example snippet (note that I now use "World_Imagery" as service):

from mpl_toolkits.basemap import Basemap
import matplotlib.pyplot as plt

bmap = Basemap(llcrnrlon=3.75, llcrnrlat=39.75, urcrnrlon=4.35, urcrnrlat=40.15, epsg=5520)
bmap.arcgisimage(service="World_Imagery", xpixels=1500, verbose= True)
plt.show()

The default value for the service argument used to be "ESRI_Imagery_World_2D", so I also changed it to the new default value "World_Imagery" so that the default behaviour works again.

molinav commented

I will close this issue as completed with PR #548. If you still experience problems, please feel free to reopen it.