ot scraper is broken

[BenjiDa]

• lsdviztools version:docker container
• Python version: 3.8
• Operating System: macos

Description

using the ot scraper to download SRTM data from open topography is broken, I have included my API key but it still throws a 401 error.

What I Did

SB_DEM = bmt.ot_scraper(source = "SRTM30",longitude_W = -123.225226, longitude_E = -122.954536, latitude_S = 38.941020, latitude_N = 39.272342,prefix = "maacama", api_key=' ') #I removed my key here and down below too.
SB_DEM.print_parameters()
SB_DEM.download_pythonic()

---------------------------------------------------------------------------
HTTPError                                 Traceback (most recent call last)
/tmp/ipykernel_70/2381537486.py in <module>
      3 SB_DEM = bmt.ot_scraper(source = "SRTM30",longitude_W = -123.225226, longitude_E = -122.954536, latitude_S = 38.941020, latitude_N = 39.272342,prefix = "maacama", api_key=' ')
      4 SB_DEM.print_parameters()
----> 5 SB_DEM.download_pythonic()

/usr/local/lib/python3.8/site-packages/lsdviztools/lsdbasemaptools/lsdmap_otgrabber.py in download_pythonic(self)
    155 
    156         # Download the file from `url` and save it locally under `file_name`:
--> 157         with urllib.request.urlopen(url_string) as response, open(filename, 'wb') as out_file:
    158             shutil.copyfileobj(response, out_file)
    159 

/usr/local/lib/python3.8/urllib/request.py in urlopen(url, data, timeout, cafile, capath, cadefault, context)
    220     else:
    221         opener = _opener
--> 222     return opener.open(url, data, timeout)
    223 
    224 def install_opener(opener):

/usr/local/lib/python3.8/urllib/request.py in open(self, fullurl, data, timeout)
    529         for processor in self.process_response.get(protocol, []):
    530             meth = getattr(processor, meth_name)
--> 531             response = meth(req, response)
    532 
    533         return response

/usr/local/lib/python3.8/urllib/request.py in http_response(self, request, response)
    638         # request was successfully received, understood, and accepted.
    639         if not (200 <= code < 300):
--> 640             response = self.parent.error(
    641                 'http', request, response, code, msg, hdrs)
    642 

/usr/local/lib/python3.8/urllib/request.py in error(self, proto, *args)
    567         if http_err:
    568             args = (dict, 'default', 'http_error_default') + orig_args
--> 569             return self._call_chain(*args)
    570 
    571 # XXX probably also want an abstract factory that knows when it makes

/usr/local/lib/python3.8/urllib/request.py in _call_chain(self, chain, kind, meth_name, *args)
    500         for handler in handlers:
    501             func = getattr(handler, meth_name)
--> 502             result = func(*args)
    503             if result is not None:
    504                 return result

/usr/local/lib/python3.8/urllib/request.py in http_error_default(self, req, fp, code, msg, hdrs)
    647 class HTTPDefaultErrorHandler(BaseHandler):
    648     def http_error_default(self, req, fp, code, msg, hdrs):
--> 649         raise HTTPError(req.full_url, code, msg, hdrs, fp)
    650 
    651 class HTTPRedirectHandler(BaseHandler):

HTTPError: HTTP Error 401: 401

[simon-m-mudd]

Hi Ben, I'm updating the container and lsdviztools to make sure this works but it might take a day or two.

[simon-m-mudd]

I have now updated lsdviztools as well as the lsdtt_pytools_docker container.
Warning: the new container required a different base image so you will have to pull an entirely new container. You may wish to do this overnight!
The updated version uses a file rather than entering the API key directly. Here is an example:

# YOU NEED TO PUT YOUR API KEY IN A FILE
your_OT_api_key_file = "my_OT_api_key.txt"

with open(your_OT_api_key_file, 'r') as file:
    print("I am reading you OT API key from the file "+your_OT_api_key_file)
    api_key = file.read().rstrip()
    print("Your api key starts with: "+api_key[0:4])

Dataset_prefix = "Xian"
source_name = "COP30"

Xian_DEM = bmt.ot_scraper(source = source_name,
                        lower_left_coordinates = [33.672715066202954, 107.54480443333152], 
                        upper_right_coordinates = [34.16323953210814, 109.4363649228437],
                        prefix = Dataset_prefix, 
                        api_key_file = your_OT_api_key_file)

If you do not want to pull the new image you can still access the updated OT scraper by updating lsdviztools to version 0.4.8.