NASA-Planetary-Science/sbpy

Orbit.from_horizons is incompatible with new Horizons query interface (?)

kaya4me2 opened this issue · 9 comments

kaya4me2 commented

High-level problem description
Horizons query interface has changed, and various kwargs like majorbody deprecated. Some basic sbpy functionality
now fails.

Readthedocs Class on this need updating: https://sbpy.readthedocs.io/en/latest/api/sbpy.data.Orbit.html#sbpy.data.Orbit

What did you do?
(working in Jupiter notebook)

import sbpy
import matplotlib.pyplot as plt
import astropy.units as u
from astropy.time import Time
from astropy.table import QTable, Table, Column
from astroquery.jplhorizons import Horizons
from sbpy.data import Ephem, Orbit

pyoorb - A Python Wrapper for OpenOrb

import pyoorb as oo

%matplotlib inline

oo.pyoorb.oorb_init()

from sbpy.data import Orbit
from astropy.time import Time

epoch = Time('2018-05-14', scale='tdb')
eph = Orbit.from_horizons('Ceres', epochs=epoch)

What did you expect?

Successful query for Ceres

What did really happen?

SSLError Traceback (most recent call last)
/anaconda3/envs/sbpy/lib/python3.7/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
709 headers=headers,
--> 710 chunked=chunked,
711 )

/anaconda3/envs/sbpy/lib/python3.7/site-packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
385 try:
--> 386 self._validate_conn(conn)
387 except (SocketTimeout, BaseSSLError) as e:

/anaconda3/envs/sbpy/lib/python3.7/site-packages/urllib3/connectionpool.py in _validate_conn(self, conn)
1039 if not getattr(conn, "sock", None): # AppEngine might not have .sock
-> 1040 conn.connect()
1041

/anaconda3/envs/sbpy/lib/python3.7/site-packages/urllib3/connection.py in connect(self)
425 ssl_context=context,
--> 426 tls_in_tls=tls_in_tls,
427 )

/anaconda3/envs/sbpy/lib/python3.7/site-packages/urllib3/util/ssl_.py in ssl_wrap_socket(sock, keyfile, certfile, cert_reqs, ca_certs, server_hostname, ssl_version, ciphers, ssl_context, ca_cert_dir, key_password, ca_cert_data, tls_in_tls)
449 ssl_sock = _ssl_wrap_socket_impl(
--> 450 sock, context, tls_in_tls, server_hostname=server_hostname
451 )

/anaconda3/envs/sbpy/lib/python3.7/site-packages/urllib3/util/ssl_.py in _ssl_wrap_socket_impl(sock, ssl_context, tls_in_tls, server_hostname)
492 if server_hostname:
--> 493 return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
494 else:

/anaconda3/envs/sbpy/lib/python3.7/ssl.py in wrap_socket(self, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, session)
422 context=self,
--> 423 session=session
424 )

/anaconda3/envs/sbpy/lib/python3.7/ssl.py in _create(cls, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, context, session)
869 raise ValueError("do_handshake_on_connect should not be specified for non-blocking sockets")
--> 870 self.do_handshake()
871 except (OSError, ValueError):

/anaconda3/envs/sbpy/lib/python3.7/ssl.py in do_handshake(self, block)
1138 self.settimeout(None)
-> 1139 self._sslobj.do_handshake()
1140 finally:

SSLError: [SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (_ssl.c:1091)

During handling of the above exception, another exception occurred:

MaxRetryError Traceback (most recent call last)
/anaconda3/envs/sbpy/lib/python3.7/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
449 retries=self.max_retries,
--> 450 timeout=timeout
451 )

/anaconda3/envs/sbpy/lib/python3.7/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
785 retries = retries.increment(
--> 786 method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
787 )

/anaconda3/envs/sbpy/lib/python3.7/site-packages/urllib3/util/retry.py in increment(self, method, url, response, error, _pool, _stacktrace)
591 if new_retry.is_exhausted():
--> 592 raise MaxRetryError(_pool, url, error or ResponseError(cause))
593

MaxRetryError: HTTPSConnectionPool(host='ssd.jpl.nasa.gov', port=443): Max retries exceeded with url: /api/horizons.api?format=text&EPHEM_TYPE=ELEMENTS&MAKE_EPHEM=YES&OUT_UNITS=AU-D&COMMAND=%22Ceres%3B%22&CENTER=%27500%4010%27&CSV_FORMAT=YES&ELEM_LABELS=YES&OBJ_DATA=YES&REF_SYSTEM=ICRF&REF_PLANE=ECLIPTIC&TP_TYPE=ABSOLUTE&TLIST=2458252.5 (Caused by SSLError(SSLError(1, '[SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (_ssl.c:1091)')))

During handling of the above exception, another exception occurred:

SSLError Traceback (most recent call last)
/var/folders/bg/mn56s0j97kb5t5d_2fv9d8m40000gn/T/ipykernel_35529/1721905493.py in
3
4 epoch = Time('2018-05-14', scale='tdb')
----> 5 eph = Orbit.from_horizons('Ceres', epochs=epoch)

/anaconda3/envs/sbpy/lib/python3.7/site-packages/sbpy/bib/core.py in wrapper(*args, **kwargs)
169 def wrapper(*args, **kwargs):
170 # only cite after successful call
--> 171 result = f(*args, **kwargs)
172 register(f, citations)
173 return result

/anaconda3/envs/sbpy/lib/python3.7/site-packages/sbpy/bib/core.py in wrapper(*args, **kwargs)
169 def wrapper(*args, **kwargs):
170 # only cite after successful call
--> 171 result = f(*args, **kwargs)
172 register(f, citations)
173 return result

/anaconda3/envs/sbpy/lib/python3.7/site-packages/sbpy/data/orbit.py in from_horizons(cls, targetids, id_type, epochs, center, **kwargs)
159 location=center, epochs=epochs)
160 try:
--> 161 elem = obj.elements(**kwargs)
162 except ValueError as e:
163 raise QueryError(

/anaconda3/envs/sbpy/lib/python3.7/site-packages/astroquery/utils/class_or_instance.py in f(*args, **kwds)
23 def f(*args, **kwds):
24 if obj is not None:
---> 25 return self.fn(obj, *args, **kwds)
26 else:
27 return self.fn(cls, *args, **kwds)

/anaconda3/envs/sbpy/lib/python3.7/site-packages/astroquery/utils/process_asyncs.py in newmethod(self, *args, **kwargs)
24 verbose = kwargs.pop('verbose', False)
25
---> 26 response = getattr(self, async_method_name)(*args, **kwargs)
27 if kwargs.get('get_query_payload') or kwargs.get('field_help'):
28 return response

/anaconda3/envs/sbpy/lib/python3.7/site-packages/astroquery/jplhorizons/core.py in elements_async(self, get_query_payload, refsystem, refplane, tp_type, closest_apparition, no_fragments, get_raw_response, cache)
802 # query and parse
803 response = self._request('GET', URL, params=request_payload,
--> 804 timeout=self.TIMEOUT, cache=cache)
805 self.uri = response.url
806

/anaconda3/envs/sbpy/lib/python3.7/site-packages/astroquery/query.py in _request(self, method, url, params, data, headers, files, save, savedir, timeout, cache, stream, auth, continuation, verify, allow_redirects, json, return_response_on_save)
321 allow_redirects=allow_redirects,
322 verify=verify,
--> 323 json=json)
324 to_cache(response, query.request_file(self.cache_location))
325 self._last_query = query

/anaconda3/envs/sbpy/lib/python3.7/site-packages/astroquery/query.py in request(self, session, cache_location, stream, auth, verify, allow_redirects, json)
74 stream=stream, auth=auth, verify=verify,
75 allow_redirects=allow_redirects,
---> 76 json=json)
77
78 def hash(self):

/anaconda3/envs/sbpy/lib/python3.7/site-packages/requests/sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
527 }
528 send_kwargs.update(settings)
--> 529 resp = self.send(prep, **send_kwargs)
530
531 return resp

/anaconda3/envs/sbpy/lib/python3.7/site-packages/requests/sessions.py in send(self, request, **kwargs)
643
644 # Send the request
--> 645 r = adapter.send(request, **kwargs)
646
647 # Total elapsed time of the request (approximately)

/anaconda3/envs/sbpy/lib/python3.7/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
515 if isinstance(e.reason, _SSLError):
516 # This branch is for urllib3 v1.22 and later.
--> 517 raise SSLError(e, request=request)
518
519 raise ConnectionError(e, request=request)

SSLError: HTTPSConnectionPool(host='ssd.jpl.nasa.gov', port=443): Max retries exceeded with url: /api/horizons.api?format=text&EPHEM_TYPE=ELEMENTS&MAKE_EPHEM=YES&OUT_UNITS=AU-D&COMMAND=%22Ceres%3B%22&CENTER=%27500%4010%27&CSV_FORMAT=YES&ELEM_LABELS=YES&OBJ_DATA=YES&REF_SYSTEM=ICRF&REF_PLANE=ECLIPTIC&TP_TYPE=ABSOLUTE&TLIST=2458252.5 (Caused by SSLError(SSLError(1, '[SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (_ssl.c:1091)')))

In [22]:

What is the actual result of the code snippet?

Throws error

Provide information on your environment:

operating system and version: MacoSX 10.14.6

from importlib_metadata import version
print(version('numpy'))
print(version('sbpy'))
print(version('astroquery'))

1.21.5
0.2.3.dev223+g9eb0523
0.4.6.dev7540

jianyangli commented

I can't reproduce the error that you encountered. The below code works well on my system.

from sbpy.data import Orbit
from astropy.time import Time
epoch = Time('2018-05-14', scale='tdb')
eph = Orbit.from_horizons('Ceres', epochs=epoch)

My system info:
MacOSX 10.15.7
sbpy == 0.2.3.dev223+g9eb0523
astroquery == 0.4.7.dev7714

From what you posted, it looks like a low-level network connection problem, rather than an sbpy or astroquery problem. Could you check again? Thanks.

jianyangli commented

Hello @kaya4me2 , I tested Orbit.from_horizons with a clean set up of sbpy in an virtual environment on my computer, and everything worked fine. Could you please try the below code snippet, assuming you've already had virtualenv installed, and let me know how it works? Let me know if you need help setting up your virtual environment.

In your terminal, type:

virtualenv .env/test_sbpy_orbit
source .env/test_sbpy_orbit/bin/activate
pip install git+https://github.com/NASA-Planetary-Science/sbpy.git
python3

Then in python, type:

from sbpy.data import Orbit
from astropy.time import Time
epoch = Time('2018-05-14', scale='tdb')
eph = Orbit.from_horizons('Ceres', epochs=epoch)
print(eph)
kaya4me2 commented
jianyangli commented

Hmmm, I don't know what's going on here. Looks like it's the failure in the low-level request package, which I'm not familiar with. I tried to search online and find some reports on similar error messages, but couldn't reproduce any in my system.

@mkelley when you have time, can you take a look and suggest how to proceed with a diagnosis? Thanks.

kaya4me2 commented
mkelley commented

I suspect this is a broader issue with SSL certificates, but let's try some more diagnostics. Bypass sbpy and go straight to astroquery:

from astroquery.jplhorizons import Horizons
q = Horizons(1, id_type='smallbody')
q.elements()

Did that fail? If so, bypass astroquery and try going straight to requests. This is the URL that my astroquery is accessing:

import requests
response = requests.get('https://ssd.jpl.nasa.gov/api/horizons.api?format=text&EPHEM_TYPE=ELEMENTS&MAKE_EPHEM=YES&OUT_UNITS=AU-D&COMMAND=%221%3B%22&CENTER=%27500%4010%27&CSV_FORMAT=YES&ELEM_LABELS=YES&OBJ_DATA=YES&REF_SYSTEM=ICRF&REF_PLANE=ECLIPTIC&TP_TYPE=ABSOLUTE&TLIST=2459698.128229722')
print(response.content)

Did that work? If so, what is the content of q.uri from the astroquery block above?

Did requests fail? What about wget on the command line?

wget 'https://ssd.jpl.nasa.gov/api/horizons.api?format=text&EPHEM_TYPE=ELEMENTS&MAKE_EPHEM=YES&OUT_UNITS=AU-D&COMMAND=%221%3B%22&CENTER=%27500%4010%27&CSV_FORMAT=YES&ELEM_LABELS=YES&OBJ_DATA=YES&REF_SYSTEM=ICRF&REF_PLANE=ECLIPTIC&TP_TYPE=ABSOLUTE&TLIST=2459698.128229722' -O orbit.txt
kaya4me2 commented

from astroquery.jplhorizons import Horizons
q = Horizons(1, id_type='smallbody')
q.elements()

fails with same error .... as does attempting to run Horizons code snippets describe at

https://astroquery.readthedocs.io/en/latest/jplhorizons/jplhorizons.html

obj = Horizons(id='Ceres', location='568', epochs={'start':'2010-01-01', 'stop':'2010-03-01','step':'10d'})
eph = obj.ephemerides()

Results in a SLSError:

SSLError: HTTPSConnectionPool(host='ssd.jpl.nasa.gov', port=443): Max retries exceeded with url: /api/horizons.api?format=text&EPHEM_TYPE=OBSERVER&QUANTITIES=%271%2C2%2C3%2C4%2C5%2C6%2C7%2C8%2C9%2C10%2C11%2C12%2C13%2C14%2C15%2C16%2C17%2C18%2C19%2C20%2C21%2C22%2C23%2C24%2C25%2C26%2C27%2C28%2C29%2C30%2C31%2C32%2C33%2C34%2C35%2C36%2C37%2C38%2C39%2C40%2C41%2C42%2C43%27&COMMAND=%22Ceres%22&SOLAR_ELONG=%220%2C180%22&LHA_CUTOFF=0&CSV_FORMAT=YES&CAL_FORMAT=BOTH&ANG_FORMAT=DEG&APPARENT=AIRLESS&REF_SYSTEM=ICRF&EXTRA_PREC=NO&CENTER=%27568%27&START_TIME=%222010-01-01%22&STOP_TIME=%222010-03-01%22&STEP_SIZE=%2210d%22&SKIP_DAYLT=NO (Caused by SSLError(SSLError(1, '[SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (_ssl.c:1131)')))

Discussing issue with SSD-API@JPL directly returned an acknowledgment that the error arrises from a JPL proxy server firewall issue. The following statement is from that exchange :

This issue (related to SSL Secure Renegotiation) appears to be related to an old “appliance” in the JPL perimeter firewall. We have reported the issue to appropriate personnel and hope for a fix in the near future. In the meantime, there may be a workaround (I’ve not verified or tested) available in this GitHub issue.

Attempting the GitHub patch suggestion is a bit complex, and we've not had success implementing it (a bit beyond our skill set with out more explicit instructions pf examples).

python packages:
numpy == 1.22.3
sbpy == 0.2.3.dev223+g9eb0523
astroquery == 0.4.6.dev7540
requests == 2.27.1

jak574 commented

skyfielders/python-skyfield#740 (comment)

Fixed now by JPL I believe (at least it works for Skyfield).

mkelley commented

Great, glad to hear this is fixed!