SSLCertVerificationError exceptions with PySNC 1.1.0 and 1.1.1
Closed this issue · 8 comments
Hello,
PySNC 1.1.0 and 1.1.1 seem to be causing ssl.SSLCertVerificationError when SSL is being intercepted in an enterprise environment.
Upgrading to Python 3.11 does not help. Locking PySNC to 1.0.7 does resolve the issue.
{"pysnc":"1.1.0","severity":"INFO"}
Traceback (most recent call last):
File "/home/gokeefe/.local/share/virtualenvs/servicenow-active-tasks-report-AWneSsyF/lib/python3.9/site-packages/urllib3/connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
File "/home/gokeefe/.local/share/virtualenvs/servicenow-active-tasks-report-AWneSsyF/lib/python3.9/site-packages/urllib3/connectionpool.py", line 386, in _make_request
self._validate_conn(conn)
File "/home/gokeefe/.local/share/virtualenvs/servicenow-active-tasks-report-AWneSsyF/lib/python3.9/site-packages/urllib3/connectionpool.py", line 1042, in _validate_conn
conn.connect()
File "/home/gokeefe/.local/share/virtualenvs/servicenow-active-tasks-report-AWneSsyF/lib/python3.9/site-packages/urllib3/connection.py", line 419, in connect
self.sock = ssl_wrap_socket(
File "/home/gokeefe/.local/share/virtualenvs/servicenow-active-tasks-report-AWneSsyF/lib/python3.9/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
ssl_sock = _ssl_wrap_socket_impl(
File "/home/gokeefe/.local/share/virtualenvs/servicenow-active-tasks-report-AWneSsyF/lib/python3.9/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
File "/home/gokeefe/.pyenv/versions/3.9.13/lib/python3.9/ssl.py", line 501, in wrap_socket
return self.sslsocket_class._create(
File "/home/gokeefe/.pyenv/versions/3.9.13/lib/python3.9/ssl.py", line 1041, in _create
self.do_handshake()
File "/home/gokeefe/.pyenv/versions/3.9.13/lib/python3.9/ssl.py", line 1310, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:1129)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/gokeefe/.local/share/virtualenvs/servicenow-active-tasks-report-AWneSsyF/lib/python3.9/site-packages/requests/adapters.py", line 487, in send
resp = conn.urlopen(
File "/home/gokeefe/.local/share/virtualenvs/servicenow-active-tasks-report-AWneSsyF/lib/python3.9/site-packages/urllib3/connectionpool.py", line 787, in urlopen
retries = retries.increment(
File "/home/gokeefe/.local/share/virtualenvs/servicenow-active-tasks-report-AWneSsyF/lib/python3.9/site-packages/urllib3/util/retry.py", line 592, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='gokeefe.service-now.com', port=443): Max retries exceeded with url: /api/now/table/sc_task?sysparm_query=assignment_group%3D117db1491bdd7b408f04caad1e4bcbcd%5EORassignment_group%3Db697561adb1017807037572e5e9619b7%5Eactive%3Dtrue%5EORDERBYopened_at&sysparm_limit=100&sysparm_offset=0&sysparm_display_value=all&sysparm_exclude_reference_link=true&sysparm_suppress_pagination_header=true (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:1129)')))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/gokeefe/gitlab/devops/devops-automation/automated-reports/servicenow-active-tasks-report/./main.py", line 168, in <module>
main(start_time)
File "/home/gokeefe/gitlab/devops/devops-automation/automated-reports/servicenow-active-tasks-report/./main.py", line 123, in main
gr.query()
File "/home/gokeefe/.local/share/virtualenvs/servicenow-active-tasks-report-AWneSsyF/lib/python3.9/site-packages/pysnc/record.py", line 453, in query
response = self._client.table_api.list(self)
File "/home/gokeefe/.local/share/virtualenvs/servicenow-active-tasks-report-AWneSsyF/lib/python3.9/site-packages/pysnc/client.py", line 194, in list
return self._send(req)
File "/home/gokeefe/.local/share/virtualenvs/servicenow-active-tasks-report-AWneSsyF/lib/python3.9/site-packages/pysnc/client.py", line 176, in _send
r = self.session.send(request, stream=stream)
File "/home/gokeefe/.local/share/virtualenvs/servicenow-active-tasks-report-AWneSsyF/lib/python3.9/site-packages/requests/sessions.py", line 701, in send
r = adapter.send(request, **kwargs)
File "/home/gokeefe/.local/share/virtualenvs/servicenow-active-tasks-report-AWneSsyF/lib/python3.9/site-packages/requests/adapters.py", line 518, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='gokeefe.service-now.com', port=443): Max retries exceeded with url: /api/now/table/sc_task?sysparm_query=assignment_group%3D117db1491bdd7b408f04caad1e4bcbcd%5EORassignment_group%3Db697561adb1017807037572e5e9619b7%5Eactive%3Dtrue%5EORDERBYopened_at&sysparm_limit=100&sysparm_offset=0&sysparm_display_value=all&sysparm_exclude_reference_link=true&sysparm_suppress_pagination_header=true (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:1129)')))
hm, has to be a default behavior of a newer requests. I didn't technically change anything there.
@gokeefe are you setting ServiceNowClient(..., verify=False) ?
Hi @vetsin,
It's actually installing the same requests version, 2.29.0, whether I'm using v1.0.7, 1.1.0, or 1.1.1, and I have the $REQUESTS_CA_BUNDLE environment variable set to point the requests module at a certificate bundle that trusts the interception certificate.
I am not setting verify to False, and would prefer not to for obvious reasons.
@gokeefe are you setting a proxy? The only change on this between 1.0.7 and 1.1.x is you can toggle verify even if you dont use a proxy...
Can you print(client.session.verify) to see if they are any different between the two versions?
With 1.0.7, client.session.verify returns True.
With 1.1.2, client.session.verify also returns True.
On that note, if I initialize the client with ServiceNowClient(..., verify=path_to_bundle), where path_to_bundle is the full path to a certificate bundle containing the interception certificate, it works fine. Not sure what changed after 1.0.7, but something seems to be causing requests to ignore the $REQUEST_CA_BUNDLE environment variable.
Figured out WHY:
When you are using the prepared request flow, keep in mind that it does not take into account the environment. This can cause problems if you are using environment variables to change the behaviour of requests. For example: Self-signed SSL certificates specified in REQUESTS_CA_BUNDLE will not be taken into account. As a result an SSL: CERTIFICATE_VERIFY_FAILED is thrown. You can get around this behaviour by explicitly merging the environment settings into your session
-- i'll make sure to maintain old behavior here