find_checks_list fails on one Windows machine for obscure SSL reasons while accessing clang.llvm.org

[JoevDubach]

clang_html/clang_visualizer.py has the following code fragment:

def find_checks_list(clang_base_url: str):
    url = clang_base_url + 'list.html'
    resp = urllib.request.urlopen(url)
    soup = BeautifulSoup(resp, "lxml")

...which I believe should be changed to:

def find_checks_list(clang_base_url: str):
    url = clang_base_url + 'list.html'
    import ssl, certifi
    resp = urllib.request.urlopen(url, context=ssl.create_default_context(cafile=certifi.where()))
    soup = BeautifulSoup(resp, "lxml")

...with an appropriate dependency on certifi being added to the package.

The former block causes a call stack on my machine, which I isolated using the test script:

import urllib
import urllib.request
urllib.request.urlopen('https://clang.llvm.org/extra/clang-tidy/checks/list.html')

...which gets the following:

Traceback (most recent call last):
  File "c:\python391\lib\urllib\request.py", line 1342, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "c:\python391\lib\http\client.py", line 1255, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "c:\python391\lib\http\client.py", line 1301, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "c:\python391\lib\http\client.py", line 1250, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "c:\python391\lib\http\client.py", line 1010, in _send_output
    self.send(msg)
  File "c:\python391\lib\http\client.py", line 950, in send
    self.connect()
  File "c:\python391\lib\http\client.py", line 1424, in connect
    self.sock = self._context.wrap_socket(self.sock,
  File "c:\python391\lib\ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "c:\python391\lib\ssl.py", line 1040, in _create
    self.do_handshake()
  File "c:\python391\lib\ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1123)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "h:\foo2.py", line 3, in <module>
    urllib.request.urlopen('https://clang.llvm.org/extra/clang-tidy/checks/list.html')
  File "c:\python391\lib\urllib\request.py", line 214, in urlopen
    return opener.open(url, data, timeout)
  File "c:\python391\lib\urllib\request.py", line 517, in open
    response = self._open(req, data)
  File "c:\python391\lib\urllib\request.py", line 534, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "c:\python391\lib\urllib\request.py", line 494, in _call_chain
    result = func(*args)
  File "c:\python391\lib\urllib\request.py", line 1385, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
  File "c:\python391\lib\urllib\request.py", line 1345, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1123)>

...whereas this isolated script succeeds in my virtualenv that has certifi==2021.10.8 installed:

import urllib
import urllib.request
import certifi
import ssl

urllib.request.urlopen('https://clang.llvm.org/extra/clang-tidy/checks/list.html', context=ssl.create_default_context(cafile=certifi.where()))

This patch was suggested by https://stackoverflow.com/a/48134650/14137958 .

When I applied the patch manually in my virtualenv, it fixed the "python -m clang_html " use case from getting the above call stack to giving the output:

2022-04-13 22:09:49,248 - clang_html.clang_visualizer -     INFO - Writing results to clang.html

[neeraj-deshmukh]

@austinbhale, it will be great to have this issue addressed ASAP. We have run into the same issue described here, and the aforementioned patch has been tested to work. Unfortunately we cannot use locally patched utilities, so must use an updated official clang-tidy-html package.
Thank you.

[austinbhale]

Sorry for the delay! This issue has been addressed in the latest release.

This has been updated in PyPI as well => https://pypi.org/project/clang-html/

[neeraj-deshmukh]

Thank you very much for the prompt action! It is greatly appreciated.