aiohttp.client_exceptions.ClientConnectorCertificateError
Closed this issue · 4 comments
[Initial] Input novel id here: n7434gm
[Main Thread] Fetching Metadata...
[Main Thread] Fetching Pages...
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/aiohttp/connector.py", line 969, in _wrap_create_connection
return await self._loop.create_connection(*args, **kwargs) # type: ignore # noqa
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py", line 1081, in create_connection
transport, protocol = await self._create_connection_transport(
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py", line 1111, in _create_connection_transport
await waiter
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/sslproto.py", line 528, in data_received
ssldata, appdata = self._sslpipe.feed_ssldata(data)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/sslproto.py", line 188, in feed_ssldata
self._sslobj.do_handshake()
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 944, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/user/Downloads/WebNovelCrawler-master/syosetu.py", line 218, in
loop.run_until_complete(syo.get_pages())
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
return future.result()
File "/Users/user/Downloads/WebNovelCrawler-master/syosetu.py", line 172, in get_pages
fetch_pages = await scheduled
File "/Users/user/Downloads/WebNovelCrawler-master/syosetu.py", line 125, in load_page
async with session.get(url, proxy=paio) as response:
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/aiohttp/client.py", line 1117, in aenter
self._resp = await self._coro
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/aiohttp/client.py", line 520, in _request
conn = await self._connector.connect(
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/aiohttp/connector.py", line 535, in connect
proto = await self._create_connection(req, traces, timeout)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/aiohttp/connector.py", line 892, in _create_connection
_, proto = await self._create_direct_connection(req, traces, timeout)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/aiohttp/connector.py", line 1051, in _create_direct_connection
raise last_exc
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/aiohttp/connector.py", line 1020, in _create_direct_connection
transp, proto = await self._wrap_create_connection(
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/aiohttp/connector.py", line 971, in _wrap_create_connection
raise ClientConnectorCertificateError(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorCertificateError: Cannot connect to host ncode.syosetu.com:443 ssl:True [SSLCertVerificationError: (1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)')]
This seems to be some common issue for Python on Mac. Please try:
# For other python version, please change the version number below
cd "/Applications/Python 3.9/"
"./Install Certificates.command"Hey, thanks so much, everything seems to be working now. If I encounter some other problems in your code, I will let you know here.
I have another question.
I can read and understand your code pretty well, but I am not exactly very familiar with asyncio in Python. As I wish to create a fork of your project and build upon it for a side project, I would like to better understand it. If it is not too much trouble, could you refer me to some tutorials about this topic?
In any case, I appreciate your help and your updating your code.
Hey, thanks so much, everything seems to be working now. If I encounter some other problems in your code, I will let you know here.
I have another question.
I can read and understand your code pretty well, but I am not exactly very familiar with asyncio in Python. As I wish to create a fork of your project and build upon it for a side project, I would like to better understand it. If it is not too much trouble, could you refer me to some tutorials about this topic?
In any case, I appreciate your help and your updating your code.
asyncio is relatively new and has various API changes during its development. Info from some random blogs or questions might be outdated and misleading so maybe just follows the documentation: https://docs.python.org/3/library/asyncio.html. If you prefer other sources, make sure they are using at least Python 3.6 (preferably 3.7+).
By the way this tool was writen during my learning of asyncio, and was updated from an old codebase using multiprocessing and requests, so there's probably somewhere I was doing wrong / inefficient things.
Thanks again. Then I will refer to the official documentation whenever necessary. If I find any inefficiencies, I will be sure to let you know!