The example extract-and-fill-forms doesn't work.
fixapi-org opened this issue · 3 comments
I tried this example:
https://github.com/x4nth055/pythoncode-tutorials/tree/master/web-scraping/extract-and-fill-forms
But failed.
I add res.raise_for_status() after the line calling session.post.
The error is:
Traceback (most recent call last):
File "/home/ubuntu/.linuxbrew/Cellar/python@3.8/3.8.5/lib/python3.8/site-packages/urllib3/connectionpool.py", line 696, in urlopen
self._prepare_proxy(conn)
File "/home/ubuntu/.linuxbrew/Cellar/python@3.8/3.8.5/lib/python3.8/site-packages/urllib3/connectionpool.py", line 964, in prepare_proxy
conn.connect()
File "/home/ubuntu/.linuxbrew/Cellar/python@3.8/3.8.5/lib/python3.8/site-packages/urllib3/connection.py", line 416, in connect
self.sock = ssl_wrap_socket(
File "/home/ubuntu/.linuxbrew/Cellar/python@3.8/3.8.5/lib/python3.8/site-packages/urllib3/util/ssl.py", line 449, in ssl_wrap_socket
ssl_sock = ssl_wrap_socket_impl(
File "/home/ubuntu/.linuxbrew/Cellar/python@3.8/3.8.5/lib/python3.8/site-packages/urllib3/util/ssl.py", line 493, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
File "/home/ubuntu/.linuxbrew/Cellar/python@3.8/3.8.5/lib/python3.8/ssl.py", line 500, in wrap_socket
return self.sslsocket_class._create(
File "/home/ubuntu/.linuxbrew/Cellar/python@3.8/3.8.5/lib/python3.8/ssl.py", line 1040, in _create
self.do_handshake()
File "/home/ubuntu/.linuxbrew/Cellar/python@3.8/3.8.5/lib/python3.8/ssl.py", line 1309, in do_handshake
self._sslobj.do_handshake()
OSError: [Errno 0] Error
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ubuntu/.linuxbrew/Cellar/python@3.8/3.8.5/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
File "/home/ubuntu/.linuxbrew/Cellar/python@3.8/3.8.5/lib/python3.8/site-packages/urllib3/connectionpool.py", line 755, in urlopen
retries = retries.increment(
File "/home/ubuntu/.linuxbrew/Cellar/python@3.8/3.8.5/lib/python3.8/site-packages/urllib3/util/retry.py", line 574, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='sometestwebsite', port=443): Max retries exceeded with url: /wp-comments-post.php (Caused by ProxyError('Cannot connect to proxy.', OSError(0, 'Error')))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "form_submitter.py", line 68, in
res = session.post(url, data=data)
File "/home/ubuntu/.linuxbrew/Cellar/python@3.8/3.8.5/lib/python3.8/site-packages/requests/sessions.py", line 590, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "/home/ubuntu/.linuxbrew/Cellar/python@3.8/3.8.5/lib/python3.8/site-packages/requests/sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "/home/ubuntu/.linuxbrew/Cellar/python@3.8/3.8.5/lib/python3.8/site-packages/requests/sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "/home/ubuntu/.linuxbrew/Cellar/python@3.8/3.8.5/lib/python3.8/site-packages/requests/adapters.py", line 510, in send
raise ProxyError(e, request=request)
requests.exceptions.ProxyError: HTTPSConnectionPool(host='sometestwebsite', port=443): Max retries exceeded with url: /wp-comments-post.php (Caused by ProxyError('Cannot connect to proxy.', OSError(0, 'Error')))
The website for test is Wordpress based.
How can we fix it?
Thanks for your advice in advance.
I've just found the authors repo and using some of these features, which are very handy.
Wordpress is such a constant briar patch and tar pit of bad design, that its almost never a good idea, good solution or anything more than a "look what I did" over the weekend for beginners. Try testing on another site, or if you figure it out post here and I'll help update this with @x4nth055 permission.
Thank you for your response.
But, actually, I didn't use WP to test the example I mentioned above.
@fixapi-org so when you say You didn't use WP... both you and your stack trace kinda indicates otherwise
wp-comments and the whole "the website for test is wordpress based"
/wp-comments-post.php (Caused by ProxyError('Cannot connect to proxy.', OSError(0, 'Error')))
The website for test is Wordpress based.
How can we fix it?
Thanks for your advice in advance.