Use API with Python: Connection aborted, RemoteDisconnected('Remote end closed connection without response')
Closed this issue · 2 comments
Question
I am trying to use the API with Python's requests library. Following the API documentation how to retrieve an entry I wrote this minimal script:
from requests import request
url = "https://regex101.com/api/regex/AaKDTO/1"
response = request("GET", url)
I expect to get a 404 response with the following content: {"error":"Regex does not exist","errorId":"b20d027a-fb34-4e34-bb4e-c2e60f568b98"}
.
However, when I run it, I get this:
Traceback (most recent call last):
File "C:\Users\fleetingbytes\env\req\lib\site-packages\urllib3\connectionpool.py", line 793, in urlopen
response = self._make_request(
File "C:\Users\fleetingbytes\env\req\lib\site-packages\urllib3\connectionpool.py", line 537, in _make_request
response = conn.getresponse()
File "C:\Users\fleetingbytes\env\req\lib\site-packages\urllib3\connection.py", line 466, in getresponse
httplib_response = super().getresponse()
File "C:\Program Files\Python310\lib\http\client.py", line 1374, in getresponse
response.begin()
File "C:\Program Files\Python310\lib\http\client.py", line 318, in begin
version, status, reason = self._read_status()
File "C:\Program Files\Python310\lib\http\client.py", line 287, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\fleetingbytes\env\req\lib\site-packages\requests\adapters.py", line 486, in send
resp = conn.urlopen(
File "C:\Users\fleetingbytes\env\req\lib\site-packages\urllib3\connectionpool.py", line 847, in urlopen
retries = retries.increment(
File "C:\Users\fleetingbytes\env\req\lib\site-packages\urllib3\util\retry.py", line 470, in increment
raise reraise(type(error), error, _stacktrace)
File "C:\Users\fleetingbytes\env\req\lib\site-packages\urllib3\util\util.py", line 38, in reraise
raise value.with_traceback(tb)
File "C:\Users\fleetingbytes\env\req\lib\site-packages\urllib3\connectionpool.py", line 793, in urlopen
response = self._make_request(
File "C:\Users\fleetingbytes\env\req\lib\site-packages\urllib3\connectionpool.py", line 537, in _make_request
response = conn.getresponse()
File "C:\Users\fleetingbytes\env\req\lib\site-packages\urllib3\connection.py", line 466, in getresponse
httplib_response = super().getresponse()
File "C:\Program Files\Python310\lib\http\client.py", line 1374, in getresponse
response.begin()
File "C:\Program Files\Python310\lib\http\client.py", line 318, in begin
version, status, reason = self._read_status()
File "C:\Program Files\Python310\lib\http\client.py", line 287, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\fleetingbytes\src\minimal\minimal.py", line 5, in <module>
res = request("GET", url)
File "C:\Users\fleetingbytes\env\req\lib\site-packages\requests\api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Users\fleetingbytes\env\req\lib\site-packages\requests\sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\fleetingbytes\env\req\lib\site-packages\requests\sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
File "C:\Users\fleetingbytes\env\req\lib\site-packages\requests\adapters.py", line 501, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
Is anything wrong with the API or the API documentation?
It seems you're bumping into whatever anti-bot/DDOS protection is running in front of /api
. I'm not privy to the details, but from testing it seems that if you use a User-Agent other than python's default it should be good to go. Tested it with a browser, insomnia, and curl from cli. As soon as I tell curl to send the header I get the same response as you.
$ curl "https://regex101.com/api/regex/AaKDTO/1"
{"error":"Regex does not exist","errorId":"30eeb9bb-ecb5-4a54-a840-c6711b8bb124"}
$ curl "https://regex101.com/api/regex/AaKDTO/1" -H "User-Agent: python-requests2.31.0"
curl: (92) HTTP/2 stream 0 was not closed cleanly: PROTOCOL_ERROR (err 1)
@working-name is right