python-mechanize/mechanize

Proxy-Authorisation always enters infinite loop due to case mismatch

HansWeltar opened this issue · 0 comments

Whenever we try to authenticate to a proxy server with a wrong password, mechanize enters an infite loop.

To reproduce:

browser.set_proxies({"http": "127.0.0.1:3128"})
browser.add_proxy_password("user", "pass")
browser.open("http://www.google.com")

error:

RuntimeError: maximum recursion depth exceeded
File "/usr/local/lib/python2.7/dist-packages/mechanize/_mechanize.py", line 253, in open
return self._mech_open(url_or_request, data, timeout=timeout)
File "/usr/local/lib/python2.7/dist-packages/mechanize/_mechanize.py", line 283, in _mech_open
response = UserAgentBase.open(self, request, data)
File "/usr/local/lib/python2.7/dist-packages/mechanize/_opener.py", line 204, in open
response = meth(req, response)
File "/usr/local/lib/python2.7/dist-packages/mechanize/_urllib2_fork.py", line 486, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/local/lib/python2.7/dist-packages/mechanize/_opener.py", line 222, in error
result = self._call_chain(*args)
File "/usr/local/lib/python2.7/dist-packages/mechanize/_urllib2_fork.py", line 364, in _call_chain
result = func(*args)
File "/usr/local/lib/python2.7/dist-packages/mechanize/_urllib2_fork.py", line 895, in http_error_407
authority, req, headers)
File "/usr/local/lib/python2.7/dist-packages/mechanize/_urllib2_fork.py", line 849, in http_error_auth_reqed
return self.retry_http_basic_auth(host, req, realm)
File "/usr/local/lib/python2.7/dist-packages/mechanize/_urllib2_fork.py", line 863, in retry_http_basic_auth
return self.parent.open(newreq)
File "/usr/local/lib/python2.7/dist-packages/mechanize/_mechanize.py", line 253, in open
return self._mech_open(url_or_request, data, timeout=timeout)
File "/usr/local/lib/python2.7/dist-packages/mechanize/_mechanize.py", line 283, in _mech_open
response = UserAgentBase.open(self, request, data)
File "/usr/local/lib/python2.7/dist-packages/mechanize/_opener.py", line 204, in open
response = meth(req, response)
File "/usr/local/lib/python2.7/dist-packages/mechanize/_urllib2_fork.py", line 486, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/local/lib/python2.7/dist-packages/mechanize/_opener.py", line 222, in error
result = self._call_chain(*args)
...

cause:
The root-cause is that header names are normalized when adding, but that a non-normalized lookup is performed to prevent the infinite loop.

in _urllib2_fork.py in ProxyBasicAuthHandler (line 886) it defines
auth_header = 'Proxy-authorization' (note the lowercase 'a')

In the function retry_http_basic_auth (_urllib2_fork.py, line 860),
the proxy-authorisation is added with normalization:
newreq.add_header(self.auth_header, auth) -- add_header does header normalization and so "Proxy-Authorization" is added to the headers.

The function retry_http_basic_auth (_urllib2_fork.py, line 857), will keep on trying authentication
as long as the new password is different from the last.
It does this with the line req.headers.get(self.auth_header, None)
However headers.get does not do normalization.
Therefore it will look for the header 'Proxy-authorization' which does not exist (Proxy-Authorization does exist.)

potential fix1:
change headers.get to get_header, which does normalization
(urllib2_fork.py, line 854)

if req.get_header(self.auth_header, None) == auth:
    return None

potential fix2:
set auth_header = 'Proxy-Authorization' in class ProxyBasicAuthHandler (urllib2_fork.py, line 886)

Either of the fixes above solves the problem for me.