encode/httpx

Not support "no_proxy" including value of ipv6 prefix style

piamo opened this issue · 5 comments

Example: no_proxy=fe11::/16

How to reproduce:

no_proxy=fe11::/16 python -c 'import httpx; c = httpx.Client()'

it will raise:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/httpx/_urlparse.py", line 346, in normalize_port
    port_as_int = int(port)
ValueError: invalid literal for int() with base 10: ':'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 695, in __init__
    self._mounts: dict[URLPattern, BaseTransport | None] = {
  File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 696, in <dictcomp>
    URLPattern(key): None
  File "/usr/local/lib/python3.9/dist-packages/httpx/_utils.py", line 370, in __init__
    url = URL(pattern)
  File "/usr/local/lib/python3.9/dist-packages/httpx/_urls.py", line 115, in __init__
    self._uri_reference = urlparse(url, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/httpx/_urlparse.py", line 248, in urlparse
    parsed_port: int | None = normalize_port(port, scheme)
  File "/usr/local/lib/python3.9/dist-packages/httpx/_urlparse.py", line 348, in normalize_port
    raise InvalidURL(f"Invalid port: {port!r}")
httpx.InvalidURL: Invalid port: ':'

httpx version: 0.27.0

Another test case to throw in that is affecting us:

 no_proxy=[::1] python -c 'import httpx; c = httpx.Client()'
...
httpx.InvalidURL: Invalid port: ':1]'

For no_proxy=127.0.0.0/8, get_environment_proxies() will get all://127.0.0.0/8.
So, when creating a Client, in which URL class will be called, ⁠/8 will be treated as a URL path finally.

python -c 'import httpx; url = httpx.URL("all://127.0.0.0/8"); print(url.path)'
/8

httpx version: 0.27.0

Duplicate of #1536

No we don't currently support subnet masks here, yes we should at least raise an error if they're in play.

Aside... what's an example of a real-world scenario where it's useful to have a subnet masking used to toggle proxy routing on/off?

I also encountered this issue when using proxy in company's network. We use proxy to visit outside networks and automatically set no_proxy with both ipv4/ipv6 masking to prevent proxying specific ranges of IPs, such as 127.0.0.0/8,169.254.0.0/16,100.64.0.0/10,172.16.0.0/12,192.168.0.0/16,10.0.0.0/8,::1,fe80::/10,fd00::/8.