[Bug?] TooManyHeaders(hdrs.X_FORWARDED_FOR) is raised due to duplicate IPs

[mlaradji]

I'm using aiohttp with aiohttp_remotes, and I want to execute the following line:

client_ip = aiohttp_remotes.XForwardedStrict(trusted=trusted_proxies).get_forwarded_for(request.headers)

This raises a TooManyHeaders(hdrs.X_FORWARDED_FOR) error. Upon inspection, I found that the line

request.headers.getall(hdrs.X_FORWARDED_FOR, [])

returns, in my case:

['192.168.0.1', '192.168.0.1']

I am still new to aiohttp_remotes, and I might be misunderstaning the error. However, it seems to me that duplicate IPs should not raise a TooManyHeaders error. If so, an easy fix would be to change the following line in aiohttp_remotes/x_forwarded.py

    def get_forwarded_for(self, headers):
        forwarded_for = headers.getall(hdrs.X_FORWARDED_FOR, [])
...

to

    def get_forwarded_for(self, headers):
        forwarded_for = list(set(headers.getall(hdrs.X_FORWARDED_FOR, [])))
...

System Information

• aiohttp: 3.5.4
• aiohttp_remotes: 0.1.2
• Python: 3.6.7

[asvetlov]

Since X-Forwarded-* limitations there is a genuine conflict when any header is encountered more than once.
That's why the library raises an exception for such case.

Standard Forwarded header supports multiple occurrences, it is the preferable approach for very many reasons.

[mlaradji]

Thanks for the reply. I need the X-Forwarded header since I'm using a reverse proxy.

As it turns out, the issue was my nginx config. It had the following line:

        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

which caused duplicate X-Forwarded-For headers.

[asvetlov]

You may be surprised but Forwarded can be used for reverse proxies as well ;)

Anyway, happy to know that the problem is solved.