Rate limiting when being reverse proxied?
Closed this issue · 4 comments
I'm able to view the main site behind a reverse proxy using this stack, with minor tweaks to docker-compose.yml and the Caddyfile.
However, everytime I try to search something I get 'rate limit exceeded'. My reverse proxy is not rate limiting anything at all (because I've turned it off for this domain) and I've set Filtron's rules.json to very high values. Does Morty include some kind of built in rate limiting I'm un aware of?
The above image is all I get to see in the browsers, but the console shows:
Well, that was quick... after numerous hours here and there toying with this over the week.... I finally realised that maybe Filtron thinks I'm a bot (because of the way I'm requesting) or something. So I just edited rules.json to change all the stop: true values to stop: false and upped the rate limits to 9999999999 and lo and behold it works! No more rate limiting.
Will try and figure out exactly which parameters were causing the issue and update the issue before I close, but that's a job for the morning at this time of night!
@unixfox i think you should reconsider saying it’s not supported.
After much trial and error minimal changes to the current setup allow it to work:
- Letting caddy just issue a self signed cert from its built in root CA (this is automatically done by just not setting up let’s encrypt). If you want custom certs this can be done with the existing docker-compose-extend or a docker-compose-override file, and just mount the files to Caddy, which can then be used in Caddyfile as per Caddy docs (it’s a single line in the config). Letting it self sign is pretty much part of normal searx-docker setup.
- Just comment out the CSP lines for morty/notmorty in Caddyfile if you already have CSP with the reverse proxy you use. If you don’t use CSP there, then just edit it accordingly. Opening a browser console, inspecting it whilst load the page will show which CSP need to be changed. This is very easy in the end with basic Google-fu.
- Edit rules.json to allow filtron to pass results to the reverse proxy by changing the rate limiting settings. This is not a problem, because all reverse proxy have rate limit facility
- Edit the .env host name to include both your FQDN and the local ip of the server / VM running searx in the SEARXHOSTNAME variable. You might only need local or FQDN, I haven’t checked yet. This is normal setup anyway.
- Change settings.yml for searx to have a bind address of the server/VM IP (e.g 192.168.1.2 or whatever) and change base_url to ‘False’. Again this is normal setup step for searx-docker.
This means that only rules.json and Caddyfile actually get changed and none of the changes are complex just not necessarily obvious.
We aren't against the ability to reverse proxy searx-docker, that's why I said "yet". It's just that it all started as a project to easily launch searx on any Linux server, and we (well dalf), didn't consider at first the reverse proxies.
If you have time to rework the current project so that it works with a reverse proxy and without a reverse proxy I would gladly accept your contribution but currently at the Searx team we don't have time to work on that sorry.