Custom DNS resolvers are not used in httpx requests

[zsims]

Nuclei version:

2.9.6

Current Behavior:

Nuclei allows custom DNS resolvers to be set with -resolvers. These resolvers are used for checks from DNS templates. However, the resolvers are not used for HTTP.

Expected Behavior:

HTTP requests used the provided DNS resolvers. Using https://developers.cloudflare.com/1.1.1.1/setup/check/ as a test.

Steps To Reproduce:

example.yaml

id: resolver-repro

info:
  name: Example
  author: zsims
  severity: medium
  description: Example

http:
  - method: GET
    path:
      - "{{BaseURL}}"

    matchers:
      - type: word
        part: body
        words:
          - 'Example Domain'

dnsmasq.conf

address=/example.com/127.0.0.1
port=5533

Verify dnsmasq setup with custom record:

$ dig example.com @localhost -p 5533 +short
127.0.0.1

resolvers.txt

127.0.0.1:5533

$ nuclei -resolvers resolvers.txt -u http://example.com -t example.yaml
...
[resolver-repro] [http] [medium] http://example.com

You'd expect this not to match given it points to localhost which doesn't serve example.com

Anything else:

https://github.com/projectdiscovery/httpx supports this out of the box

[Mzack9999]

@zsims I tried to reproduce the issue with the dev branch, but it seems fixed as the DNS requests, as visible in the following screenshot, go through the loopback interface towards the resolver specified in the resolvers.txt file

[zsims]

Thanks for digging into this @Mzack9999. On the latest dev (0993ac4cbb) on macOS the resolvers are seemingly still ignored.

$ go run . -resolvers ~/repro/resolvers.txt -u http://example.com -t ~/repro/example.yaml -v
                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v2.9.8

                projectdiscovery.io

[INF] Current nuclei version: v2.9.8 (latest)
[INF] Current nuclei-templates version: v9.5.6 (latest)
[INF] New templates added in latest release: 3
[INF] Templates loaded for current scan: 1
[INF] Targets loaded for current scan: 1
[VER] [resolver-repro] Sent HTTP request to http://example.com
[resolver-repro] [http] [medium] http://example.com

However, I wasn't able to reproduce in Linux (Docker) (per https://github.com/zsims/nuclei-3856) seems this might be specific to macOS?

$ git clone https://github.com/zsims/nuclei-3856.git
$ cd nuclei-3856
$ ./reproduce.sh
; <<>> DiG 9.18.16-1~deb12u1-Debian <<>> example.com A @127.0.0.1
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 56764
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;example.com.                   IN      A

;; ANSWER SECTION:
example.com.            0       IN      A       127.0.0.1

;; Query time: 1 msec
;; SERVER: 127.0.0.1#53(127.0.0.1) (UDP)
;; WHEN: Tue Jul 11 22:39:39 UTC 2023
;; MSG SIZE  rcvd: 56


                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v2.9.8

                projectdiscovery.io

[INF] nuclei-templates are not installed, installing...
[INF] Successfully installed nuclei-templates at /root/nuclei-templates
[INF] Current nuclei version: v2.9.8 (latest)
[INF] Current nuclei-templates version: v9.5.6 (latest)
[INF] New templates added in latest release: 3
[INF] Templates loaded for current scan: 1
[INF] Targets loaded for current scan: 1
[WRN] [resolver-repro] Could not execute request for http://example.com: GET http://example.com giving up after 2 attempts: Get "http://example.com": could not connect to any address found for host
[INF] No results found. Better luck next time!

[Mzack9999]

I'm still unable to reproduce it on osx:

$ git branch
...
* dev
...
$ cat resolver.txt
127.0.0.1:53
$ cat test.yaml
id: resolver-repro

info:
  name: Example
  author: zsims
  severity: medium
  description: Example

http:
  - method: GET
    path:
      - "{{BaseURL}}"

    matchers:
      - type: word
        part: body
        words:
          - 'Example Domain'

$ docker run -i -p 53:53/tcp -p 53:53/udp -e DNS='*.example.com=192.168.1.1' -t cytopia/bind # in another terminal

$ echo http://aaa.example.com | go run . -t test.yaml  -v -r resolver.txt -debug

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v2.9.10

                projectdiscovery.io

[INF] Current nuclei version: v2.9.10 (latest)
[INF] Current nuclei-templates version: v9.6.0 (latest)
[INF] New templates added in latest release: 33
[INF] Templates loaded for current scan: 1
[INF] Targets loaded for current scan: 1
[INF] [resolver-repro] Dumped HTTP request for http://aaa.example.com

GET / HTTP/1.1
Host: aaa.example.com
User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1866.237 Safari/537.36
Connection: close
Accept: */*
Accept-Language: en
Accept-Encoding: gzip

[DBG] [resolver-repro] Dumped HTTP response http://aaa.example.com

HTTP/1.1 200 OK
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html; charset=utf-8
Date: Fri, 11 Aug 2023 16:40:10 GMT

<!doctype html>
<html>
<head>
...

As visible the request goes to 192.168.1.1

[zsims]

Hmmm, I must have something interfering.

Thanks for looking at this, and sorry for the inconvenience. I'll do some digging - but feel free to close.