DoH server (ECH) issues

Question

DoH server (ECH) issues

Closed this issue a month ago · 16 comments

I'm not really sure what I did wrong

❯ openssl ecparam -genkey -name prime256v1 -out key.pem

❯ openssl req -new -x509 -days 36500 -key key.pem -out cert.pem -config san.cnf

❯ certutil -addstore -enterprise -f CA cert.pem
CA "Intermediate Certification Authorities"
Signature matches Public Key
Certificate "WR2" added to store.
CertUtil: -addstore command completed successfully.

❯ DnsSafeguard
DNS cache cleared
DoH server Listening on 127.0.0.1:443
QUIC Connecting
QUIC Connecting
QUIC Connection Established
QUIC Connection Established
DoH server: received fatal alert: CertificateUnknown

Raw access seems to work though

❯ DnsSafeguard
DNS cache cleared
DoH server Listening on 127.0.0.1:443
QUIC Connecting
QUIC Connecting
QUIC Connection Established
QUIC Connection Established
DoH server: tls handshake eof
DoH server: received fatal alert: UnknownCA
QUIC Connecting
connection is closed
QUIC 0RTT Connection Established
DoH server: peer is incompatible: SupportedVersionsExtensionRequired

❯ Test-NetConnection 127.0.0.1 -Port 443

ComputerName     : 127.0.0.1
RemoteAddress    : 127.0.0.1
RemotePort       : 443
InterfaceAlias   : Loopback Pseudo-Interface 1
SourceAddress    : 127.0.0.1
TcpTestSucceeded : True

❯ curl -sSL https://127.0.0.1/dns-query
curl: (60) SSL peer certificate or SSH remote key was not OK
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the webpage mentioned above.

❯ Invoke-WebRequest https://127.0.0.1/dns-query
Invoke-WebRequest: Authentication failed because the remote party sent a TLS alert: 'ProtocolVersion'.

Accessing https://127.0.0.1/dns-query in Firefox (security.enterprise_roots.enabled is set to true) results in:

config.json

{
	"protocol": "h3",
	"server_name": "doh3.dns.nextdns.io",
	"custom_http_path": "/redacted",
	"socket_addrs": "45.90.28.0:443",
	"udp_socket_addrs": "127.0.0.1:53",
	"disable_domain_sni": true,
	"fragmenting": {
		"enable": false,
		"method": "single"
	},
	"noise": {
		"enable": false,
		"ntype": "rand",
		"content": "google.com",
		"packet_length": 1,
		"packets": 1,
		"sleep": 1000,
		"continues": false
	},
	"ipv6": {
		"enable": true,
		"protocol": "h3",
		"server_name": "doh3.dns.nextdns.io",
		"custom_http_path": "/redacted",
		"socket_addrs": "[2a07:a8c0::]:443",
		"udp_socket_addrs": "[::1]:53",
		"disable_domain_sni": true,
		"fragmenting": {
			"enable": false,
			"method": "single"
		},
		"noise": {
			"enable": false,
			"ntype": "rand",
			"content": "google.com",
			"packet_length": 1,
			"packets": 3,
			"sleep": 500,
			"continues": false
		}
	},
	"quic": {
		"congestion_controller": "bbr",
		"keep_alive_interval": 5,
		"datagram_receive_buffer_size": 16777216,
		"datagram_send_buffer_size": 8388608,
		"connecting_timeout_sec": 2
	},
	"connection": {
		"h1_multi_connections": 3,
		"dot_nonblocking_dns_query_lifetime": 5,
		"reconnect_sleep": 1,
		"max_reconnect": 5,
		"max_reconnect_sleep": 30
	},
	"doh_server": {
		"enable": true,
		"listen_address": "127.0.0.1:443",
		"certificate": "cert.pem",
		"key": "key.pem",
		"log_errors": true
	},
	"rules": []
}

Answer 1 · 2024-12-04T01:21:53.000Z

You have to install a certificate to system so the system can trust.
To install a certificate so the Windows 11 can trust it, follow these steps:

Rename the Certificate File:
- Rename cert.pem to cert.crt The icon of cert.crt should change.
Install the Certificate:
- Right-click on cert.crt and select Install Certificate.
- In the Store Location section, select Local Machine and click Next.
- Choose Place all certificates in the following store, then click Browse.
- Select Trusted Root Certification Authorities and click OK.
- Click Next and complete the wizard.
Verify in Firefox: Try accessing the desired site in Firefox. If it doesn’t work, repeat the installation process but select Third-Party Root Certification Authorities instead of Trusted Root Certification Authorities.
Update DnsSafeguard Configuration: Don’t forget to rename cert.pem to cert.crt in the DnsSafeguard configuration file.

Answer 2 · 2024-12-04T01:40:23.000Z

Yea I've tried the Trusted Root Certification Authorities, Intermediate Certificate Authorities, & Third-Party Root Certification Authorities stores, but neither seems to work. Test-NetConnection seems to be the only thing that actually does the tls exchange, but no request clients are working. Chromium doesn't work either.

Answer 3 · 2024-12-04T03:01:12.000Z

Try this link https://127.0.0.1/?dns=PhcBAAABAAAAAAAABnZvcnRleARkYXRhCW1pY3Jvc29mdANjb20AAAEAAQ. The browser should save DNS query response as a 59Kb file.

Answer 4 · 2024-12-04T03:03:24.000Z

You should use https://127.0.0.1/dns-query{?dns} for browsers and don't miss {?dns} part.

Edit: The {?dns} part indicates that the browser should encode the DNS query to base64url format as an HTTP URL query.

Answer 5 · 2024-12-04T04:45:16.000Z

Try this link https://127.0.0.1/?dns=PhcBAAABAAAAAAAABnZvcnRleARkYXRhCW1pY3Jvc29mdANjb20AAAEAAQ. The browser should save DNS query response as a 59Kb file.

Yes, this works.

You should use https://127.0.0.1/dns-query{?dns} for browsers and don't miss {?dns} part.

This doesn't seem to. Setting network.trr.custom_uri & network.trr.uri to https://127.0.0.1/dns-query with or without the trailing dns-query & {?dns} doesn't. Nor does ECH work without the TRR resolver (setting network.trr.mode to 0 or 5), regardless of whether network.dns.native_https_query_win10 is true, or not. Realistically TRR shouldn't need to be enabled (ff 129+).

Using q doesn't seem to work either, but it's error might be more useful.

❯ q google.com '@https://127.0.0.1/'
time="2024-12-03T23:40:53-05:00" level=fatal msg="requesting https://127.0.0.1:443/?dns=zH8BAAABAAAAAAAABmdvb2dsZQNjb20AAA8AAQ: Get \"https://127.0.0.1:443/?dns=zH8BAAABAAAAAAAABmdvb2dsZQNjb20AAA8AAQ\": net/http: HTTP/1.x transport connection broken: malformed HTTP response \"\\x00\\x00\\x00\\x04\\x00\\x00\\x00\\x00\\x00\""

❯ q google.com '@https://127.0.0.1/dns-query'
time="2024-12-03T23:40:57-05:00" level=fatal msg="requesting https://127.0.0.1:443/dns-query?dns=NwUBAAABAAAAAAAABmdvb2dsZQNjb20AAAEAAQ: Get \"https://127.0.0.1:443/dns-query?dns=NwUBAAABAAAAAAAABmdvb2dsZQNjb20AAAEAAQ\": net/http: HTTP/1.x transport connection broken: malformed HTTP response \"\\x00\\x00\\x00\\x04\\x00\\x00\\x00\\x00\\x00\""

❯ q google.com '@https://127.0.0.1/' --http2
time="2024-12-03T23:41:00-05:00" level=fatal msg="unpacking DNS response from https://127.0.0.1:443/?dns=ZmcBAAABAAAAAAAABmdvb2dsZQNjb20AABAAAQ: dns: overflowing header size"

❯ q google.com '@https://127.0.0.1/dns-query' --http2
time="2024-12-03T23:41:05-05:00" level=fatal msg="unpacking DNS response from https://127.0.0.1:443/dns-query?dns=nBABAAABAAAAAAAABmdvb2dsZQNjb20AABAAAQ: dns: buffer size too small"

Answer 6 · 2024-12-04T05:53:55.000Z

It seems the issue is related to the HTTP version. Currently, the local DoH server supports only HTTP/2 (H2), while your client is using HTTP/1.1. I’m working on adding support for HTTP/1.1.

The good news is that the link I provided is working, indicating that the local H2 server is functioning correctly and the certificate is trusted. Please wait 1-3 days for me to implement HTTP/1.1 support for the local DoH server.

The bug you’re encountering is actually a client-side issue with your browser. Your browser is sending an H2 ALPN but then using HTTP/1.1.

Answer 7 · 2024-12-04T06:00:00.000Z

Also i have to implement the POST method for the DoH server. I'll notify you for testing.

Answer 8 · 2024-12-05T22:19:54.000Z

@brian6932 Can you build the app and test?

Answer 9 · 2024-12-06T00:57:24.000Z

Getting the same issue on 58226fe HTTP/2. New error for HTTP/1.1.

❯ q google.com '@https://127.0.0.1/'
time="2024-12-05T19:59:52-05:00" level=fatal msg="requesting https://127.0.0.1:443/?dns=qn0BAAABAAAAAAAABmdvb2dsZQNjb20AAAEAAQ: Get \"https://127.0.0.1:443/?dns=qn0BAAABAAAAAAAABmdvb2dsZQNjb20AAAEAAQ\": net/http: HTTP/1.x transport connection broken: malformed HTTP status code \"OK\""

❯ q google.com '@https://127.0.0.1/dns-query'
time="2024-12-05T20:00:43-05:00" level=fatal msg="requesting https://127.0.0.1:443/dns-query?dns=p80BAAABAAAAAAAABmdvb2dsZQNjb20AAAEAAQ: Get \"https://127.0.0.1:443/dns-query?dns=p80BAAABAAAAAAAABmdvb2dsZQNjb20AAAEAAQ\": net/http: HTTP/1.x transport connection broken: malformed HTTP status code \"OK\""

Answer 10 · 2024-12-06T23:36:34.000Z

@brian6932, I fixed the HTTP/1.1 malformed bug and the H2 POST method handling issue. The browser must decide whether to use ECH or not; we can't control that. You might want to check if the normal local DoH is working.

Check the new alpn option in the config file, which indicates the HTTP versions served. The variants are h2 and http/1.1. If you use both like "alpn": ["h2", "http/1.1"], then both versions are served. If you keep having issues with H2, use only http/1.1 like "alpn": ["http/1.1"] if it works better for you.

For me, both H2 and HTTP/1.1 work fine with both Edge and Firefox. However, Edge performs better using the GET method with https://127.0.0.1:443/dns-query{?dns} URL, while Firefox works better with the POST method using https://127.0.0.1:443/dns-query.

edit: Also, ECH works fine with both browsers.

Answer 11 · 2024-12-07T20:05:33.000Z

Yes Firefox works now, when setting network.trr.custom_uri & network.trr.uri to https://127.0.0.1.
I unfortunately can't get any to work OOTB, network.trr.mode 0 or 5 with network.dns.native_https_query_win10 true doesn't work for whatever reason (this is possibly currently broken, in the DNSQuery_A API on Windows 10), neither does q, or doge (these 2 don't seem to use the same native resolver).

❯ q google.com '@https://127.0.0.1'
time="2024-12-07T14:59:39-05:00" level=fatal msg="unpacking DNS response from https://127.0.0.1:443/dns-query?dns=7t8BAAABAAAAAAAABmdvb2dsZQNjb20AABAAAQ: dns: overflowing header size"

❯ q google.com '@https://127.0.0.1' --http2
time="2024-12-07T14:59:50-05:00" level=fatal msg="unpacking DNS response from https://127.0.0.1:443/dns-query?dns=kdQBAAABAAAAAAAABmdvb2dsZQNjb20AABAAAQ: dns: overflowing header size"

❯ doge google.com -H '@https://127.0.0.1/'
Error [tls]: The message received was unexpected or badly formatted. (os error -2146893018)
Error [tls]: The message received was unexpected or badly formatted. (os error -2146893018)

Answer 12 · 2024-12-07T20:36:31.000Z

I’m not sure what’s causing these issues. For HTTP/1.1, I’m adhering to standards similar to those used by Google and Cloudflare’s public DoH servers. The HTTP response, including headers and body, is sent in a single TCP segment, just like Google and Cloudflare’s public DoH servers. For HTTP/2, responses are managed by the H2 crate, which also follows the standards. There might be some conditions I missed that are causing bugs, so I need to write more tests. Thanks for testing, and I’ll notify you if I need further testing assistance.

Answer 13 · 2024-12-07T22:40:47.000Z

@brian6932 This looks fine to me.

Answer 14 · 2024-12-07T23:00:57.000Z

Updated to 2a8d51d, and still getting this, but with a little more detail. Querying A, AAAA, NS, MX, & HTTPS names work fine, but TXT fails. When the command is used without the 3rd arg, it queries all types.

❯ q google.com TXT '@https://127.0.0.1'
time="2024-12-07T17:59:12-05:00" level=fatal msg="unpacking DNS response from https://127.0.0.1:443/dns-query?dns=kUsBAAABAAAAAAAABmdvb2dsZQNjb20AABAAAQ: dns: overflow unpacking uint16"

❯ q google.com TXT '@https://127.0.0.1' --http2
time="2024-12-07T17:59:19-05:00" level=fatal msg="unpacking DNS response from https://127.0.0.1:443/dns-query?dns=ZcYBAAABAAAAAAAABmdvb2dsZQNjb20AABAAAQ: dns: overflowing header size"

❯ q google.com TXT '@https://doh3.dns.nextdns.io'
google.com. 23m43s TXT "MS=E4A68B9AB2BB9670BCE15412F62916164C0B20BB"
google.com. 23m43s TXT "apple-domain-verification=30afIBcvSuDV2PLX"
google.com. 23m43s TXT "cisco-ci-domain-verification=479146de172eb01ddee38b1a455ab9e8bb51542ddd7f1fa298557dfa7b22d963"
google.com. 23m43s TXT "docusign=05958488-4752-4ef2-95eb-aa7ba8a3bd0e"
google.com. 23m43s TXT "docusign=1b0a6754-49b1-4db5-8540-d2c12664b289"
google.com. 23m43s TXT "facebook-domain-verification=22rm551cu4k0ab0bxsw536tlds4h95"
google.com. 23m43s TXT "globalsign-smime-dv=CDYX+XFHUw2wml6/Gb8+59BsH31KzUr6c1l2BPvqKX8="
google.com. 23m43s TXT "google-site-verification=4ibFUgB-wXLQ_S7vsXVomSTVamuOXBiVAzpR5IZ87D0"
google.com. 23m43s TXT "google-site-verification=TV9-DBe4R80X4v0M4U_bd_J9cpOJM0nikft0jAgjmsQ"
google.com. 23m43s TXT "google-site-verification=wD8N7i1JTNTkezJ49swvWW48f8_9xveREV4oB-0Hf5o"
google.com. 23m43s TXT "onetrust-domain-verification=de01ed21f2fa4d8781cbc3ffb89cf4ef"
google.com. 23m43s TXT "v=spf1 include:_spf.google.com ~all"

❯ q google.com TXT '@https://doh3.dns.nextdns.io' --http2
google.com. 23m33s TXT "MS=E4A68B9AB2BB9670BCE15412F62916164C0B20BB"
google.com. 23m33s TXT "apple-domain-verification=30afIBcvSuDV2PLX"
google.com. 23m33s TXT "cisco-ci-domain-verification=479146de172eb01ddee38b1a455ab9e8bb51542ddd7f1fa298557dfa7b22d963"
google.com. 23m33s TXT "docusign=05958488-4752-4ef2-95eb-aa7ba8a3bd0e"
google.com. 23m33s TXT "docusign=1b0a6754-49b1-4db5-8540-d2c12664b289"
google.com. 23m33s TXT "facebook-domain-verification=22rm551cu4k0ab0bxsw536tlds4h95"
google.com. 23m33s TXT "globalsign-smime-dv=CDYX+XFHUw2wml6/Gb8+59BsH31KzUr6c1l2BPvqKX8="
google.com. 23m33s TXT "google-site-verification=4ibFUgB-wXLQ_S7vsXVomSTVamuOXBiVAzpR5IZ87D0"
google.com. 23m33s TXT "google-site-verification=TV9-DBe4R80X4v0M4U_bd_J9cpOJM0nikft0jAgjmsQ"
google.com. 23m33s TXT "google-site-verification=wD8N7i1JTNTkezJ49swvWW48f8_9xveREV4oB-0Hf5o"
google.com. 23m33s TXT "onetrust-domain-verification=de01ed21f2fa4d8781cbc3ffb89cf4ef"
google.com. 23m33s TXT "v=spf1 include:_spf.google.com ~all"

Answer 15 · 2024-12-08T02:21:39.000Z

@brian6932, I fixed the bug. The issue was that I initially thought the DNS query max size was 512 bytes for both request and response, but the response can actually be up to 4096 bytes. So, I changed the buffer size for all protocols, and the q app is now working well. Thanks for testing; I couldn't have figured it out without you.

Answer 16 · 2024-12-08T02:52:03.000Z

I am still getting the issue above with doge, but the error's too vague for me to really tell anything from it. It's likely just lacking support for self-signed certs.