HTTP3 Server and Client from samples
master-of-darkness opened this issue · 15 comments
Hi there!
I'm trying to figure out why the HQ client from the samples can't connect to the HQ server (also from the samples). I am able to connect using HTTP/2 (HTTPS) from the browser, but QUIC doesn't seem to work at all.
I noticed that hqParams.supportedAlpns includes hq-29. Could the issue be related to the ALPN or the certificate?
server run command:
sudo ./proxygenhttp3_2serv --mode=server \
--h2port=8080 \
--port=8443 \
--protocol=hq-29 \
--host=127.0.0.1
client run command:
./hq --mode=client --path=/ --host=127.0.0.1 --port=8443 --protocol=hq-29
server log:
I1101 23:22:18.819690 14063 HQServer.cpp:286] HQ server started at: 127.0.0.1:8443
E1101 23:26:13.730489 14073 HQSession.cpp:413] next protocol not supported: hq-29 sess=proto=, UA=, client CID=, server CID=4000000a27839153, downstream=127.0.0.1:49528, 127.0.0.1:8443=local, drain=none
E1101 23:26:13.730515 14073 HQSession.cpp:492] Peer closed with error err=Connect failed msg=ALPN not supported proto=, UA=, client CID=, server CID=4000000a27839153, downstream=127.0.0.1:49528, 127.0.0.1:8443=local, drain=none
client log:
I1101 23:26:13.728768 14319 HQClient.cpp:48] HQClient connecting to 127.0.0.1:8443
E1101 23:26:13.730736 14319 HQClient.cpp:65] Failed to establish QUIC connection: Client closed by peer reason=Internal error
E1101 23:26:13.730742 14319 HQClient.cpp:255] HQClient failed to connect, error=Internal Error, msg=Client closed by peer reason=Internal error
E1101 23:26:13.730860 14319 QuicTransportBase.cpp:417] close threw exception Cannot encrypt (insufficient space for tag) client CID= server CID=4000000a27839153 peer address=127.0.0.1:8443
update 02.11.2023:
HQClient works well with h3, but firefox and chrome still work only with http2
Also, in both browsers I have
alt-svc | h3-27=":443"; ma=3600
I usually skip the --protocol arg completely -- does that fix your issues? I know chrome -> hq server is working.
Unfortunately it doesn't work :(
If you just run
./hq --mode=server --cert=<your cert> --key=<your key> --port 8443 -v 4
And hit with chrome, what's the server output?
If you just run
./hq --mode=server --cert=<your cert> --key=<your key> --port 8443 -v 4And hit with chrome, what's the server output?
E1109 12:34:45.815054 7856 main.cpp:44] Cannot open /tmp/logs
I1109 12:34:45.816812 7857 AeadTokenCipher.cpp:39] Updating token secrets
I1109 12:34:45.816824 7857 AeadTokenCipher.cpp:48] Updating token secrets, num=1
I1109 12:34:45.817651 7856 AeadTokenCipher.cpp:39] Updating token secrets
I1109 12:34:45.817658 7856 AeadTokenCipher.cpp:48] Updating token secrets, num=1
I1109 12:34:45.818532 7857 SSLContextManager.cpp:833] cert /etc/http32webserver/cert/localhost.crt Identity: localhost
I1109 12:34:45.818539 7857 SSLContextManager.cpp:836] cert /etc/http32webserver/cert/localhost.crt SAN:
I1109 12:34:45.818593 7857 SSLContextManager.cpp:732] For vip vip_, setting sid_ctx localhost
I1109 12:34:45.818710 7857 SSLContextManager.cpp:740] On VipID=[::1]:6667 context=0x7f6788017d40
I1109 12:34:45.818728 7857 SSLContextManager.cpp:1091] Adding CN/Subject-alternative-name "localhost" for SNI search
I1109 12:34:45.819213 7857 AeadTokenCipher.cpp:39] Updating token secrets
I1109 12:34:45.819218 7857 AeadTokenCipher.cpp:48] Updating token secrets, num=1
I1109 12:34:45.819875 7857 SSLContextManager.cpp:833] cert /etc/http32webserver/cert/localhost.crt Identity: localhost
I1109 12:34:45.819881 7857 SSLContextManager.cpp:836] cert /etc/http32webserver/cert/localhost.crt SAN:
I1109 12:34:45.819924 7857 SSLContextManager.cpp:732] For vip vip_, setting sid_ctx localhost
I1109 12:34:45.819931 7857 SSLContextManager.cpp:740] On VipID=[::1]:6667 context=0x7f67880bf360
I1109 12:34:45.819937 7857 SSLContextManager.cpp:1091] Adding CN/Subject-alternative-name "localhost" for SNI search
I1109 12:34:45.820389 7857 AeadTokenCipher.cpp:39] Updating token secrets
I1109 12:34:45.820394 7857 AeadTokenCipher.cpp:48] Updating token secrets, num=1
I1109 12:34:45.821027 7857 SSLContextManager.cpp:833] cert /etc/http32webserver/cert/localhost.crt Identity: localhost
I1109 12:34:45.821031 7857 SSLContextManager.cpp:836] cert /etc/http32webserver/cert/localhost.crt SAN:
I1109 12:34:45.821075 7857 SSLContextManager.cpp:732] For vip vip_, setting sid_ctx localhost
I1109 12:34:45.821081 7857 SSLContextManager.cpp:740] On VipID=[::1]:6667 context=0x7f67880cde90
I1109 12:34:45.821087 7857 SSLContextManager.cpp:1091] Adding CN/Subject-alternative-name "localhost" for SNI search
I1109 12:34:45.821547 7857 AeadTokenCipher.cpp:39] Updating token secrets
I1109 12:34:45.821552 7857 AeadTokenCipher.cpp:48] Updating token secrets, num=1
I1109 12:34:45.822196 7857 SSLContextManager.cpp:833] cert /etc/http32webserver/cert/localhost.crt Identity: localhost
I1109 12:34:45.822201 7857 SSLContextManager.cpp:836] cert /etc/http32webserver/cert/localhost.crt SAN:
I1109 12:34:45.822264 7857 SSLContextManager.cpp:732] For vip vip_, setting sid_ctx localhost
I1109 12:34:45.822273 7857 SSLContextManager.cpp:740] On VipID=[::1]:6667 context=0x7f67880dc390
I1109 12:34:45.822283 7857 SSLContextManager.cpp:1091] Adding CN/Subject-alternative-name "localhost" for SNI search
I1109 12:34:45.822836 7857 AeadTokenCipher.cpp:39] Updating token secrets
I1109 12:34:45.822841 7857 AeadTokenCipher.cpp:48] Updating token secrets, num=1
I1109 12:34:45.823822 7857 SSLContextManager.cpp:833] cert /etc/http32webserver/cert/localhost.crt Identity: localhost
I1109 12:34:45.823828 7857 SSLContextManager.cpp:836] cert /etc/http32webserver/cert/localhost.crt SAN:
I1109 12:34:45.823876 7857 SSLContextManager.cpp:732] For vip vip_, setting sid_ctx localhost
I1109 12:34:45.823884 7857 SSLContextManager.cpp:740] On VipID=[::1]:6667 context=0x7f67880eb2d0
I1109 12:34:45.823891 7857 SSLContextManager.cpp:1091] Adding CN/Subject-alternative-name "localhost" for SNI search
I1109 12:34:45.824409 7857 AeadTokenCipher.cpp:39] Updating token secrets
I1109 12:34:45.824412 7857 AeadTokenCipher.cpp:48] Updating token secrets, num=1
I1109 12:34:45.825254 7857 SSLContextManager.cpp:833] cert /etc/http32webserver/cert/localhost.crt Identity: localhost
I1109 12:34:45.825261 7857 SSLContextManager.cpp:836] cert /etc/http32webserver/cert/localhost.crt SAN:
I1109 12:34:45.825313 7857 SSLContextManager.cpp:732] For vip vip_, setting sid_ctx localhost
I1109 12:34:45.825321 7857 SSLContextManager.cpp:740] On VipID=[::1]:6667 context=0x7f67880f99d0
I1109 12:34:45.825330 7857 SSLContextManager.cpp:1091] Adding CN/Subject-alternative-name "localhost" for SNI search
I1109 12:34:45.825836 7857 AeadTokenCipher.cpp:39] Updating token secrets
I1109 12:34:45.825839 7857 AeadTokenCipher.cpp:48] Updating token secrets, num=1
I1109 12:34:45.826503 7857 SSLContextManager.cpp:833] cert /etc/http32webserver/cert/localhost.crt Identity: localhost
I1109 12:34:45.826508 7857 SSLContextManager.cpp:836] cert /etc/http32webserver/cert/localhost.crt SAN:
I1109 12:34:45.826552 7857 SSLContextManager.cpp:732] For vip vip_, setting sid_ctx localhost
I1109 12:34:45.826558 7857 SSLContextManager.cpp:740] On VipID=[::1]:6667 context=0x7f67880f8950
I1109 12:34:45.826565 7857 SSLContextManager.cpp:1091] Adding CN/Subject-alternative-name "localhost" for SNI search
I1109 12:34:45.827070 7857 AeadTokenCipher.cpp:39] Updating token secrets
I1109 12:34:45.827075 7857 AeadTokenCipher.cpp:48] Updating token secrets, num=1
I1109 12:34:45.827760 7857 SSLContextManager.cpp:833] cert /etc/http32webserver/cert/localhost.crt Identity: localhost
I1109 12:34:45.827769 7857 SSLContextManager.cpp:836] cert /etc/http32webserver/cert/localhost.crt SAN:
I1109 12:34:45.827828 7857 SSLContextManager.cpp:732] For vip vip_, setting sid_ctx localhost
I1109 12:34:45.827836 7857 SSLContextManager.cpp:740] On VipID=[::1]:6667 context=0x7f67881168a0
I1109 12:34:45.827843 7857 SSLContextManager.cpp:1091] Adding CN/Subject-alternative-name "localhost" for SNI search
I1109 12:34:45.828923 7866 QuicServer.cpp:274] No valid takenover fd found for address=[::1]:8443. binding on worker=0x2b7a170 workerId=0 processId=0
I1109 12:34:45.828984 7867 QuicServer.cpp:274] No valid takenover fd found for address=[::1]:8443. binding on worker=0x2b7a940 workerId=1 processId=0
I1109 12:34:45.829016 7868 QuicServer.cpp:274] No valid takenover fd found for address=[::1]:8443. binding on worker=0x2b7b110 workerId=2 processId=0
I1109 12:34:45.829037 7869 QuicServer.cpp:274] No valid takenover fd found for address=[::1]:8443. binding on worker=0x2b7b8e0 workerId=3 processId=0
I1109 12:34:45.829056 7870 QuicServer.cpp:274] No valid takenover fd found for address=[::1]:8443. binding on worker=0x2b7c0d0 workerId=4 processId=0
I1109 12:34:45.829073 7872 QuicServer.cpp:274] No valid takenover fd found for address=[::1]:8443. binding on worker=0x2b7c8e0 workerId=5 processId=0
I1109 12:34:45.829092 7873 QuicServer.cpp:274] No valid takenover fd found for address=[::1]:8443. binding on worker=0x2b7d0d0 workerId=6 processId=0
I1109 12:34:45.829123 7874 QuicServer.cpp:274] No valid takenover fd found for address=[::1]:8443. binding on worker=0x2b7d8e0 workerId=7 processId=0
I1109 12:34:45.829139 7874 QuicServer.cpp:285] Initialized all workers in the eventbase
I1109 12:34:45.829190 7856 HQServer.cpp:285] HQ server started at: [::1]:8443
I1109 12:34:47.248520 7858 AsyncSSLSocket.cpp:340] actual destruction of AsyncSSLSocket(this=0x7f6780002550, evb=0x7f6780000d80, fd=folly::NetworkSocket(-1), state=3, sslState=9, events=0)
I1109 12:34:47.248592 7858 AsyncSocket.cpp:3169] this=0x7f6780003560, AsyncSocket::handleRead() got 517 bytes
I1109 12:34:47.255234 7858 AsyncSocket.cpp:3169] this=0x7f6780003560, AsyncSocket::handleRead() got 30 bytes
I1109 12:34:47.255358 7858 FizzAcceptorHandshakeHelper.cpp:153] Fizz handshake error with (peer=[::1]:45860, local=[::1]:6667) after 6 ms; 547 bytes received & 2120 bytes sent: fizz::FizzException: received alert: certificate_unknown, in state ExpectingFinished
I1109 12:34:47.255385 7858 Acceptor.cpp:548] Acceptor=0x7f6788007680 onEmpty()
I1109 12:34:48.002563 7859 AsyncSSLSocket.cpp:340] actual destruction of AsyncSSLSocket(this=0x7f67840021e0, evb=0x7f6784000d40, fd=folly::NetworkSocket(-1), state=3, sslState=9, events=0)
I1109 12:34:48.002631 7859 AsyncSocket.cpp:3169] this=0x7f6784002f00, AsyncSocket::handleRead() got 517 bytes
I1109 12:34:48.010644 7859 AsyncSocket.cpp:3169] this=0x7f6784002f00, AsyncSocket::handleRead() got 30 bytes
I1109 12:34:48.010754 7859 FizzAcceptorHandshakeHelper.cpp:153] Fizz handshake error with (peer=[::1]:45868, local=[::1]:6667) after 8 ms; 547 bytes received & 2120 bytes sent: fizz::FizzException: received alert: certificate_unknown, in state ExpectingFinished
I1109 12:34:48.010775 7859 Acceptor.cpp:548] Acceptor=0x7f678804b2d0 onEmpty()
On older version(10.16)
I1109 13:27:25.661306 67818 QuicServerWorker.cpp:1134] QuicServer readerr: AsyncSocketException: AsyncUDPSocket::getReadBuffer() returned empty buffer, type = Invalid arguments
I1109 13:27:25.661463 67811 QuicServerWorker.cpp:1384] QuicServer shutdown all connections. addressMap=0 connectionIdMap=0
I1109 13:27:25.661550 67812 QuicServerWorker.cpp:1384] QuicServer shutdown all connections. addressMap=0 connectionIdMap=0
I1109 13:27:25.661600 67813 QuicServerWorker.cpp:1384] QuicServer shutdown all connections. addressMap=0 connectionIdMap=0
I1109 13:27:25.661643 67814 QuicServerWorker.cpp:1384] QuicServer shutdown all connections. addressMap=0 connectionIdMap=0
I1109 13:27:25.661676 67815 QuicServerWorker.cpp:1384] QuicServer shutdown all connections. addressMap=0 connectionIdMap=0
I1109 13:27:25.661716 67816 QuicServerWorker.cpp:1384] QuicServer shutdown all connections. addressMap=0 connectionIdMap=0
I1109 13:27:25.661752 67817 QuicServerWorker.cpp:1384] QuicServer shutdown all connections. addressMap=0 connectionIdMap=0
I1109 13:27:25.661772 67818 QuicServerWorker.cpp:1384] QuicServer shutdown all connections. addressMap=0 connectionIdMap=0
Relevant log line:
I1109 12:34:47.255358 7858 FizzAcceptorHandshakeHelper.cpp:153] Fizz handshake error with (peer=[::1]:45860, local=[::1]:6667) after 6 ms; 547 bytes received & 2120 bytes sent: fizz::FizzException: received alert: certificate_unknown, in state ExpectingFinished
Is your cert unexpired and trusted by your browser?
Relevant log line:
I1109 12:34:47.255358 7858 FizzAcceptorHandshakeHelper.cpp:153] Fizz handshake error with (peer=[::1]:45860, local=[::1]:6667) after 6 ms; 547 bytes received & 2120 bytes sent: fizz::FizzException: received alert: certificate_unknown, in state ExpectingFinishedIs your cert unexpired and trusted by your browser?
My certificate is unexpired and added to my browser
I have the same issue on macOS, hq http3 not working with browsers but it works with HQClient.
Update: It works on FireFox nightly.
I'm working with proxygen <v2023.10.16.00>. HQServer works well with these parameters : --host=:: --port=443 --h2port==443 --protocol=h3 --quic_version=0. Google Chrome 119.0.6045.159, Mozilla Firefox 120.0, Microsoft Edge 119.0.2151.58 could setup a h3 connection to HQServer without problem.
I'm working with proxygen <v2023.10.16.00>. HQServer works well with these parameters : --host=:: --port=443 --h2port==443 --protocol=h3 --quic_version=0. Google Chrome 119.0.6045.159, Mozilla Firefox 120.0, Microsoft Edge 119.0.2151.58 could setup a h3 connection to HQServer without problem.
I tested it on Google Chrome on macOS, but unfortunately, it didn't work. It seems that your test was conducted on Windows OS.
I'm working with proxygen <v2023.10.16.00>. HQServer works well with these parameters : --host=:: --port=443 --h2port==443 --protocol=h3 --quic_version=0. Google Chrome 119.0.6045.159, Mozilla Firefox 120.0, Microsoft Edge 119.0.2151.58 could setup a h3 connection to HQServer without problem.
I tested it on Google Chrome on macOS, but unfortunately, it didn't work. It seems that your test was conducted on Windows OS.
I built and tested on Rocky Linux 9 - not Windows, btw i didn't use the default TLS certificates in the sample code https://github.com/facebook/proxygen/blob/main/proxygen/httpserver/samples/hq/FizzContext.cpp, just imported the real TLS certificates from Let's Encrypt
What's the current status here - are folks able to make it work with chrome/firefox on various platforms?
What's the current status here - are folks able to make it work with chrome/firefox on various platforms?
Hi, on MacBook Pro, Apple M2 Max as the client and Proxygen as the server on a linux machine.
- Firefox Nightly supports h3.
- Chrome supports h3.
- Safari does not receive h3 from Proxygen, although it does receive h3 from the Cloudflare asset server. (Should I downgrade QUIC version in proxygen? )
Can you open a separate issue for Safari + proxygen h3 server?
Are you using these args: --h2port==443 --protocol=h3 --quic_version=0 ?
These are important for some browsers, which require getting an Alt-Svc header from the h2 server before attempting QUIC. Please include the -v 4 debug logs from the server on a failed attempt.