coaps+ws: SSL Context defaults to `None` which make websockets connect through HTTP
jca-klk opened this issue · 6 comments
Hello!
I am experimenting with aiocoap and WebSockets over TLS (sheme coaps+ws://
).
The basic client (/clientGET.py
) fails when run with coaps+ws
against the basic server (/server.py
) configured behind a reverse-proxy. The server behaves correctly when I connect with a basic websockets
module client.
A Wireshark capture showed that websockets
attempts to connect to the server using clear-text HTTP, despite of the scheme. The TCP port, however, was correctly set to 443.
I dug down the issue to transports/ws.py
_connect_task
where ssl_context
is set to None
.
Passing ssl=None
to websockets.connect()
makes it not create the TLS context with create_default_context()
.
websockets documentation on websockets.client.connect:
You can set ssl to a SSLContext to enforce TLS settings. When connecting to a wss:// URI, if ssl isn’t provided, a TLS context is created with create_default_context().
In this case, ssl
is provided, but set to None by default.
It seems that the issue comes from credentials.py: CredentialsMap.ssl_client_context() which returns None
when no credentials are configured, and expects the "user to fill in ssl.create_default_context()".
I believe aiocoap users should not have to configure any specific credentials when connecting to websockets over TLS, but rather have sane default as websockets
module provides, i.e. ssl.create_default_context()
somehow.
I am not sure how to fix this bug though, as I am not familiar with this library and how it handles Credentials.
Steps to reproduce
Server side
On a Linux machine with a public IP address and associated domain name, I ran the Caddy server as a TLS endpoint and reverse-proxy using the following Caddyfile. (Replace coap.example.com
with the machine domain name). Caddy's reverse-proxy automatically handles websockets, and handles also TLS certificate request.
coap.example.com
reverse_proxy :8683
On the same server, I ran the example server server.py
with no changes.
Client side
I ran the clientGET.py
changing the Message()
uri
to coaps+ws://coap.example.com/time
.
I also enabled DEBUG logging.
It fails as follow (hostname redacted):
python clientGET.py
DEBUG:asyncio:Using selector: EpollSelector
DEBUG:websockets.client:= connection is CONNECTING
DEBUG:websockets.client:> GET /.well-known/coap HTTP/1.1
DEBUG:websockets.client:> Host: coap.example.com
DEBUG:websockets.client:> Upgrade: websocket
DEBUG:websockets.client:> Connection: Upgrade
DEBUG:websockets.client:> Sec-WebSocket-Key: 9VvdWi8weg9LjMFbtWb1Bw==
DEBUG:websockets.client:> Sec-WebSocket-Version: 13
DEBUG:websockets.client:> Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits
DEBUG:websockets.client:> Sec-WebSocket-Protocol: coap
DEBUG:websockets.client:> User-Agent: Python/3.11 websockets/12.0
DEBUG:websockets.client:! failing connection with code 1006
DEBUG:websockets.client:x half-closing TCP connection
DEBUG:websockets.client:= connection is CLOSED
Failed to fetch resource:
did not receive a valid HTTP response
Wireshark capture of the request shows that something is wrong: "Dangerous misconfiguration".
Frame 4: 411 bytes on wire (3288 bits), 411 bytes captured (3288 bits)
Ethernet II, Src: HewlettP_XX:XX:XX (XX:XX:XX:XX:XX:XX), Dst: Redacted_YY:YY:YY (YY:YY:YY:YY:YY:YY)
Internet Protocol Version 4, Src: xxx.xxx.xxx.xxx, Dst: yyy.yyy.yyy.yyy
Transmission Control Protocol, Src Port: 38174, Dst Port: 443, Seq: 1, Ack: 1, Len: 345
Hypertext Transfer Protocol
[Expert Info (Warning/Security): Unencrypted HTTP protocol detected over encrypted port, could indicate a dangerous misconfiguration.]
GET /.well-known/coap HTTP/1.1\r\n
Host: coap.example.com\r\n
Upgrade: websocket\r\n
Connection: Upgrade\r\n
Sec-WebSocket-Key: 9VvdWi8weg9LjMFbtWb1Bw==\r\n
Sec-WebSocket-Version: 13\r\n
Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits\r\n
Sec-WebSocket-Protocol: coap\r\n
User-Agent: Python/3.11 websockets/12.0\r\n
\r\n
[Full request URI: http://coap.example.com/.well-known/coap]
[HTTP request 1/1]
[Response in frame: 7]
Resolution
Commenting out transports/ws.py
line 286 altogether "fixes" the problem, but I guess there is a better solution.
Extra information
Please include the output of
python3 -m aiocoap.cli.defaults
in your report,
run in the environment in which you experienced the issues.
Python version: 3.11.6 (main, Oct 8 2023, 05:06:43) [GCC 13.2.0]
aiocoap version: 0.4.7
Modules missing for subsystems:
dtls: everything there
oscore: everything there
linkheader: everything there
prettyprint: everything there
Python platform: linux
Default server transports: oscore:tinydtls:tcpserver:tcpclient:tlsserver:tlsclient:ws:udp6
Selected server transports: oscore:tinydtls:tcpserver:tcpclient:tlsserver:tlsclient:ws:udp6
Default client transports: oscore:tinydtls:tcpclient:tlsclient:ws:udp6
Selected client transports: oscore:tinydtls:tcpclient:tlsclient:ws:udp6
SO_REUSEPORT available (default, selected): True, True
Thanks, I'm having a look.
On first glance, while this is definitely bad and needs to get fixed, I don't see how this would go unnoticed, because it'd need an equally broken server that takes unencrypted connections on an encrypted port, so I'm not treating this as security critical (the severity would be minimal, given it'd need an already compromised peer). If you disagree, please feel free to file for a CVE number for this, and report it here.
That websockets behaves subtly different from OpenSSL is regrettable. It should be straightforward to fix now that you have identified the issue, although it saddens me that I'll have to add the explicit code to use the system's CA storage (which is a choice I don't like for security reasons but which makes sense for practical reasons, and OpenSSL's defaults meant that I was deferring to their defaults rather than recommending system CAs myself -- anyhow, this needs fixing).
Hi! Thanks for the quick reply.
No, I do not think this is a security issue. The reverse-proxy replies with HTTP/1.0 400 Bad Request
with data Client sent an HTTP request to an HTTPS server.
It does not go through the proxy and the client closes the TCP connection right away.
My issue is indeed on the functional part, with default settings.
I already encountered such issue with the websockets
module, and I chose to only pass ssl
parameter when required, using something like that:
wsargs = {}
if ssl_is_required:
wsargs['ssl'] = my_ssl_context
ws = await websockets.connect(uri, **wsargs)
I am not sure it is easy to do in here. I feel I miss some context after reading commit 6270d93
Could you give #326 a try? It should fix this.
For context on the change that broke this: aiocoap can also run in the browser, in which case it can only use WebSockets connections, and they exit the browser through a websocket module that just calls out to the browser's JavaScript WebSocket API. That one is not implemented using the ssl module, and has no influence on which certificates are accepted.
Tests couldn't really catch the breakage because no certificates created for a test situation will be valid under the default (system-wide) context, unless the tests install their fake CA into the system root, which I'd consider excessive.
I confirm #326 works with my setup.
Thank you for the quick responses and fix. Kudos 🎉
Indeed that should have been is not
-- pyodide isn't quite easy to test, and I haven't gotten to running it so far.
Once I'm done with those tests, this will be merged.
Thanks for reporting this, and your quick feedback!
Now tested on pyodide:
import micropip
await micropip.install("https://some-server-that-has-the-right-CORS-headers/aiocoap-0.4.7.post0-py3-none-any.whl")
from aiocoap import *
ctx = await Context.create_client_context()
req = Message(code=GET, uri="coaps+ws://rd.coap.amsuess.com/.well-known/core")
await ctx.request(req).response