How to add TLS support to a websocket server for wss to work?

[abhijeetbhagat]

I have a wasm client app (using websocket support provided by the ws_stream_wasm crate) that connects to a websocket server (using websocket support provided by the async-tungstenite crate). Non-secure connection is working good. However, now I want to add a secure (wss) connection between them.
This is what I am trying to do on the server side using async-tls:

fn load_config(options: &Options) -> io::Result<ServerConfig> {
    let certs = load_certs(&options.cert)?;
    let mut keys = load_keys(&options.key)?;

    // we don't use client authentication
    let mut config = ServerConfig::new(NoClientAuth::new());
    config
          // set this server to use one cert together with the loaded private key
          .set_single_cert(certs, keys.remove(0))
          .map_err(|err| io::Error::new(io::ErrorKind::InvalidInput, err))?;

    Ok(config)
}
...

async fn handle_connection(acceptor: &TlsAcceptor, stream: TcpStream) -> Result<(), Error> {
    // Calling `acceptor.accept` will start the TLS handshake
    let handshake = acceptor.accept(stream);
    // The handshake is a future we can await to get an encrypted
    // stream back.
    let mut tls_stream = handshake.await?;

    let ws_stream = async_tungstenite::accept_async(tls_stream)
        .await
        .expect("Error during the websocket handshake occurred");
    ...   
}

The above was just copied and modified from here.

When my wasm client app now connects using a wss url, I get a Custom { kind: InvalidData, error: AlertReceived(DecryptError) } on this line: let mut tls_stream = handshake.await?;.
The wasm client app isn't passing any certificates or other information specific to TLS. All I did was just replaced ws with wss in the url part.

Is this how I should be adding support for wss on the server?

EDIT: Is there a function in async-tungstenite that I can use directly to accept a client connection over wss automatically instead of using async-tls?

[sdroege]

If you omit the whole WebSocket part, can you accept connections via that TCP connection? I have no idea how these things work with wasm and if it's even supposed to work like this.

In any case this doesn't look like an error coming from async-tungstenite but something from your TLS stack.

EDIT: Is there a function in async-tungstenite that I can use directly to accept a client connection over wss automatically instead of using async-tls?

There isn't. You'd set up your connection accepting code yourself and then once that's done pass the connections to async-tungstenite.

[abhijeetbhagat]

@sdroege you are right! regenerating the key file as a .rsa instead of .pem fixed the problem for some reason. thank you for an excellent lib!