Make use of TLS session resumption

[Bouncheck]

This issue is half-proposal and half to keep the progress on this somewhere.

It should be possible to optimize establishing subsequent connections by using TLS session resumption mechanisms. This should save a noticeable amount of time and resources, especially when a lot of connections is being used.

It may even be possible to run multiple sessions with the same session information concurrently. While this seems to be improper use of this feature, the tests so far have shown that its technically possible. It would be necessary to check more diligently what would be the security implications and potential problems in more complicated scenarios.

Session resumption requires support from both the server and the client side.

Client side

I'm not really familiar with the way scylla rust driver handles making connections, but I can provide an example rust snippet that creates 1 connection and then additional 19 that reuse session information from the first one. Rust openssl allows doing that by using set_new_session_callback to grab session information.

use std::net::TcpListener;
use std::thread;
use openssl::ssl::SslContext;
use openssl::ssl::SslMethod;
use std::path::Path;
use openssl::ssl::SslFiletype;
use openssl::ssl::SslSessionCacheMode;
use std::sync::Arc;
use std::sync::Mutex;
use std::net::TcpStream;
use openssl::ssl::Ssl;
use std::io::*;

fn main() {
    let mut ctx = SslContext::builder(SslMethod::tls()).unwrap();
    ctx.set_session_cache_mode(SslSessionCacheMode::CLIENT | SslSessionCacheMode::NO_INTERNAL);
    let session = Arc::new(Mutex::new(None));
    ctx.set_new_session_callback({
        let session = session.clone();
        move |_, sess| *session.lock().unwrap() = Some(sess)
    });
    let ctx = ctx.build();
    let port = 443;
    let ssl = Ssl::new(&ctx).unwrap();
    let stream = TcpStream::connect(("google.com", port)).unwrap();
    let mut stream = ssl.connect(stream).unwrap();
    stream.write_all(b"GET / HTTP/1.1\r\nConnection: Keep-Alive\r\n\r\n").unwrap();
    stream.read_exact(&mut [0]).unwrap();

    let mut streams = Vec::new();
    for i in 1..20 {
        let mut ssl = Ssl::new(&ctx).unwrap();
        unsafe {
            ssl.set_session(session.lock().unwrap().as_ref().unwrap())
                .unwrap();
        }
        let stream = TcpStream::connect(("google.com", port)).unwrap();
        let mut stream = ssl.connect(stream).unwrap();
//        assert!(stream.ssl().session_reused());
        println!("session reused: {}", stream.ssl().session_reused());    
        stream.write_all(b"GET /robots.txt HTTP/1.1\r\nConnection: Keep-Alive\r\n\r\n").unwrap();

        let mut res = [0; 9000]; // robots.txt seems to be a little over 9000 length
        stream.read_exact(&mut res).unwrap();
        let mut tmp = String::from_utf8_lossy(&res).to_string();
        tmp.truncate(160); // printing just a little for debugging purposes
        dbg!(tmp);

        stream.read_exact(&mut [0]).unwrap();
        streams.push(stream);

    }
    // Writing another, different request and printing to ensure we are not just reading old data
    println!("Writing GET to all streams");
    for mut stream in streams {
        //assert!(stream.ssl().session_reused());
        println!("session reused: {}", stream.ssl().session_reused());
        stream.write_all(b"GET / HTTP/1.1\r\n\r\n").unwrap();
        stream.flush();	
        let mut res = [0;1000];
        stream.read_exact(&mut res).unwrap();
        let mut tmp = String::from_utf8_lossy(&res).to_string();
        dbg!(tmp);
    }
}

Server side

Currently Scylla does not support session resumption. Adding such feature properly is probably quite complicated, but we can enable one mechanism called session tickets quite easily. This mechanism does not require the server to store the session information - instead the clients have to present encrypted tickets. One way of enabling it for development purposes would be modyfing seastar/src/net/tls.cc the following way:

index 240853df..f22a8f21 100644
--- a/src/net/tls.cc
+++ b/src/net/tls.cc
@@ -62,6 +62,8 @@ module seastar;
 
 namespace seastar {
 
+gnutls_datum_t session_tickets_key;
+
 class net::get_impl {
 public:
     static std::unique_ptr<connected_socket_impl> get(connected_socket s) {
@@ -1008,6 +1010,10 @@ class session : public enable_lw_shared_from_this<session> {
                     make_ready_future<>()), _session([t] {
                 gnutls_session_t session;
                 gtls_chk(gnutls_init(&session, GNUTLS_NONBLOCK|uint32_t(t)));
+                if (session_tickets_key.size == 0) {
+                    gtls_chk(gnutls_session_ticket_key_generate(&session_tickets_key));
+                }
+                gtls_chk(gnutls_session_ticket_enable_server(session, &session_tickets_key));
                 return session;
             }(), &gnutls_deinit) {
         gtls_chk(gnutls_set_default_priority(*this));

Scylla.yaml will also require enabling client_encryption_options:

native_transport_port_ssl: 9142
client_encryption_options:
    enabled: true
    certificate: conf/db.crt
    keyfile: conf/db.key
    require_client_auth: False

You can verify your setup by running

openssl s_client -connect 127.0.0.1:9142 -reconnect -tls1_2

If everything is working correctly you should see the first connection described as New

---
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 4096 bit
Secure Renegotiation IS supported

and subsequent ones as Reused

---
Reused, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384

[mykaul]

I imagined saving the session ticket in the clients table (from server side)