Add reuse_port support to TcpSocketOptions for multi-threaded TCP accept

[PooyaEimandar]

What is the problem your feature solves, or the need it fulfills?

Pingora currently does not support binding multiple run_forever() threads to the same TCP port for HTTP/1.1 and HTTP/2 servers. This limits horizontal scalability because the underlying socket is managed internally and does not allow setting the SO_REUSEPORT option. As a result, only one thread can bind and accept connections on a given port, creating a bottleneck under high connection load.

Although we can set the number of threads for the Service, only a single TCP socket is attached to one acceptor thread, which restricts true concurrency at the connection acceptance level.

This limitation affects users who need to scale their HTTP server to handle tens of thousands of concurrent connections efficiently across multiple CPU cores. High-performance applications require parallel TCP accept loops to avoid latency spikes and throughput drops.

Describe the solution you'd like

Expose more socket options such as reuse_port or allow to set socket2 per pingora manually.

Describe alternatives you've considered

Running a single run_forever() thread: This becomes a bottleneck at the accept layer and cannot utilize multiple CPU cores effectively.

What other solutions, features, or workarounds have you considered that might also solve the issue?
What are the tradeoffs for these alternatives compared to what you're proposing?

Additional context

[drcaramelsyrup]

This makes sense to us to allow setting as a config option and would welcome PRs on this.

[rudrakhp]

@drcaramelsyrup gave this a shot in #643 do review, thanks!

[Hu1-Li]

does it work with LB example in https://github.com/cloudflare/pingora/blob/main/pingora-proxy/examples/load_balancer.rs , i try this:

    let mut sock_opt = TcpSocketOptions::default();
    sock_opt.so_reuseport = Some(true);
    lb.add_tcp_with_settings("0.0.0.0:17547", sock_opt.clone());

, but ss -tulnp show only one queue for this LB program, anything i missed? @rudrakhp

[rudrakhp]

@Hu1-Li can you try initializing like in the unit tests:

        let sock_opt = TcpSocketOptions {
            so_reuseport: Some(true),
            ..Default::default()
        };

As evident from the unit tests, looks like binding multiple listeners to same port is working after this setting is enabled.

[drcaramelsyrup]

Merged the commit from #643 to allow so_reuseport, thank you.