Support for clients on realservers
kaner opened this issue · 1 comments
Maybe I'm missing something, but out-of-the-box seesaw doesn't seem to support having the backend servers initiate TCP connections to outside servers (say, Gmail mail servers) for LVS-DR.
Here is a longer form explanation of an example setup with one outside client, one outside server, seesaw load balancer with a VIPand three backend real servers:
CIP = Client IP address (outside)
SIP = Server IP address (outside)
VIP = Virtual IP address of the seesaw service / vserver
RIP1 = Real (backend) server IP address number 1
RIP2 = Real (backend) server IP address number 2
RIP3 = Real (backend) server IP address number 3
Typically for LVS-DR, a client from outside will initiate TCP (say, port 25) connection with [source IP=CIP, source port=34567, destination IP=VIP, destination port=25] to the load balancer. Load balancer (if it has a matching vserver configured for port 25) then forwards via MAC address re-write to one of the backend servers, say server with RIP3. TCP Server living on that server sees a TCP SYN packet with [source IP=CIP, source port=34567, destination IP=VIP, destination port=25], the VIP being configured on one of its dummy interfaces. The TCP server on port 25 on the backend server with RIP3 will then respond with a SYN,ACK TCP packet with [source IP=VIP, source port=25, destination IP=CIP, destination port=34567] directly to the router and the packets end up on the client with IP address=CIP that originally initiated the TCP connection.
I am just describing this for clarity. This is what normal LVS-DR looks like and what seesaw typically does. This is when TCP connections are initiated from an outside client towards seesaw/VIP.
What about the reverse? Say I have a TCP client on one of the real (backend) servers. I want this client to initiate TCP connection with [source IP=VIP, source port=34568, destination IP=SIP, destination IP=25]. For the initial SYN package, this works fine, as it is send directly via the router, not through the seesaw director node. The outside server listening on SIP then responds with SYN,ACK packet with [source IP=SIP, source port=25, destination IP=VIP, destination port=34568]. Then the problem occurs: The seesaw node sees the packet coming in and, since its sent towards a port that isn't configured as a vserver (ephemeral port 34568), drops the packet and the connection isn't established.
Now, if I add the following configuration to the seesaw node by hand, it does work:
# iptables -I INPUT -p tcp -m tcp -d $VIP --sport 25 -j MARK --set-mark 0x1
# ipvsadm -A -f 1 -s rr
# ipvsadm -a -f 1 -r $RIP3 -g # Just using RIP3 as an example here
Is there any other way to do this? As it looks now, I would have to fork seesaw to add this functionality into the code so it works with seesaw node switchover etc.
Attempting to draw the example setup.
Normal server mode:
+----+
|RIP1|
+----+
+---+ +-------------+
|CIP| | | +----+
+-+-+ +---------+ | | |RIP2|
| SYN | | S | seesaw | +----+
+---->| Router +-->| with |
| | | VIP +--------+
+---+ | | | | SYN |
|SIP| +---------+ +-------------+ v
+---+ ^ +----+
+-----------------------------+RIP3|
SYN,ACK +----+
Client connection from the backend:
+----+
|RIP1|
+---+ +-------------+ +----+
|CIP| | |
+---+ +---------+ | |
SYN,ACK| |S,A| seesaw | +----+
+---->| Router +-->| with | |RIP2|
| | | | VIP | +----+
+-+-+ | | | |
|SIP| <-+---------+ +-------------+
+---+ S ^ +----+
| SYN |RIP3|
+------------------------------+----+