socktopus provides a high-throughput (high latency and non-guaranteed) UDP datagram socket between two systems over the Internet through multipathing.

Inspired by Multipath TCP which utilises multiple physical interfaces to create a faster channel, socktopus instead uses TCP 'reflectors' (via xinetd, iptables, or application proxy servers) installed on a number of relay systems spread across the Internet. These systems are then used to multipath a flow through to its destination, relying on TCP back pressure and non-blocking writes to do the magic under the hood.

Currently under development, it is being written to assist with streaming across the Internet as the author is finding out the hard way how slow the Internet is when you are trying to stream in real time over one billion HTTP access log lines a day.

Anyone finding they are hitting a choke point between their endpoints whilst trying to stream, or batch send, large amounts of data will hopefully find socktopus useful.

Pre-flight

socktopus requires Perl 5.14 or higher and the following libraries:

Start off by fetching the source which requires you to have git installed on your workstation:

git clone https://github.com/jimdigriz/socktopus.git

Debian 'wheezy' 7.x

sudo apt-get update
sudo apt-get install -yy --no-install-recommends perl liburi-perl libdata-guid-perl libconvert-asn1-perl libanyevent-perl cpanminus
sudo cpanm AnyEvent::Handle::UDP

Usage

Connector (active) end that calls out to a passive instance:

env AE_LOG=main=+log UDPLOC=1234 UDPREM=1235:1236 ./socktopus tcp://127.0.0.1

Listener (passive) end that receives connections:

env AE_LOG=main=+log SERVICE=23461 ./socktopus

Notes

logging is controlled by the environment variable AE_LOG
if no arguments are present, the process goes into listener mode
arguments are URIs
the first URI is the destination host you are connecting to (the partner system running socktopus) and must be a tcp URI
SERVICE is the TCP listening socket, takes the syntax [host:][serv] where host defaults to [::] and serv to 23461
UDP{LOC,REM} take the syntax [host:][serv:][rhost:]rserv, where (r)host defaults to localhost and serv defaults to 23461
UDPLOC describes the local source host/port and destination host/port tuple
UDPREM describes the remote source host/port and destination host/port tuple

One use case for the UDP data channel created is to run VTun providing you with a high-throughput network interface.

URIs

Example URIs are:

tcp://host.example.com:1234
socks://user:pass@socks.example.com:1111

The following URI schemes are supported:

tcp (default port: 23461): connecting to this raw TCP socket will proxy/NAT you to your destination but of course requires you to set up something like iptables DNAT or xinetd redirect on the node to do the forwarding for you
socks (default port: 1080) [not supported]: SOCKS server
http (default port: 8080) [not supported]: HTTP proxy that supports the CONNECT method

For the non-tcp schemes, the application proxy will be asked to connect to the target system in the first argument.

Testing

in two terminals, run the example usage commands as given above
open a third terminal and run: sudo tcpdump -i lo -n -p -X portrange 1234-1236 or port 23461 or icmp
open a fourth terminal and run: { printf 'hello'; sleep 1; } | nc -q0 -u -p 1234 127.0.0.1 23461
you should see your tcpdump terminal flicker with some TCP activity
you will see though the UDP being consumed but not re-admitted at the other end, this is because the connection is triggered to be established, but is not yet in a state were packet forwarding works
re-run the last command (nc one liner) again and this time you should see your UDP packet emitted from the other end

jimdigriz/socktopus

Related Links