yiippee/gnet

🦋 gnet is a high-performance, lightweight, non-blocking, event-driven networking framework written in pure Go./ gnet 是一个高性能、轻量级、非阻塞的基于事件驱动的 Go 网络框架。

English | 🇨🇳中文

📖 Introduction

gnet is an event-driven networking framework that is fast and small. It makes direct epoll and kqueue syscalls rather than using the standard Go net package, and works in a similar manner as netty and libuv.

The goal of this project is to create a server framework for Go that performs on par with Redis and Haproxy for packet handling.

gnet sells itself as a high-performance, lightweight, non-blocking, event-driven networking framework written in pure Go which works on transport layer with TCP/UDP/Unix-Socket protocols, so it allows developers to implement their own protocols(HTTP, RPC, WebSocket, Redis, etc.) of application layer upon gnet for building diversified network applications, for instance, you get an HTTP Server or Web Framework if you implement HTTP protocol upon gnet while you have a Redis Server done with the implementation of Redis protocol upon gnet and so on.

gnet derives from the project: evio while having a much higher performance.

🚀 Features

• High-performance event-loop under networking model of multiple threads/goroutines
• Built-in load balancing algorithm: Round-Robin
• Built-in goroutine pool powered by the library ants
• Built-in memory pool with bytes powered by the library pool
• Concise APIs
• Efficient memory usage: Ring-Buffer
• Supporting multiple protocols: TCP, UDP, and Unix Sockets
• Supporting two event-driven mechanisms: epoll on Linux and kqueue on FreeBSD
• Supporting asynchronous write operation
• Flexible ticker event
• SO_REUSEPORT socket option
• Built-in multiple codecs to encode/decode network frames into/from TCP stream: LineBasedFrameCodec, DelimiterBasedFrameCodec, FixedLengthFrameCodec and LengthFieldBasedFrameCodec, referencing netty codec, also supporting customized codecs
• Supporting Windows platform with event-driven mechanism of IOCP Go stdlib: net
• Additional load-balancing algorithms: Random, Least-Connections, Consistent-hashing and so on
• TLS support
• Implementation of gnet Client

💡 Key Designs

Networking Model of Multiple Threads/Goroutines

Multiple Reactors

gnet redesigns and implements a new built-in networking model of multiple threads/goroutines: 『multiple reactors』 which is also the default networking model of multiple threads of netty, Here's the schematic diagram:

and it works as the following sequence diagram:

Multiple Reactors + Goroutine-Pool

You may ask me a question: what if my business logic in EventHandler.React contains some blocking code which leads to blocking in event-loop of gnet, what is the solution for this kind of situation？

As you know, there is a most important tenet when writing code under gnet: you should never block the event-loop in the EventHandler.React, otherwise, it will lead to a low throughput in your gnet server, which is also the most important tenet in netty.

And the solution to that could be found in the subsequent networking model of multiple threads/goroutines of gnet: 『multiple reactors with thread/goroutine pool』which pulls you out from the blocking mire, it will construct a worker-pool with fixed capacity and put those blocking jobs in EventHandler.React into the worker-pool to make the event-loop goroutines non-blocking.

The architecture diagram of networking model:『multiple reactors with thread/goroutine pool』architecture is in here:

and it works as the following sequence diagram:

gnet implements the networking model:『multiple reactors with thread/goroutine pool』by the aid of a high-performance goroutine pool called ants that allows you to manage and recycle a massive number of goroutines in your concurrent programs, the full features and usages in ants are documented here.

gnet integrates ants and provides the pool.NewWorkerPool method that you can invoke to instantiate a ants pool where you are able to put your blocking code logic in EventHandler.React and invoke the function of gnet.Conn.AsyncWrite to send out data asynchronously in worker pool after you finish the blocking process and get the output data, which makes the goroutine of event-loop non-blocking.

The details about integrating gnet with ants are shown here.

Auto-scaling Ring Buffer

There are two ring-buffers inside gnet: inbound buffer and outbound buffer to buffer and manage inbound/outbound network data.

The purpose of implementing inbound and outbound ring-buffers in gnet is to transfer the logic of buffering and managing network data based on application protocol upon TCP stream from business server to framework and unify the network data buffer, which minimizes the complexity of business code so that developers are able to concentrate on business logic instead of the underlying function.

🎉 Getting Started

Prerequisites

gnet requires Go 1.9 or later.

Installation

go get -u github.com/panjf2000/gnet

gnet is available as a Go module, with Go 1.11 Modules support (Go 1.11+), just simply import "github.com/panjf2000/gnet" in your source code and go [build|run|test] will download the necessary dependencies automatically.

Usage Examples

The detailed documentation is located in here: docs of gnet, but let's pass through the brief instructions first.

It is easy to create a network server with gnet. All you have to do is just to make your implementation of gnet.EventHandler interface and register your event-handler functions to it, then pass it to the gnet.Serve function along with the binding address(es). Each connection is represented as a gnet.Conn interface that is passed to various events to differentiate the clients. At any point you can close a client or shutdown the server by return a Close or Shutdown action from an event.

The simplest example to get you started playing with gnet would be the echo server. So here you are, a simplest echo server upon gnet that is listening on port 9000:

Echo server without blocking logic

package main

import (
	"log"

	"github.com/panjf2000/gnet"
)

type echoServer struct {
	*gnet.EventServer
}

func (es *echoServer) React(c gnet.Conn) (out []byte, action gnet.Action) {
	out = c.Read()
	c.ResetBuffer()
	return
}

func main() {
	echo := new(echoServer)
	log.Fatal(gnet.Serve(echo, "tcp://:9000", gnet.WithMulticore(true)))
}

As you can see, this example of echo server only sets up the EventHandler.React function where you commonly write your main business code and it will be invoked once the server receives input data from a client. The output data will be then sent back to that client by assigning the out variable and return it after your business code finish processing data(in this case, it just echo the data back).

Echo server with blocking logic

package main

import (
	"log"
	"time"

	"github.com/panjf2000/gnet"
	"github.com/panjf2000/gnet/pool"
)

type echoServer struct {
	*gnet.EventServer
	pool *pool.WorkerPool
}

func (es *echoServer) React(c gnet.Conn) (out []byte, action gnet.Action) {
	data := append([]byte{}, c.Read()...)
	c.ResetBuffer()

	// Use ants pool to unblock the event-loop.
	_ = es.pool.Submit(func() {
		time.Sleep(1 * time.Second)
		c.AsyncWrite(data)
	})

	return
}

func main() {
	p := pool.NewWorkerPool()
	defer p.Release()
	
	echo := &echoServer{pool: p}
	log.Fatal(gnet.Serve(echo, "tcp://:9000", gnet.WithMulticore(true)))
}

Like I said in the 『Multiple Reactors + Goroutine-Pool』section, if there are blocking code in your business logic, then you ought to turn them into non-blocking code in any way, for instance you can wrap them into a goroutine, but it will result in a massive amount of goroutines if massive traffic is passing through your server so I would suggest you utilize a goroutine pool like ants to manage those goroutines and reduce the cost of system resources.

All gnet examples:

Echo Server

package main
import (
	"flag"
	"fmt"
	"log"

	"github.com/panjf2000/gnet"
)

type echoServer struct {
	*gnet.EventServer
}

func (es *echoServer) OnInitComplete(srv gnet.Server) (action gnet.Action) {
	log.Printf("Echo server is listening on %s (multi-cores: %t, loops: %d)\n",
		srv.Addr.String(), srv.Multicore, srv.NumLoops)
	return
}
func (es *echoServer) React(c gnet.Conn) (out []byte, action gnet.Action) {
	out = c.Read()
	c.ResetBuffer()
	return
}

func main() {
	var port int
	var multicore bool

	// Example command: go run echo.go --port 9000 --multicore true
	flag.IntVar(&port, "port", 9000, "server port")
	flag.BoolVar(&multicore, "multicore", true, "multicore")
	flag.Parse()
	echo := new(echoServer)
	log.Fatal(gnet.Serve(echo, fmt.Sprintf("tcp://:%d", port), gnet.WithMulticore(multicore)))
}

HTTP Server

package main

import (
	"bytes"
	"flag"
	"fmt"
	"log"
	"os"
	"strconv"
	"strings"
	"time"

	"github.com/panjf2000/gnet"
)

var res string

type request struct {
	proto, method string
	path, query   string
	head, body    string
	remoteAddr    string
}

type httpServer struct {
	*gnet.EventServer
	noparse bool
}

func (hs *httpServer) OnInitComplete(srv gnet.Server) (action gnet.Action) {
	log.Printf("HTTP server is listening on %s (multi-cores: %t, loops: %d)\n",
		srv.Addr.String(), srv.Multicore, srv.NumLoops)
	return
}

func (hs *httpServer) React(c gnet.Conn) (out []byte, action gnet.Action) {
	data := c.Read()
	if hs.noparse && bytes.Contains(data, []byte("\r\n\r\n")) {
		// for testing minimal single packet request -> response.
		out = appendresp(nil, "200 OK", "", res)
		c.ResetBuffer()
		return
	}
	// process the pipeline
	var req request
	leftover, err := parsereq(data, &req)
	if err != nil {
		// bad thing happened
		out = appendresp(out, "500 Error", "", err.Error()+"\n")
		action = gnet.Close
		return
	} else if len(leftover) == len(data) {
		// request not ready, yet
		return
	}
	// handle the request
	req.remoteAddr = c.RemoteAddr().String()
	out = appendhandle(out, &req)
	c.ResetBuffer()
	return
}

func main() {
	var port int
	var multicore bool
	var aaaa bool
	var noparse bool

	// Example command: go run http.go --port 8080 --multicore true
	flag.IntVar(&port, "port", 8080, "server port")
	flag.BoolVar(&aaaa, "aaaa", false, "aaaaa....")
	flag.BoolVar(&noparse, "noparse", true, "do not parse requests")
	flag.BoolVar(&multicore, "multicore", true, "multicore")
	flag.Parse()

	if os.Getenv("NOPARSE") == "1" {
		noparse = true
	}

	if aaaa {
		res = strings.Repeat("a", 1024)
	} else {
		res = "Hello World!\r\n"
	}

	http := &httpServer{noparse: noparse}
	// We at least want the single http address.
	addr := fmt.Sprintf("tcp"+"://:%d", port)
	// Start serving!
	log.Fatal(gnet.Serve(http, addr, gnet.WithMulticore(multicore)))
}

// appendhandle handles the incoming request and appends the response to
// the provided bytes, which is then returned to the caller.
func appendhandle(b []byte, req *request) []byte {
	return appendresp(b, "200 OK", "", res)
}

// appendresp will append a valid http response to the provide bytes.
// The status param should be the code plus text such as "200 OK".
// The head parameter should be a series of lines ending with "\r\n" or empty.
func appendresp(b []byte, status, head, body string) []byte {
	b = append(b, "HTTP/1.1"...)
	b = append(b, ' ')
	b = append(b, status...)
	b = append(b, '\r', '\n')
	b = append(b, "Server: gnet\r\n"...)
	b = append(b, "Date: "...)
	b = time.Now().AppendFormat(b, "Mon, 02 Jan 2006 15:04:05 GMT")
	b = append(b, '\r', '\n')
	if len(body) > 0 {
		b = append(b, "Content-Length: "...)
		b = strconv.AppendInt(b, int64(len(body)), 10)
		b = append(b, '\r', '\n')
	}
	b = append(b, head...)
	b = append(b, '\r', '\n')
	if len(body) > 0 {
		b = append(b, body...)
	}
	return b
}

// parsereq is a very simple http request parser. This operation
// waits for the entire payload to be buffered before returning a
// valid request.
func parsereq(data []byte, req *request) (leftover []byte, err error) {
	sdata := string(data)
	var i, s int
	var head string
	var clen int
	var q = -1
	// method, path, proto line
	for ; i < len(sdata); i++ {
		if sdata[i] == ' ' {
			req.method = sdata[s:i]
			for i, s = i+1, i+1; i < len(sdata); i++ {
				if sdata[i] == '?' && q == -1 {
					q = i - s
				} else if sdata[i] == ' ' {
					if q != -1 {
						req.path = sdata[s:q]
						req.query = req.path[q+1 : i]
					} else {
						req.path = sdata[s:i]
					}
					for i, s = i+1, i+1; i < len(sdata); i++ {
						if sdata[i] == '\n' && sdata[i-1] == '\r' {
							req.proto = sdata[s:i]
							i, s = i+1, i+1
							break
						}
					}
					break
				}
			}
			break
		}
	}
	if req.proto == "" {
		return data, fmt.Errorf("malformed request")
	}
	head = sdata[:s]
	for ; i < len(sdata); i++ {
		if i > 1 && sdata[i] == '\n' && sdata[i-1] == '\r' {
			line := sdata[s : i-1]
			s = i + 1
			if line == "" {
				req.head = sdata[len(head)+2 : i+1]
				i++
				if clen > 0 {
					if len(sdata[i:]) < clen {
						break
					}
					req.body = sdata[i : i+clen]
					i += clen
				}
				return data[i:], nil
			}
			if strings.HasPrefix(line, "Content-Length:") {
				n, err := strconv.ParseInt(strings.TrimSpace(line[len("Content-Length:"):]), 10, 64)
				if err == nil {
					clen = int(n)
				}
			}
		}
	}
	// not enough data
	return data, nil
}

Push Server

package main

import (
	"flag"
	"fmt"
	"log"
	"sync"
	"time"

	"github.com/panjf2000/gnet"
)

type pushServer struct {
	*gnet.EventServer
	tick             time.Duration
	connectedSockets sync.Map
}

func (ps *pushServer) OnInitComplete(srv gnet.Server) (action gnet.Action) {
	log.Printf("Push server is listening on %s (multi-cores: %t, loops: %d), "+
		"pushing data every %s ...\n", srv.Addr.String(), srv.Multicore, srv.NumLoops, ps.tick.String())
	return
}
func (ps *pushServer) OnOpened(c gnet.Conn) (out []byte, action gnet.Action) {
	log.Printf("Socket with addr: %s has been opened...\n", c.RemoteAddr().String())
	ps.connectedSockets.Store(c.RemoteAddr().String(), c)
	return
}
func (ps *pushServer) OnClosed(c gnet.Conn, err error) (action gnet.Action) {
	log.Printf("Socket with addr: %s is closing...\n", c.RemoteAddr().String())
	ps.connectedSockets.Delete(c.RemoteAddr().String())
	return
}
func (ps *pushServer) Tick() (delay time.Duration, action gnet.Action) {
	log.Println("It's time to push data to clients!!!")
	ps.connectedSockets.Range(func(key, value interface{}) bool {
		addr := key.(string)
		c := value.(gnet.Conn)
		c.AsyncWrite([]byte(fmt.Sprintf("heart beating to %s\n", addr)))
		return true
	})
	delay = ps.tick
	return
}
func (ps *pushServer) React(c gnet.Conn) (out []byte, action gnet.Action) {
	out = c.Read()
	c.ResetBuffer()
	return
}

func main() {
	var port int
	var multicore bool
	var interval time.Duration
	var ticker bool

	// Example command: go run push.go --port 9000 --tick 1s
	flag.IntVar(&port, "port", 9000, "server port")
	flag.BoolVar(&multicore, "multicore", true, "multicore")
	flag.DurationVar(&interval, "tick", 0, "pushing tick")
	flag.Parse()
	if interval > 0 {
		ticker = true
	}
	push := &pushServer{tick: interval}
	log.Fatal(gnet.Serve(push, fmt.Sprintf("tcp://:%d", port), gnet.WithMulticore(multicore), gnet.WithTicker(ticker)))
}

Codec Client/Server

Client:

// Reference https://github.com/smallnest/goframe/blob/master/_examples/goclient/client.go

package main

import (
	"encoding/binary"
	"fmt"
	"net"

	"github.com/smallnest/goframe"
)

func main() {
	conn, err := net.Dial("tcp", "127.0.0.1:9000")
	if err != nil {
		panic(err)
	}
	defer conn.Close()

	encoderConfig := goframe.EncoderConfig{
		ByteOrder:                       binary.BigEndian,
		LengthFieldLength:               4,
		LengthAdjustment:                0,
		LengthIncludesLengthFieldLength: false,
	}

	decoderConfig := goframe.DecoderConfig{
		ByteOrder:           binary.BigEndian,
		LengthFieldOffset:   0,
		LengthFieldLength:   4,
		LengthAdjustment:    0,
		InitialBytesToStrip: 4,
	}

	fc := goframe.NewLengthFieldBasedFrameConn(encoderConfig, decoderConfig, conn)
	err = fc.WriteFrame([]byte("hello"))
	if err != nil {
		panic(err)
	}
	err = fc.WriteFrame([]byte("world"))
	if err != nil {
		panic(err)
	}

	buf, err := fc.ReadFrame()
	if err != nil {
		panic(err)
	}
	fmt.Println("received: ", string(buf))
	buf, err = fc.ReadFrame()
	if err != nil {
		panic(err)
	}
	fmt.Println("received: ", string(buf))
}

Server:

package main

import (
	"encoding/binary"
	"flag"
	"fmt"
	"log"
	"time"

	"github.com/panjf2000/gnet"
	"github.com/panjf2000/gnet/pool"
)

type codecServer struct {
	*gnet.EventServer
	addr       string
	multicore  bool
	async      bool
	codec      gnet.ICodec
	workerPool *pool.WorkerPool
}

func (cs *codecServer) OnInitComplete(srv gnet.Server) (action gnet.Action) {
	log.Printf("Test codec server is listening on %s (multi-cores: %t, loops: %d)\n",
		srv.Addr.String(), srv.Multicore, srv.NumLoops)
	return
}

func (cs *codecServer) React(c gnet.Conn) (out []byte, action gnet.Action) {
	if cs.async {
		data := append([]byte{}, c.ReadFrame()...)
		_ = cs.workerPool.Submit(func() {
			c.AsyncWrite(data)
		})
		return
	}
	out = c.ReadFrame()
	return
}

func testCodecServe(addr string, multicore, async bool, codec gnet.ICodec) {
	var err error
	if codec == nil {
		encoderConfig := gnet.EncoderConfig{
			ByteOrder:                       binary.BigEndian,
			LengthFieldLength:               4,
			LengthAdjustment:                0,
			LengthIncludesLengthFieldLength: false,
		}
		decoderConfig := gnet.DecoderConfig{
			ByteOrder:           binary.BigEndian,
			LengthFieldOffset:   0,
			LengthFieldLength:   4,
			LengthAdjustment:    0,
			InitialBytesToStrip: 4,
		}
		codec = gnet.NewLengthFieldBasedFrameCodec(encoderConfig, decoderConfig)
	}
	cs := &codecServer{addr: addr, multicore: multicore, async: async, codec: codec, workerPool: pool.NewWorkerPool()}
	err = gnet.Serve(cs, addr, gnet.WithMulticore(multicore), gnet.WithTCPKeepAlive(time.Minute*5), gnet.WithCodec(codec))
	if err != nil {
		panic(err)
	}
}

func main() {
	var port int
	var multicore bool

	// Example command: go run server.go --port 9000 --multicore true
	flag.IntVar(&port, "port", 9000, "server port")
	flag.BoolVar(&multicore, "multicore", true, "multicore")
	flag.Parse()
	addr := fmt.Sprintf("tcp://:%d", port)
	testCodecServe(addr, true, false, nil)
}

For more details, check out here: examples of gnet.

I/O Events

Current supported I/O events in gnet:

• EventHandler.OnInitComplete fires when the server has been initialized and ready to accept new connections.
• EventHandler.OnOpened fires once a connection has been opened.
• EventHandler.OnClosed fires after a connection has been closed.
• EventHandler.React fires when the server receives inbound data from a socket/connection. (usually it is where you write the code of business logic)
• EventHandler.Tick fires right after the server starts and then fires every specified interval.
• EventHandler.PreWrite fires just before any data has been written to client.

Ticker

The EventHandler.Tick event fires ticks at a specified interval. The first tick fires right after the gnet server starts up and if you intend to set up a ticker event, don't forget to pass an option: gnet.WithTicker(true) to gnet.Serve.

events.Tick = func() (delay time.Duration, action Action){
	log.Printf("tick")
	delay = time.Second
	return
}

UDP

The gnet.Serve function can bind to UDP addresses.

• All incoming and outgoing packets will not be buffered but sent individually.
• The EventHandler.OnOpened and EventHandler.OnClosed events are not available for UDP sockets, only the React event.

Multi-threads

The gnet.WithMulticore(true) indicates whether the server will be effectively created with multi-cores, if so, then you must take care of synchronizing memory between all event callbacks, otherwise, it will run the server with a single thread. The number of threads in the server will be automatically assigned to the value of runtime.NumCPU().

Load Balancing

The current built-in load balancing algorithm in gnet is Round-Robin.

SO_REUSEPORT

gnet server is able to utilize the SO_REUSEPORT option which allows multiple sockets on the same host to bind to the same port and the OS kernel takes care of the load balancing for you, it wakes one socket per accpet event coming to resolved the thundering herd.

By default, gnet is not going to be haunted by the thundering herd under its networking model:『multiple reactors』which gets only one main reactor to listen on "address:port" and accept new sockets. So this SO_REUSEPORT option is trivial in gnet but note that it will fall back to the old networking model of evio when you enable the SO_REUSEPORT option.

Just use functional options to set up SO_REUSEPORT and you can enjoy this feature:

gnet.Serve(events, "tcp://:9000", gnet.WithMulticore(true), gnet.WithReusePort(true)))

Multiple built-in codecs for TCP stream

There are multiple built-in codecs in gnet which allow you to encode/decode frames into/from TCP stream.

So far gnet has four kinds of built-in codecs: LineBasedFrameCodec, DelimiterBasedFrameCodec, FixedLengthFrameCodec and LengthFieldBasedFrameCodec, which generally meets most scenarios, but still gnet allows users to customize their own codecs in their gnet servers by implementing the interface gnet.ICodec and replacing the default codec in gnet with customized codec via functional options.

Here is an example with codec, showing you how to leverage codec to encode/decode network frames into/from TCP stream.

📊 Performance

Contrasts to the similar networking libraries

On Linux (epoll)

Test Environment

# Machine information
        OS : Ubuntu 18.04/x86_64
       CPU : 8 Virtual CPUs
    Memory : 16.0 GiB

# Go version and configurations
Go Version : go1.12.9 linux/amd64
GOMAXPROCS=8

Echo Server

HTTP Server

On FreeBSD (kqueue)

Test Environment

# Machine information
        OS : macOS Mojave 10.14.6/x86_64
       CPU : 4 CPUs
    Memory : 8.0 GiB

# Go version and configurations
Go Version : go version go1.12.9 darwin/amd64
GOMAXPROCS=4

Echo Server

HTTP Server

📄 License

Source code in gnet is available under the MIT License.

👏 Contributors

Please read our Contributing Guidelines before opening a PR and thank you to all the developers who already made contributions to gnet!

🙏 Thanks

• evio
• netty
• ants
• pool
• goframe

📚 Relevant Articles

• A Million WebSockets and Go
• Going Infinite, handling 1M websockets connections in Go
• Go netpoll I/O 多路复用构建原生网络模型之源码深度解析
• gnet: 一个轻量级且高性能的 Golang 网络库

JetBrains OS licenses

gnet had been being developed with GoLand IDE under the free JetBrains Open Source license(s) granted by JetBrains s.r.o., hence I would like to express my thanks here.

💰 Backers

Support us with a monthly donation and help us continue our activities.

💎 Sponsors

Become a bronze sponsor with a monthly donation of $10 and get your logo on our README on Github.

☕️ Buy me a coffee