WorkMQ

A message queue system written in Go.

This is a message queue implementation written in Go. It allows to declare some queues, number of workers and processors that will process data sent in these queues.

Initially used for a Golang workshop study case, we've decided to put it open-source.

Installation

First, install it:

$ go get -u github.com/unikorp/workmq

Then, import it in your application code:

import (
    "github.com/unikorp/workmq"
)

Configuration

Queues and workers configuration is managed using a config.json file in the root directory.

Here is an example JSON with 2 queues, listening on UDP port and exposing the given HTTP port:

{
  "ports": {
    "udp": ":10001",
    "http": ":8080"
  },
  "queues": {
    "queue.1s": {
      "processor": "processor.logger.1s",
      "num_workers": 150
    },
    "queue.2s": {
      "processor": "processor.logger.2s",
      "num_workers": 200
    }
  }
}

Here, we have 2 queues:

queue.1s that will be processed by registered processor processor.logger.1s and will use 150 workers (goroutines),
queue.2s that will be processed by registered processor processor.logger.2s and will use 200 workers (goroutines).

Usage

Here is a code example that initializes WorkMQ, registers a processor and start handling messages:

package main

import (
	"fmt"
	"time"

	"github.com/unikorp/workmq"
)

func main() {
	app := workmq.Init()

	app.AddProcessor("processor.logger.1s", func(worker *workmq.Worker, message workmq.Message) {
		time.Sleep(time.Second * 1)

		fmt.Printf("Worker #%d (queue: \"%s\") manages message %s\n", worker.ID, worker.Queue, message.Body)
	})

	app.Handle()
}

Send data

You can send message data over UDP by sending a JSON string with the following structure:

{ "queue": "queue.1s", "body": "<your data>" }