Myxamatosis or myxi for short, is designed to be an AMQP aware load-balancer and proxy for RabbitMQ.
It is currently in a prototype stage.
- Route traffic to clusters or individual nodes based on AMQP username
- Transparently set all queues as mirrored
- Automatic federation of backends based on exchange locality
- Statsd integration (via an optional
myxi_stats
application) - Extension via testable/side-effect-free middleware
make
make boot
Myxi has a growing number of Unit, Integration, and Property Based Tests. There are a number of Makefile targets which are used to run the respective suites.
make test # All
make unit # Unit + Property tests only
make integration # Integration suites
Both unit
and integration
targets support an ENV
variable T
specifying a specific suite to run. For example:
make unit T=myxi_util # The eunit module, minus the '_test.erl' suffix
make integration T=myxi_frontend # A common_test suite, minus the '_SUITE.erl' suffix
There is also a sub-directory ./dev
which contains a set of foreman related configuration to start the two backend nodes
referenced in the default configuration.
Myxi supports statsd integration. The url for the statsd
instance and the graphite
namespace prefix are configurable via:
{statsd, [
{namespace, "graphite.namespace"},
{url, 'ENVIRONMENT_VARIABLE'}
]}
This needs to be entered in the apps/myxi_stats/src/myxi_stats.app.src
file or
under the myxi_stats
section in the app.config
.
TCP keepalive packets can be sent from myxi to the connected client and server sockets. See: gen_tcp
{tcp, [
{keepalive, true},
{max_connections, 1024}
]}
A frontend consists of the configuration for an acceptor pool and listen socket, and a routing mechanism to determine which backend+balancer accepted connections will be directed to.
{frontends, [
[{ip, "0.0.0.0"},
{port, 5672},
{acceptors, 30},
{router, myxi_user_router, [
[{user, <<"rabbit">>}, {backend, rabbit}],
[{user, <<"chinchilla">>}, {backend, chinchilla}]
]}]
]}
acceptors
is the initial number of processes accepting connections on the specific listen address.
Currently only myxi_user_router
is supported, which uses the LOGIN
component
from the AMQP handshake to select a backend.
Backends in myxi consist of 3 main parts:
balancer is a running instance of the myxi_balancer
behaviour which performs
balancing based on the implementation configured in {balancer, ...}
.
Currently only a very simple round-robin based algorithm is available.
middleware is any number of implementations of the myxi_middleware
behaviour.
Middleware is wrapped (left->right), and can potentially perform pre-hooks, post-callbacks, or modifications of the AMQP methods as they are forwarded from client->server.
nodes is a list of RabbitMQ node names and the port they are accepting AMQP connections on. Since RabbitMQ runs Distributed Erlang using -sname
, the listed nodes must also
confirm to short node names.
{backends, [
{rabbit, [
{balancer, myxi_roundrobin_balancer},
{middleware, [
myxi_topology_middleware,
myxi_federation_middleware,
myxi_ha_middleware
]},
{nodes, [
[{node, 'rabbit@13inches'},
{port, 5673}]
]}
]},
{chinchilla, [
{balancer, myxi_roundrobin_balancer},
{middleware, [
myxi_topology_middleware,
myxi_federation_middleware,
myxi_ha_middleware
]},
{nodes, [
[{node, 'chinchilla@13inches'},
{port, 5674}]
]}
]}
]}
The frontend/backend configuration needs to be entered in the apps/myxi_proxy/src/myxi_proxy.app.src
file or
under the myxi_proxy
section in the app.config
.
myxi_topology_middleware builds an in memory topology map (currently only exchanges), of AMQP resources and their backend locations.
As backend nodes come up (at boot, or after being unavailable), their resources are mapped via rpc:call
.
exchange.delcare
methods are passed through the proxy their success is verified asynchronously and
added to the map.
myxi_federation_middleware ensures that calls to queue.bind
for an exchange which
exists on a different physical backend will succeed.
A federation exchange will be created on the current node with the upstream being the backend on which the exchange actually lives. This requires some convention regarding configuration, see: (https://github.com/brendanhay/myxi/tree/master/dev) for an example.
myxi_ha_middleware ensures clients need no knowledge about the backend HA setup
by ensuring all queue.declare
method calls have x-ha-policy=all
appended to their arguments.
The plan is to extend this to a percentage or some other sane/non-all value in the future.
For any problems, comments or feedback please create an issue here on GitHub.
Myxi is released under the Mozilla Public License Version 2.0