An Apache Mesos framework for Riak KV, a distributed NoSQL key-value data store that offers high availability, fault tolerance, operational simplicity, and scalability.
Note: This project is an early proof of concept. The code is a beta release and there may be bugs, incomplete features, incorrect documentation or other discrepancies.
- Installation
- Open Source Mesos Users (Non-DCOS)
- DCOS Users
- Architecture
- Development / Contributing
For the framework to work properly in your environment, a custom config file may need to be used. Here is a minimal example:
dcos-riak.json
{
"riak": {
"master": "zk://master.mesos:2181/mesos",
"zk": "master.mesos:2181",
"user": "root",
"framework-name": "riak"
}
}
{
"riak": {
"master": "zk://master.mesos:2181/mesos",
"zk": "master.mesos:2181",
"user": "root",
"framework-name": "riak",
"url": "http://riak-tools.s3.amazonaws.com/riak-mesos/ubuntu/riak_mesos_linux_amd64_0.1.1.tar.gz"
}
}
{
"riak": {
"master": "zk://master.mesos:2181/mesos",
"zk": "master.mesos:2181",
"user": "root",
"framework-name": "riak"
"url": "http://riak-tools.s3.amazonaws.com/riak-mesos/centos/riak_mesos_linux_amd64_0.1.1.tar.gz"
}
}
The Riak framework should have at least 2GB of RAM and 0.5 CPUs to perform successfully.
dcos package update
dcos package install riak --options=dcos-riak.json
Command line utility for the Riak Mesos Framework / DCOS Service.
This utility provides tools for modifying and accessing your Riak
on Mesos installation.
Usage: dcos riak <subcommands> [options]
Subcommands:
cluster list
cluster create
node list
node add [--nodes <number>]
proxy config [--zk <host:port> [--os centos|coreos|ubuntu][--disable-super-chroot]]
proxy install [--zk <host:port> [--os centos|coreos|ubuntu][--disable-super-chroot]]
proxy uninstall
proxy endpoints [--public-dns <host>]
Options (available on most commands):
--cluster <cluster-name> Default: riak-cluster
--framework <framework-name> Default: riak
--debug
--help
--info
--version
Create a 3 node cluster named 'riak-cluster' (this is the default name).
dcos riak cluster create
dcos riak node add --nodes 3
Create a second 1 node cluster named 'riak-test-cluster'.
dcos riak cluster create --cluster riak-test-cluster
dcos riak node add --cluster riak-test-cluster
The Riak Mesos Director smart proxy can be easily installed on your DCOS cluster with these commands:
dcos riak proxy install
dcos riak proxy install --os ubuntu
dcos riak proxy install --os centos
Once it is up and running, explore your proxy and Riak cluster using the following command:
dcos riak proxy endpoints --public-dns <host>
The output should look something like this:
Load Balanced Riak Cluster (HTTP)
http://<host>:10002
Load Balanced Riak Cluster (Protobuf)
http://<host>:10003
Riak Mesos Director API (HTTP)
http://<host>:10004
To remove the proxy from marathon, run this command:
dcos riak proxy uninstall
All of the tasks created by the Riak framework can be killed with the following:
dcos package uninstall riak
Note: Currently, Zookeeper entries are left behind by the framework even after uninstall. To completely remove these entries, use the tools binary included in the framework package download (links in docs/MESOS_USAGE.md). Execute the following command to remove the framework ZK references:
./tools_linux_amd64 -zk=master.mesos:2181 -name=riak -command="delete-framework"
Replace "-name=riak" with the framework name if different than "riak".
The Riak Mesos Framework scheduler will attempt to spread Riak nodes across as many different mesos agents as possible to increase fault tolerance. If there are more nodes requested than there are agents available, the scheduler will then start adding more Riak nodes to existing agents.
Due to the nature of Apache Mesos and the potential for Riak nodes to come and go on a regular basis, client applications using a Mesos based cluster must be kept up to date on the cluster's current state. Instead of requiring this intelligence to be built into Riak client libraries, a smart proxy application named Director
has been created which can run alongside client applications.
For more information related to the Riak Mesos Director and instructions on running it outside of Mesos, please read docs/DIRECTOR.md
For build and testing information, visit docs/DEVELOPMENT.md.