calamari: A Python repository from jcsp

https://travis-ci.org/ceph/calamari.svg?branch=master

Calamari server

Calamari is a management and monitoring service for Ceph, exposing a high level REST API.

This repository contains the Calamari server-side code. There is an accompanying user interface built on the Calamari REST API at https://github.com/ceph/calamari-clients

Resources

Calamari developer documentation: http://calamari.readthedocs.org
Calamari mailing list: http://lists.ceph.com/listinfo.cgi/ceph-calamari-ceph.com
Calamari issue tracker: http://tracker.ceph.com/projects/calamari

Setting up a development environment

This code is meant to be runnable in two ways: in production mode where it is installed systemwide from packages, or in development mode where it is running out of a git repo somewhere in your home directory. The rest of this readme will discuss setting up a git clone in development mode.

In a hurry? you don't have to follow the detailed development mode installation instructions below if you use the vagrant setup in vagrant/devmode. You'll need a box named precise64 and a Vagrant that supports the salt provisioner (1.3.5 and later).

Installing dependencies

If you haven't already, install some build dependencies (apt, yum, brew, whatever) which will be needed by the pip packages:

sudo apt-get install python-virtualenv git python-dev swig libzmq-dev g++ postgresql-9.1 postgresql-server-dev-9.1

If you're on ubuntu, there are a couple packages that are easier to install with apt than with pip (and because of m2crypto weirdness)

sudo apt-get install python-cairo python-m2crypto

For RH systems:

sudo yum  install python-virtualenv git python-devel swig zeromq-devel gcc-c++ postgresql-server postgresql-devel pycairo m2crypto

Create a virtualenv (if you are on ubuntu and using systemwide installs of cairo and m2crypto, then pass --system-site-packages)
Install dependencies with pip install -r requirements/{debian,rh}/requirements.txt and pip install -r requirements/{debian,rh}/requirements.force.txt.
Install graphite and carbon, which require some special command lines:

pip install carbon --install-option="--prefix=$VIRTUAL_ENV" --install-option="--install-lib=$VIRTUAL_ENV/lib/python2.7/site-packages"
pip install git+https://github.com/ceph/graphite-web.git@calamari --install-option="--prefix=$VIRTUAL_ENV" --install-option="--install-lib=$VIRTUAL_ENV/lib/python2.7/site-packages"

Grab the GUI code, build it and place the build products in webapp/content so that when it's installed you have a webapp/content directory containing admin, dashboard and login.

Aside: unless you like waiting around, set PIP_DOWNLOAD_CACHE in your environment

Getting ready to run

Calamari server consists of multiple modules, link them into your virtualenv:

pushd calamari-common ; python setup.py develop ; popd
pushd rest-api ; python setup.py develop ; popd
pushd cthulhu ; python setup.py develop ; popd
pushd minion-sim ; python setup.py develop ; popd
pushd calamari-web ; python setup.py develop ; popd

Graphite needs some folders created:

mkdir -p ${VIRTUAL_ENV}/storage/log/webapp
mkdir -p ${VIRTUAL_ENV}/storage

The development-mode config files have some absolute paths that need rewriting in a fresh checkout, there's a script for this:

~/calamari$ dev/configure.py
Calamari repo is at: /home/vagrant/calamari, user is vagrant
Writing /home/vagrant/calamari/dev/etc/salt/master
Writing /home/vagrant/calamari/dev/calamari.conf
Complete.  Now run:
 1. `CALAMARI_CONFIG=dev/calamari.conf calamari-ctl initialize`
 2. supervisord -c dev/supervisord.conf -n

Set up postgres, the easiest way to do this is by using salt:

~/calamari$ sudo salt-call --local state.template salt/local/postgres.sls

If you want to create your database another way, examine the .sls file to see the settings that are expected. If you want to use another database like SQLite, you can edit the paths in calamari.conf.

Create the cthulhu service's database:

CALAMARI_CONFIG=dev/calamari.conf calamari-ctl initialize

Aside: the CALAMARI_CONFIG environment variable causes all the calamari services to read their configuration from an alternative location. In a package-installed system the config is loaded from /etc/calamari/calamari.conf. We use an overridden config file in development instead of having any "if DEBUG" flags anywhere in the code.

Running the server

The server processes are run for you by supervisord. A healthy startup looks like this:

calamari john$ supervisord -n -c dev/supervisord.conf
2013-12-02 10:26:51,922 INFO RPC interface 'supervisor' initialized
2013-12-02 10:26:51,922 CRIT Server 'inet_http_server' running without any HTTP authentication checking
2013-12-02 10:26:51,923 INFO supervisord started with pid 31453
2013-12-02 10:26:52,925 INFO spawned: 'salt-master' with pid 31456
2013-12-02 10:26:52,927 INFO spawned: 'carbon-cache' with pid 31457
2013-12-02 10:26:52,928 INFO spawned: 'calamari-frontend' with pid 31458
2013-12-02 10:26:52,930 INFO spawned: 'cthulhu' with pid 31459
2013-12-02 10:26:54,435 INFO success: salt-master entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2013-12-02 10:26:54,435 INFO success: carbon-cache entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2013-12-02 10:26:54,435 INFO success: calamari-frontend entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2013-12-02 10:26:54,435 INFO success: cthulhu entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

Supervisor will print complaints if something is not starting up properly. Check in the various *.log files to find out why something is broken, or run processes individually by hand to debug them (see the commands in supervisord.conf).

At this point you should have a server up and running at http://localhost:8000/ and be able to log in to the UI.

Connecting Ceph servers to Calamari

Simulated minions

Impersonate some Ceph servers with the minion simulator:

minion-sim --count=3

Real minions

If you have a real live Ceph cluster, install salt-minion on each of the servers, and configure it to point to your development instance host (mine is 192.168.0.5, substitute yours)

wget -O - https://raw.github.com/saltstack/salt-bootstrap/develop/bootstrap-salt.sh
| sudo sh && echo "master: 192.168.0.5" >> /etc/salt/minion && service
salt-minion restart

Allowing minions to join

Authorize the simulated salt minions to connect to the calamari server:

salt-key -c dev/etc/salt -L
salt-key -c dev/etc/salt -A

You should see some debug logging in cthulhu.log, and if you visit /api/v2/cluster in your browser a Ceph cluster should appear.

jcsp/calamari