Bohr is a scientist who continuously observes your system and submits his results to journals, forever, till he dies. Silly guy.
The bohr
program is a single-executable file that can be run on
any host to collect system (CPU, memory, disk, filesystem, network,
&c.) and service (database, webserver, end-user application &c.)
data and forward it to downstream data stores and/or other services.
Jump to:
- Installation
- Usage
- Configuration
- Available Observers
- Available Journals
- Customizing Bohr
- Contributing to Bohr
Tools similar to Bohr include:
The fundamental concept in Bohr is an observation: a measurement of some aspect of the environment. Measurements always have a host and a time. They are typically (but not necessarily) quantitative in nature and can be decorated with various metadata.
Bohr does not store, downsample, plot/graph, index, alert on, or analyze any of its observations. It merely publishes them to downstream services (called journals) which you configure.
Running Bohr processes on different hosts do not need to communicate with each other in any way.
- Reliability
- The collection of metrics is the first step in a long-chain of systems and code supporting business-critical monitoring and alerting. If it fails, everything downstream of it fails, so Bohr tries to keep observing, no matter what.
- Transparency
- It’s always easy to ask Bohr what observations it makes based on the current configuration.
- Modularity
- Bohr lets you include and exclude observations and journals with fine-grained control.
- Extensibility
- Bohr can be extended via a Clojure DSL to make new, custom observations of your systems or submit to new downstream journals.
- Efficiency
- Bohr consumes minimal resources as it runs.
- Portability
- Bohr runs on any system which can run Java.
- Adaptability
- Bohr adapt the observations it makes appropriately for the environment it’s running within (e.g. - OS X vs. Linux).
- Simplicity
- Bohr is packaged as a single, self-contained, executable Java jar and auto-discovers common settings.
Bohr was written in Clojure for two major reasons:
- the JVM
- allows porting across different runtimes (Windows, OS X, Linux, &c.)
- allows simple packaging as a self-executable JAR file
- connects to JMX directly, making it possible to observe (the plethora of) Java-based services out there
- LISPs
- have
eval
allowing Bohr to extend itself dynamically at runtime without recompilation - have permissive syntax & macros for Bohr’s DSL
- have
Bohr has three central concepts:
- Observations are data.
- Journals are code which publish data to downstream services.
- Observers are code which make observations and submit them to journals.
Using Bohr effectively requires:
- Choosing journals which publish to your environment’s data collection and/or processing services (see available journals below).
- For the relevant aspects of your host and its services:
a) Choosing bundled observers if available (see available observers below). b) Writing your own observers in Bohr’s DSL (see customizing Bohr below).
- Deploying Bohr with an appropriate configuration file defining & configuring these journals & observers (see deployment below).
More details on each of these concepts follows.
An observation is simply a measurement of some aspect of a host, a service it runs, or anything else you can think of.
Each observation has a
- name
- A string, typically dot-separated (e.g. -
mem.util.free
). (required) - timestamp
- When the observation was made. (required) [Default: current time]
- value
- Any string, numeric, or boolean value (or
nil
). [Default: =nil=] - units
- A string representing the units of
value
. [Default: =nil=] - description
- A string decribing the observation. [Default: =nil=]
- attributes
- A map of key-value pairs to associate with the observation. [Default: ={}=]
An observer makes observations and submits them to journals.
Each observer has a
- name
- A string describing that observer’s subject
(e.g.
memory
). (required) - period
- An integer time in seconds between consecutive
observations made by this observer. A
nil
value implies an observer which only runs once, at Bohr startup. [Default: =nil=]
A journal accepts submitted observations from observers and publishes them to external services.
Each journal has a
- name
- A string describing this journal’s publishing target
(e.g.
riemann
). (required)
For simplicity, Bohr is distributed as a single file which you can download and run:
$ curl https://github.com/dhruvbansal/bohr/releases/download/v0.1.0/bohr > bohr
$ chmod +x bohr
$ ./bohr --help
While running, Bohr does not daemonize itself or manage any of its output. You should configure services such as upstart and logrotate. Example configuration files for such services are provided in the examples directory.
An Ansible role for installing Bohr is also available, see the bohr-ansible-role repository.
Bohr’s user interface is the command line program bohr
(see
Installation above if you don’t have bohr
installed). You can get
help from Bohr via the following command:
$ bohr --help
Bohr has three modes of operation:
In summary mode, Bohr will make all his observations (see Configuration below) and print out his ready-to-publish values in a human-readable table. No observations will actually be submitted to journals to be published downstream.
Bohr will default to summary mode when invoked:
$ bohr ...
Summary mode works with configuration files and passing additional observers and journals on the command-line:
$ bohr my_observer.clj my_journal.clj --config /etc/bohr/bohr.yml --config /etc/bohr/conf.d
This makes summary mode useful to gain transparency into what Bohr is observing.
The --once
(or -o
) option
$ bohr --once ...
$ bohr -o ...
makes Bohr skip printing a summary table and instead submit all observations to journals ONCE. This is useful when testing or debugging because Bohr will simply “ping” downstream services with a single wave of observations instead of continuously sending them (see the periodic mode below).
If Bohr doesn’t detect any journals at runtime, it will default to
the console
journal which prints observations to STDOUT
.
As in summary mode, configuration files and additional observers and journals passed in on the command line can be used:
$ bohr my_observer.clj my_journal.clj --config /etc/bohr/bohr.yml --config /etc/bohr/conf.d --once
The --periodic
(or -p
) option
$ bohr --periodic ...
$ bohr -p ...
makes Bohr run continuously, making and submitting observations to journals based on their pre-defined periods. This is the mode Bohr should run in production.
If Bohr doesn’t detect any journals at runtime, it will default to
the console
journal which prints observations to STDOUT
.
As in summary mode, configuration files and additional observers and journals passed in on the command line can be used:
$ bohr my_observer.clj my_journal.clj --config /etc/bohr/bohr.yml --config /etc/bohr/conf.d --periodic
Bohr accepts several configuration options via the command-line but more complex configuration for specific observers or journals is best provided via a YAML configuration file.
Tell Bohr to read a YAML configuration file (or a directory of such
files) by passing the --config
option to the bohr
command:
$ bohr --config=/etc/bohr/bohr.yml --config=/etc/bohr/conf.d ...
Bohr’s configuration files can be used to:
- limit which bundled observers Bohr will run
- provide locations from which to load external observer & journal definitions
- declare which journals Bohr will submit observations to, and provide configuration options for them (hosts, ports, credentials, &c.)
- filter observers and observations
- provide observer-specific configuration options (processes, files, directories to watch, &c.)
A complete example configuration file lists available configuration options.
Bohr will merge configuration files in the order it reads them, with
later files taking precedence. This merge is done intelligently,
allowing for configurations such as the following example using the
ps
observer, which can be configured to watch the processes
defined in the processes.tracked
list:
# in app1.yml
---
processes.tracked:
- name: app1
user: app1_user
cmd: 'java.*com.example.app1.*start'
# in app2.yml
---
processes.tracked:
- name: app2
user: root
cmd: 'python.*app2'
Now invoking Bohr as follows:
$ bohr --config app1.yml --config app2.yml ...
would cause the ps
observer to watch both applications as the
processes.tracked
lists are concatenated during the merge. This
intelligent merging also works for hashes.
The end-result is that it is easy to shard Bohr’s configuration
among many different (per-application, perhaps) files in a directory
(say /etc/bohr.conf.d
) and have Bohr intelligently observe all of
them. This is especially useful from a DevOps perspective.
Bohr knows how to observe the following aspects of your systems:
TBD – just run `bohr` for now and look at its summary…
In addition to publishing observations to STDOUT
via the console
journal, Bohr knows how to publish to the following services:
- Riemann
- Many more planned…
Bohr can be extended in two ways:
- by writing new observers
- by writing new journals to which observers submit observations
Clojure scripts implementing new observers and journals are automatically evaluated in a namespace containing several helper functions and macros – essentially a lightweight DSL.
These scripts can be passed to Bohr directly on the command line:
$ bohr my_observer.clj my_journal.clj ...
or included via a configuration file:
---
load:
- /var/lib/bohr/*.clj
Observers are data structures with the following fields:
- name
- The name of the observer (e.g. -
mem
) - period
- The number of seconds an observer will wait between observations.
- prefix
- A prefix to add to the name of any observations made by the observer.
- suffix
- A suffix to add to the name of any observations made by the observer.
- units
- Default units for any observations made by the observer.
- attributes
- Default key-value pairs for any observations made by the observer.
Observers also contain a function which should implement making the observation and submitting it journals.
The following functions are available from within observers
Observers are created using Bohr’s observe
DSL macro.
A journal is a Clojure function which takes the following arguments defining an observation:
- name
- The name of the observation (e.g. -
mem.util.free
) - value
- The value of the observation (e.g. - 37)
- options
- A map with the following keys:
- units
- The units of the value (e.g. - “B”, “s”, ‘%”, &c.)
- desc
- A description of the observation (e.g. - “Percentage of memory free”)
- attributes
- Arbitrary key-value data about the observation
and publishes this to some downstream data store or service. A
function is defined to be a journal via the define-journal!
function. The example below defines a simple version of the
built-in console
journal:
;; in simple_console_journal.clj
(define-journal!
"simple_console"
(fn [name value options]
(println name value)))
See the built-in journals for more detailed examples.
To contribute, follow the instructions below on how to develop on Bohr and then create a pull request!
Bohr uses the Leiningen build tool. Once you have the lein
command installed, checkout a copy of the Bohr source:
$ git clone https://github.com/dhruvbansal/bohr
$ cd bohr
Leiningen has many useful commands, but the following are especially so:
This is used for running the bohr
command from the current source
(not the compiled classes). Make sure to include the double-hyphen
(--
) to separate Leiningen options from options passed to bohr
:
$ lein run -- --help
$ lein run -- --config bohr.yml --periodic ...
Starts a Clojure REPL in the =’bohr.core= namespace.
$ lein repl
...
bohr.core=> (println (observer-count))
0
nil
bohr.core=>
Compiles and packages Bohr into a single executable JAR file:
$ lein bin
...
$ ./target/bohr --help
This task is used when preparing a new Bohr release.
It’s important to perform integration tests with the final
executable JAR as there are several kinds of bugs (mostly related
to resolving resource files) which crop up due to different
behavior in development (lein run -- ...
) and “production”
(lein bin
followed by target/bohr ...
).
Runs Bohr’s test-suite.
$ lein test
See the testing section below for more details.
The core code for Bohr lives in the src/bohr
directory just like
in any other Clojure project. This core defines the central
concepts of Bohr (observers, journals, configuration, logging, &c.)
but not any particular implementations of observers or journals.
Available observer and journal implementations are instead defined
in the resources
directory. These files will not be compiled
when Bohr’s core is compiled but they will be contained within the
self-executable JAR file Bohr is distributed as. They are loaded
from this JAR file at runtime, if required.
Development on Bohr consists then of two distinct kinds of activity:
- working on Bohr core itself in
src/bohr
- implementing particular observers and/or journals in
resources
TBD.
Copyright © 2016 Dhruv Bansal
Distributed under the Apache Public License version 2.0.