A Clojure library for accessing the PuppetDB REST API.
Highlights:
- Supports HTTPS with Puppet's certificates.
- Provides syntactic sugar for PuppetDB queries.
- Results come back as a lazy sequence of maps with keywordized keys.
- Supports paged queries, which come back as a complete (but lazy) set of results.
I intend for this library to be pretty simple. There are basically three things to consider: connecting, sending a request to an endpoint, and writing queries.
Both HTTP and HTTPS connections are supported.
The clj-puppetdb.core/connect
function minimally requires a host URL, including the protocol and port:
(ns clj-puppetdb.ssl-example
(:require [clj-puppetdb.core :as pdb]
[clojure.java.io :as io]))
(def client (pdb/connect "http://puppetdb:8080"))
If you're connecting over plain HTTP, no other options are supported.
If you want to connect to PuppetDB more securely, clj-puppetdb supports using SSL certificates. If you're running clj-puppetdb from a node that's managed by Puppet, you'll already have the certificates you need. Just supply them in a map after the host URL:
(ns clj-puppetdb.ssl-example
(:require [clj-puppetdb.core :as pdb]
[clojure.java.io :as io]))
;; The certname for this node is "clojure"
(def certs
{:ssl-ca-cert (io/file "/var/lib/puppet/ssl/certs/ca.pem")
:ssl-cert (io/file "/var/lib/puppet/ssl/certs/clojure.pem")
:ssl-key (io/file "/var/lib/puppet/ssl/private_keys/clojure.pem")})
(def client (pdb/connect "https://puppetdb:8081" certs))
A couple of things to note here:
- The certs are located under Puppet's SSLDIR, the location of which varies depending on OS and configuration. Use
sudo puppet config print ssldir
to find the location on your system. - The certs must be supplied as instances of
java.io.File
, not simply paths/strings. - If you're running clj-puppetdb on a node that isn't managed by Puppet, grab the files from a node that is. It'll still work.
Once you've got a connection (conn
in the below examples), you can pass it to clj-puppetdb.core/query
along with a path:
(pdb/query client "/v4/facts") ;; return all known facts
(pdb/query client "/v4/facts/operatingsystem") ;; return the "operatingsystem" fact for all nodes
The only real restriction on using the query
function like this is that the path must be URL-encoded. You can query any version of any endpoint.
The query
function also supports PuppetDB queries with a tiny bit of syntactic sugar:
;; To find all Linux nodes with more than 30 days of uptime:
(pdb/query client "/v4/nodes" [:and
[:= [:fact "kernel"] "Linux"]
[:> [:fact "uptime_days"] 30]])
Here are some notes/caveats:
- Keywords, symbols, strings, numbers, regex literals, etc. all work and are generally interchangeable. For the sake of clarity, I recommend using keywords for operators and keys.
- The
~
operator is definitely an exception. Because it's a reader macro character, you can't use it in a symbol or keyword. Instead, use the string"~"
or the keyword:match
, which will be converted automatically. - If you're having a hard time getting your query to work, you can use
clj-puppetdb.query/query->json
to see the JSON representation of your query vector. - The excellent PuppetDB Query Tutorial has tons of great information and examples. Since JSON arrays are generally valid Clojure vectors, you can actually copy/paste those examples directly into a call to
clj-puppetdb.core/query
.
For queries that may return an extremely large set of results, clj-puppetdb supports paged queries. The clj-puppetdb.core/lazy-query
function behaves much like clj-puppetdb.core/query
, but for these important differences:
- You must also supply a map containing the
:limit
and:order-by
keys. - Results are requested from the PuppetDB server only as they are consumed.
Here's a simple example of a paged query:
(ns clj-puppetdb.paging-example
(:require [clj-puppetdb.core :as pdb]))
(def certs
{:ssl-ca-cert (io/file "/var/lib/puppet/ssl/certs/ca.pem")
:ssl-cert (io/file "/var/lib/puppet/ssl/certs/clojure.pem")
:ssl-key (io/file "/var/lib/puppet/ssl/private_keys/clojure.pem")}
(def client (pdb/connect "https://puppetdb:8081" certs))
(def lazy-facts
(pdb/lazy-query client "/v4/facts"
{:limit 500 :offset 0 :order-by [{:field :name :order "asc"}]}))
The map of parameters at the end of the call to lazy-query
is worth unpacking a bit:
- The
:limit
key is required, and determines how many results to return at a time. - The
:offset
key is optional (default is 0) and determines how many results to skip at the beginning of the query. - The
:order-by
key is required, and sorts the results on the server-side. It consists of a vector of maps, where each map specifies a:field
to sort by (e.g.,:certname
or:value
) and either"asc"
for ascending order or"desc"
for descending order.
You can also supply a query vector:
(def lazy-linux-nodes
(pdb/lazy-query client "/v4/nodes" [:= [:fact "kernel"] "Linux"]
{:limit 500 :order-by [{:field :certname :order "asc"}]}))
Here are some things that I'm working on that will hopefully make this library a bit more robust:
- Handle HTTP(S) connections a bit better. Cache the certificates for SSL, do timeouts properly, etc.
- Validate queries before sending them off to the server.
Copyright © 2014 Justin Holguin