pupmod-simp-libkv: A Ruby repository from brandonrdn

FIXME: Ensure the badges are correct and complete, then remove this message!

Description
Setup - The basics of getting started with libkv
Usage - Configuration options and additional functionality
Function Reference
Development - Guide for contributing to the module
- Acceptance Tests - Beaker env variables

Description

libkv is an abstract library that allows puppet to access a distributed key value store, like consul or etcd. This library implements all the basic key/value primitives, get, put, list, delete. It also exposes any 'check and set' functionality the underlying store supports. This allows building of safe atomic operations, to build complex distributed systems. This library supports loading 'provider' modules that exist in other modules, and provides a first class api.

For example, you can use the following to store hostnames, and then read all the known hostnames from consul and generate a hosts file:

libkv::put("/hosts/${::clientcert}", $::ipaddress)

$hosts = libkv::list("/hosts")
$hosts.each |$host, $ip | {
  host { $host:
    ip => $ip,
  }
}

Each key specified must contain only the following characters:

a-z
A-Z
0-9
The following special characters: ._:-/

Additionally, '/./' and '/../' are disallowed in all providers as key components. The key name also must begin with '/'

When any libkv function is called, it will first call lookup() and attempt to find a value for libkv::url from hiera. This url specifies the provider name, the host, the port, and the path in the underlying store. For example:

libkv::url: 'consul://127.0.0.1:8500/puppet'
libkv::url: 'consul+ssl://1.2.3.4:8501/puppet'
libkv::url: 'file://'
libkv::url: 'etcd://127.0.0.1:2380/puppet/%{environment}/'
libkv::url: 'consul://127.0.0.1:8500/puppet/%{trusted.extensions.pp_department}/%{environment}'

Additionally libkv uses lookup to store authentication information. This information can range from ssl client certificates, access tokens, or usernames and passwords. It is exposed as a hash named libkv::auth, and will be merged by default. The keys in the auth token are passed as is to the provider, and can vary between providers. Please read the documentation on configuring 'libkv::auth' for each provider

libkv currently supports the following providers:

mock - useful for testing, as it provides a kv store that is destroyed after each catalog compilation
consul - Allows connectivity to an existing consul service

With the intention to support the following:

etcd - Allows connectivity to an existing etcd service
simp6-legacy - Implements the SIMP 6 legacy file storage api.
file - Implements a non-ha flat file storage api.

This module is a component of the System Integrity Management Platform, a compliance-management framework built on Puppet.

If you find any issues, they may be submitted to our bug tracker.

Setup

What libkv affects

FIXME: Ensure the What libkv affects section is correct and complete, then remove this message!

If it's obvious what your module touches, you can skip this section. For example, folks can probably figure out that your mysql_instance module affects their MySQL instances.

If there's more that they should know about, though, this is the place to mention:

A list of files, packages, services, or operations that the module will alter, impact, or execute.
Dependencies that your module automatically installs.
Warnings or other important notices.

Setup Requirements OPTIONAL

FIXME: Ensure the Setup Requirements section is correct and complete, then remove this message!

If your module requires anything extra before setting up (pluginsync enabled, etc.), mention it here.

If your most recent release breaks compatibility or requires particular steps for upgrading, you might want to include an additional "Upgrading" section here.

Beginning with libkv

FIXME: Ensure the Beginning with libkv section is correct and complete, then remove this message!

The very basic steps needed for a user to get the module up and running. This can include setup steps, if necessary, or it can be an example of the most basic use of the module.

Usage

FIXME: Ensure the Usage section is correct and complete, then remove this message!

This section is where you describe how to customize, configure, and do the fancy stuff with your module here. It's especially helpful if you include usage examples and code samples for doing things with your module.

Function reference

libkv::get

Connects to the backend and retrieves the data stored at key

Any $data = libkv::get(String key)

Returns: Any

Usage:

 $database_server = libkv::get("/database/${::fqdn}")
 class { "wordpress":
 	db_host => $database_server,
 }

libkv::put

Sets the data at key to the specified value

Boolean $suceeeded = libkv::put(String key, Any value)

Returns: Boolean

Usage:

libkv::put("/hosts/${::fqdn}", "${::ipaddress}")

libkv::delete

Deletes the specified key. Must be a single key

Boolean $suceeeded = libkv::delete(String key)

Returns: Boolean

Usage:

$response = libkv::delete("/hosts/${::fqdn}")

libkv::exists

Returns true if key exists

Boolean $exists = libkv::exists(String key)

Returns: Boolean

Usage:

 if (libkv::exists("/hosts/${::fqdn}") == true) {
 	notify { "/hosts/${::fqdn} exists": }
 }

libkv::list

Lists all keys in the folder named key

Hash $list = libkv::list(String key)

Returns: Hash

Usage:

 $list = libkv::list('/hosts')
 $list.each |String $host, String $ip| {
 	host { $host:
 		ip => $ip,
 	}
 }

libkv::deletetree

Deletes the whole folder named key. This action is inherently unsafe.

Boolean $succeeded = libkv::deletetree(String key)

Returns: Boolean

Usage:

$response = libkv::deletetree("/hosts")

libkv::atomic_create

Store value in key, but only if key does not exist already, and do so atomically

Boolean $suceeeded = libkv::atomic_create(String key, Any value)

Returns: Boolean

Usage:

 $id = rand(0,2048)
 $result = libkv::atomic_create("/serverids/${::fqdn}", $id)
 if ($result == false) {
 	$serverid = libkv::get("/serverids/${::fqdn}")
 } else {
 	$serverid = $id
 }
 notify("the server id of ${serverid} is indempotent!")

libkv::atomic_delete

Delete key, but only if key still matches the value of previous

Boolean $suceeded = libkv::atomic_delete(String key, Hash previous)

Returns: Boolean

Usage:

 $previous = libkv::atomic_get("/env/${::fqdn}")
 $result = libkv::atomic_delete("/env/${::fqdn}", $previous)

libkv::atomic_get

Get the value of key, but return it in a hash suitable for use with other atomic functions

Hash $previous = libkv::atomic_get(String key)

Returns: Hash

Usage:

 $previous = libkv::atomic_get("/env/${::fqdn}")
 notify { "previous value is ${previous["value"]}": }

libkv::atomic_put

Set key to value, but only if the key is still set to previous

Boolean $suceeeded = libkv::atomic_put(String key, Any value, Hash previous)

Returns: Boolean

Usage:

 $newvalue = 'new'
 $previous = libkv::atomic_get("/env/${::fqdn}")
 $result = libkv::atomic_put("/env/${::fqdn}", $newvalue, $previous)
 if ($result == true) {
 	$real = $newvalue
 } else {
 	$real = libkv::get("/env/${::fqdn}")
 }
 notify { "I updated to $real atomically!": }

libkv::atomic_list

List all keys in folder key, but return them in a format suitable for other atomic functions

Hash $list = libkv::atomic_list(String key)

Returns: Hash

Usage:

# Add a host resource for everything under /hosts

 $list = libkv::atomic_list('/hosts')
 $list.each |String $host, Hash $data| {
 	host { $host:
 		ip => $data['value'],
 	}
 }

# For each host in /hosts, atomically update the value to 'newip'

 $list = libkv::atomic_list('/hosts')
 $list.each |String $host, Hash $data| {
 	libkv::atomic_put("/hosts/${host}", "newip", $data)
 }

libkv::empty_value

Return an hash suitable for other atomic functions, that represents an empty value

Hash $empty_value = libkv::empty_value()

Returns: Hash

Usage:

 $empty = libkv::empty()
 $result = libkv::atomic_get("/some/key")
 if ($result == $empty) {
 	notify { "/some/key doesn't exist": }
 }

libkv::info

Return a hash of informtion on the underlying provider. Provider specific

Hash $provider_information = libkv::info()

Returns: Hash

Usage:

 $info = libkv::info()
 notify { "libkv connection is: ${info}": }

libkv::supports

Return an array of all supported functions

Array $supported_functions = libkv::supports()

Returns: Array

Usage:

 $supports = libkv::supports()
 if ($supports in 'atomic_get') {
 	libkv::atomic_get('/some/key')
 } else {
 	libkv::get('/some/key')
 }

libkv::pop_error

Return the error message for the last call

libkv::provider

Return the name of the current provider

String $provider_name = libkv::provider()

Returns: String

Usage:

 $provider = libkv::provider()
 notify { "libkv connection is: ${provider}": }

libkv::watch

libkv::watchtree

libkv::newlock

Development

Please read our Contribution Guide.

Acceptance tests

This module includes Beaker acceptance tests using the SIMP Beaker Helpers. By default the tests use Vagrant with VirtualBox as a back-end; Vagrant and VirtualBox must both be installed to run these tests without modification. To execute the tests run the following:

bundle install
bundle exec rake beaker:suites

Please refer to the SIMP Beaker Helpers documentation for more information.

brandonrdn/pupmod-simp-libkv

Table of Contents

Description

Setup

What libkv affects

Setup Requirements OPTIONAL

Beginning with libkv

Usage

Function reference

libkv::get

libkv::put

libkv::delete

libkv::exists

libkv::list

libkv::deletetree

libkv::atomic_create

libkv::atomic_delete

libkv::atomic_get

libkv::atomic_put

libkv::atomic_list

libkv::empty_value

libkv::info

libkv::supports

libkv::pop_error

libkv::provider

libkv::watch

libkv::watchtree

libkv::newlock

Development

Acceptance tests