/python-redis-multiwrite

Simultaneous redis write operations against many servers.

Primary LanguagePython

Documentation

Installation

To setup the tests:

virtualenv .venv
. .venv/bin/activate
pip install nose redis eventlet

Then you can repeatedly run:

nosetests

To install the package redismultiwrite onto your system:

sudo python setup.py install

Usage

Create redis.StrictRedis[1] objects for the local datacenter redis instance. This object will be the authoritative connection, and will determine the success or failure of subsequent functions.

Then create a list of redis.StrictRedis objects for each remote datacenter redis instance. Cross-datacenter commands will also propogate to these connections before return. Create the redismultiwrite.RedisMultiWrite object using these two pieces of information:

local = redis.StrictRedis(host='localhost')
remote = [redis.StrictRedis(host='redis.example.com')]
conn = redismultiwrite.RedisMultiWrite(local, remote)

The new RedisMultiWrite object will functionally mimic a StrictRedis object, routing all standard calls to the local StrictRedis object.

The RedisMultiWrite object will also allow each method to be suffixed with _everywhere. These methods correspond to the original, with the added benefit of simultaneously being performed against all the remote redis connections.

For example, if I called delete_everywhere, a DELETE command would be sent to every supplied local and remote redis connection. However, only the local redis connection's response will be returned (or thrown) by delete_everywhere. The responses or exceptions generated by remote redis connections will be logged and discarded.

If the optional flag wait_for_remote to the RedisMultiWrite constructor is given True, then the _everywhere methods will wait until each remote connection has completed. By default, the functions return as soon as the local connection has finished and remote connections continue to wait in the background.

For transactions (like the pipe() method of StrictRedis), there is a method pipe_everywhere(). This command takes a sequence of two-item tuples: a command string and a tuple of argument strings. For example:

conn.pipe_everywhere([('get', ('mykey', )),
                      ('delete', ('mykey', ))])

This library uses eventlet to perform simultaneous socket operations.