/proxenos

A rendezvous hashing and service routing toolkit for Python.

Primary LanguagePythonMIT LicenseMIT

proxenos

https://travis-ci.org/darvid/proxenos.svg?branch=master Join the chat at https://gitter.im/python-proxenos/Lobby

A Python toolkit for Rendezvous/Highest Random Weight (HRW) hash based routing.

>>> import proxenos.mappers.consul
>>> import proxenos.rendezvous
>>> mapper = proxenos.mappers.consul.ConsulClusterMapper(
...     host='localhost', port=8500)
>>> mapper.update()
>>> mapper.cluster
{SocketAddress(host=IPAddress('::'), port=8000),
 SocketAddress(host=IPAddress('::'), port=8001),
 SocketAddress(host=IPAddress('::'), port=8300),
 SocketAddress(host=IPAddress('::'), port=8500)}
>>> mapper.select('github.com', proxenos.rendezvous.HashMethod.SIPHASH)
SocketAddress(host=IPAddress('::'), port=8001)
$ proxenos select -b consul -h localhost -p 8500 github.com
0.0.0.0:8300

Features

  • Support for multiple service discovery backends possible. Currently only support for Consul is implemented, but other backends coming eventually.
  • Supports many different hash functions and PRFs. The default hashing method is the SipHash PRF.
  • Provides a command-line interface for quickly testing/debugging node selection.

Installation

$ pip install proxenos[consul]
$ # If you'd like Murmur3 support, use:
$ pip install proxenos[consul,murmur]

Configuration

Proxenos can be configured through environment variables.

  • PROXENOS_MMH_SEED (int): The seed used for MurMur3 hashes.
  • PROXENOS_MMH_OPTIMIZE (str): Set to x64 or auto.
  • PROXENOS_PRF (str): The default hashing method to use. Defaults to siphash. Can be set to one of md5, sha1, sha224, sha256, sha384, sha512, mmh_32, mmh_64, mmh_128, or siphash.
  • PROXENOS_SIPHASH_KEY (str): The 128-bit SipHash key. A default key is provided, but users are advised to generate their own.

Notes

proxenos uses Python's PRNG for calculating the weight of nodes in a cluster, rather than using a linear congruential generator (LCG) as described in the original paper on Rendezvous hashing. Seeded weights generated are limited to sys.maxsize, which means the resulting weights will differ depending on the platform architecture.

Similar projects