samuelcolvin/arq

Redis Cluster support

kamkash opened this issue · 1 comments

kamkash commented

I have a requirement to support Redis in cluster and non-cluster mode.
I have been researching adding Redis Cluster support to arq.
This is what I have done so far:

  1. Add "cluster: bool = False" flag to RedisSettings
  2. import RedisCluster
    from redis.asyncio.cluster import RedisCluster
  3. In cluster mode extend class ArqRedis from RedisCluster
    class ArqRedis(RedisCluster)

This works and I am able to connect to a Redis cluster.
However, there are several issues with redis.asyncio.cluster.RedisCluster

  1. It does not support the pipeline command with transactions
  2. It does not support MULTI/EXEC

arq uses the above commands heavily for atomic access to jobs, job results, etc
Does anyone have any insights into this? Has anyone looked at cluster support in arq?
I looked at aioredis-cluster python client, and that too has the same missing transactions support

Wh1isper commented

I developed brq for the same reason, maybe you can have a look :D

  • 👍1