[Feature / Question] : Redis ACL

[lukepolo]

Preface

We have a very complex system in which we allow edge devices add jobs to a centralized redis cluster.

The Problem

In this multi-tenent system, we need to limit each edge system's access to only their own data.

What we have tried

We first attempted to add a redis ACL :

such as the ACL command ACL SETUSER client1 on >client1password allcommands ~app:version:client1-*
(allcommands would be replaced by the commands that bullmq needs)

We quickly learned that bullmq actually touches a lot of other keys (not too surprising). We looked into groups etc, and it seems what we were trying todo would be impossible.

Is there any other way that we could achieve this?

[manast]

Could you provide a bit more info, what specifically should every edge device have access to? it's only jobs or all the jobs in a given group?

[manast]

Also, these edge systems, they can only add jobs to the queue, right? what about the workers, do they also need to have limited access?

[lukepolo]

1. Every Edge device should only have access to add a job
2. will not have any workers on those edge systems

and yes only its jobs in the given group
it's only jobs or all the jobs in a given group?

[manast]

So what about defining an ACL rule for the job key for a given edge? The job keys are the ones that actually store data, the rest of the keys are only storing job ids and group ids. So basically for every edge client we would need to define a set of ACL keys that cover all the keys needed to add the job (for example the wait list, the delay list, groups, etc), AND a pattern for the job keys produced by a given edge system. You will need to generate the job ids, something like: uuid-edgeid and only allow to access job keys with the given edgeid...

[lukepolo]

yah thats where I ended up , except . I found that the waiting list had jobs from other keys (they could view them) , and since they have access to that key they could remove the other keys etc.

I could be wrong.

[manast]

So the job keys for a given edge system will be: bull:${queueName}:${uuid}-${edgeId} and every client will need to define an ACL pattern like ~bull:*:*-edgeId.

[manast]

The wait list, etc only store job ids, so there is no sensible data there. You only need to protect the jobs keys.

[lukepolo]

awesome thats great to hear , we will do that then!

[manast]

Here is a list of keys you will need at least to add access to all edge systems:

bull:queuename:groups*
bull:queuename:wait
bull:queuename:paused
bull:queuename:meta
bull:queuename:delayed
bull:queuename:events

[lukepolo]

prototype seems to be working

 const acl = ["on", `>${password}`, "allcommands"].concat(
        ...queues.map((queue) => {
          return [
            `~${bullMqPrefix}:${queue}:groups*`,
            `~${bullMqPrefix}:${queue}:id`,
            `~${bullMqPrefix}:${queue}:wait`,
            `~${bullMqPrefix}:${queue}:paused`,
            `~${bullMqPrefix}:${queue}:meta`,
            `~${bullMqPrefix}:${queue}:delayed`,
            `~${bullMqPrefix}:${queue}:events`,
            `~${bullMqPrefix}:${queue}:priority`,
            `~${bullMqPrefix}:${queue}:completed`,
            `~${bullMqPrefix}:${queue}:${username}-*`,
          ];
        })
      );

now to go through all the calls bullmq makes and change the allcommands to the commands we actually use ><