OOB communication (e.g. via pub/sub) and client-side queue filtering

[mnarayan01]

Three desirable features:

1. Support for dynamically changing limits, i.e. Issue #4. PR #6 provides some support in this direction, but it would be nice to be able to go further.
2. Support for queue-based concurrency throttling at a per-worker level. Note that PR #6 actually already provides this, just less efficiently than it might otherwise be done.
3. Ability to globally disable/re-enable a particular queue, i.e. Issue #18.

I've listed the three of these in a single issue as the following two high-level enhancements would be applicable to both:

1. Support "pushing" data to the various workers. The obvious mechanism to accomplish this would be pub/sub, though the redis engine being used will need to effectively support these operations.

This would enable (1) from above and assist with (3).
2. Support for temporarily disabling queues on the worker end.

This would enable (2) from above and assist with (3).

Caveats

The big caveat here is that it's debatable how related to this project the above actually are. I've seen some evidence of existing support for "pushing" data to workers via the current UI; if such support exists and is sufficiently robust, then we'll actually need to do very little. If not, however, then things grow more complicated.

This caveat applies even more heavily to support for disabling queues on the worker end; the very granularity involved (job versus queue) is different from the rest of the repo. Really the reasons to include it in here would be a vague sharing of purpose and, probably more importantly, convenience.

[mnarayan01]

Existing machinery for OOB communication:

• Send: Sidekiq::Process#signal
• Receive: Sidekiq::Launcher#❤

[mnarayan01]

Could potentially take advantage of Sidekiq::redis_pool being a ConnectionPool to support pub/sub style communication, but it may not be worth it.

[mnarayan01]

Substantially resolved by PR #21.