Scope leadership to a particular context or plugin

[sorentwo]

Is your feature request related to a problem? Please describe.

Leadership is acquired for an entire instance node and purely scoped by name. That causes problems when nodes run heterogeneous plugins, particularly in a "web and worker" setup.

Describe the Solution You'd Like

Scope leadership to instance name and some additional context like the plugin name. This requires adding a new Peer.leader?/3 callback that accepts some scope:

  @callback leader?(Config.t() | GenServer.server(), scope :: any(), timeout :: timeout()) :: boolean()

That changes leadership checks from Peer.leader?(conf) to Peer.leader?(conf, __MODULE__).

Describe Alternatives You've Considered

Document how to work around this in troubleshooting. However, this isn't the kind of fiddly detail that users should have to worry about.

The biggest drawback to this approach is that it increases the number of queries in a minute from N (roughly the number of nodes * 2) to N * M (number of active plugins). That's still a single to low double-digits number of queries per minute for a small cluster.

[sorentwo]

With the removal of Oban.Web.Plugins.Stats and the deprecation of Oban.Plugins.Gossip, this isn't as much of a problem now.