[FEATURE] Expose wadm-internal events on a new stream

[protochron]

Right now the wadm_events stream is defined as a WorkQueue which is great for controlling what Wadm works on but is far less useful for external consumers of wadm. This is because you can only ever have a single consumer for any WorkQueue!

The issue here is that the wadm_events stream is the only place where manifest deploy and undeploy events are logged, so without polling for updates there is not a great way to be able to subscribe for only those events. An alternative which I'm pursuing is creating a stream mirror of the wadm_events stream so that I can create a new consumer so that the wasmcloud-operator can take actions whenever a new manifest is deployed or one is undeployed.

With the new features in NATS 2.10 (similar to #81), we can be much smarter in how we create new consumers and expose a stream for downstream consumers. What I'd like to do is:

• keep relaying the existing notifications to wadm_events for now.
• relay all deploy and undeploy notifications to a new stream (wadm_scheduler_events? Not sure of a good name yet). The stream's subject should be something like wadm_scheduler_events.>, and wadm should add the deploy and undeploy notifications on their own prefixed subjects, ex. wadm_scheduler_events.$lattice_id.manifest_deployed and wadm_scheduler_events.$lattice_id.manifest_undeployed. Doing so would allow consumers of the stream to only subscribe to only events they are interested on particular lattices or even only for specific events without needing to parse a CloudEvent and check the payload for the event type before deciding on what to process. This should not be a work queue and should instead be a stream with sensible limits. Downstream consumers can then mirror the stream into a work queue or anything else they want and build out consumers on that stream.
• Eventually remove the manifest deployed and undeployed events from the wadm_events stream and use a different work queue to provide the same functionality. This work might overlap with #81 since we wouldn't necessarily need one main mirror stream to drive all work in wadm.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If this has been closed too eagerly, please feel free to tag a maintainer so we can keep working on the issue. Thank you for contributing to wasmCloud!