Enhancement: Attach sinks at runtime
WhitWaldo opened this issue · 4 comments
I'd like to request a feature of attaching sinks to the pipeline at runtime during the initial setup. Presently, the sinks are in a read-only collection, so they can't be modified after the fact and it only supports reading in from either a file or IConfiguration. Ideally, I'd like to set up most of my configuration in a file, then supplement it with content at runtime.
My specific use case concerns the Elasticsearch sink and the use of credentials. Ideally, I don't want to keep these secrets in a JSON file that gets committed, nor do I want to deal with the hassle of otherwise making the file available on all the destination machines. I'd much rather put the credentials in Key Vault, then retrieve them at application load, ingest the configuration file and then supplement with the Elasticsearch sink and the newly retrieved credentials then.
Is this something you'd be open to a pull-request for?
Thank you for the suggestion, Whit!
I can think of several ways we might satisfy your requirement (which is, as I understand it, to be able to use Elasticsearch output with credentials stored in KeyVault).
How important is it for you to have an EventFlow pipeline up and running (with sinks other than the Elasticsearch one operating) while you are retrieving the credentials for Elasticsearch sink and configuring the sink?
In my case, it's not important at all that EventFlow be up and running first. I presently have the setup happening right off the bat, so inserting Key Vault retrieval logic as the only thing before it would make it easy to source issues without any logging (in case it crashes).
Besides, since it'd be the only thing there, I could just put it in a try/catch and attach a fallback sink in case of error.
OK, thanks. The simplest solution that comes to my mind is what is described in the README: https://github.com/azure/diagnostics-eventflow#store-secret-securely Would this approach suffice?
I do think that'll do it - completely missed this on the first pass through the docs. Thank you!