Add ZeroMQ support
Closed this issue · 4 comments
I am using argo workflows in a streaming signal processing application. As a simple example, I have a data acquisition template, followed by several data processing templates. We want to process the data in near real time, so we need to minimize latency and overhead to passing data between each of the processing containers. We have historically used ZeroMQ for this data transfer and it has been working well for us. ZeroMQ is widely used in signal processing applications.
It looks to me that using dataflows would better fit how we are processing the data, but it does not presently support ZeroMQ. It would be nice to support both ZeroMQ sources/sinks and ZeroMQ messaging between pods. Please consider adding support for it.
Would you be interested in submitting a PR to add this? Here is an example of when Jetstream was added:
In general, I'm not opposed to contributions, but I have never used argo-dataflows (I've hardly used argo-workflows) and I've never done anything in go. I think this would be more than I could take on at the moment.
Stale issue message