Consuming from different clusters
dpalumbo5 opened this issue · 2 comments
Hello, we are currently using Racecar for a few of our consumers in our system that work as intended, but we hit a snag trying to add in new consumers that are get consuming from a different cluster/url. We are trying to add consumers that will get messages from kafka topics that are hosted on a separate cluster with different broker urls and can't figure out how to configure both within the same micro-service.
When we deploy the service it seems that the Racecar config is global and will only allow us to consume from one of the clusters due to the Racecar.config.brokers setting.
We tried setting the RACECAR_BROKERS argument for our new consumer on a new cluster the deployment level, but it appears to be overwritten by our Racecar config for the other consumers.
Is it possible to have multiple Racecar configurations within a given service?
It might be that you should remove the code setting Racecar.config.brokers and rely exclusively on an env var for that setting.
@dasch thank you so much! we’ll take a look at trying that. We ended up with a workaround in our racecar config file. We defined two different configs and we apply them based on the name of the broker(s). So for the second config, we just appended our deployment command with “PROTON” which is the name of the second cluster and if that’s present we use a different config.
Appreciate you getting back to us!