Add discoveryInfo section for Broker Task
Closed this issue · 2 comments
Mesos provides a mechanism for service discovery applications to get the information for specific tasks, for example ports that are used and protocols. Right now kafka framework doesn't use this. I would be really helpful to have this information per broker added, especially now when Kafka 0.9.0.0 supports SSL on a different port.
Example:
broker-1
with plaintext
for broker-to-broker communication and SSL
for producers
and consumers
, and JMX
endpoint for monitoring Kafka with some app inside the cluster
discoveryInfo {
name: broker-1,
environment: prod,
location: us,
version: 0.9.0.0,
ports: [{
name: plaintext,
number: 9092,
protocol: tcp,
visibility: FRAMEWORK // only for internal framework use
},{
name: ssl,
number: 9093,
protocol: tcp,
visibility: EXTERNAL // for secure message producing and consuming
},{
name: jmx,
number: 9010,
protocol: tcp,
visibility: CLUSTER // for monitoring kafka from inside the cluster
}]
}
This would be good to-do when we move to the new HTTP API and also move to supporting new Mesos features too. This should be near top of lists to-do with that imho.
Also, it will be very helpful to have ADVERTISED_PORT
from broker config to go into discoveryInfo as well, so we can build some proxying on top of that info.