Get only general metrics in confluent kafka

[dabelousov]

Hi, im connect that exporter with default config (added broker host and tls path) to confluent kafka cluster v2.7 with TLS.
But get only General / Cluster Metrics, and no any other metrics. What i do wrong? It is not compatible with confluent kafka?
Have try to change scrapeMode to offsetsTopic, but no result.

[weeco]

Hello,
could you please clarify what you mean by "General metrics"? I'm unsure what metrics are there and what aren't. Please also post the KMinion logs.

The offsetsTopic mode is not available in shared clusters at Confluent as they hide the topic - that's the only restriction I'm aware of.

[dabelousov]

Thank you for the quick response!
By "general metrics" i mean that metrics only - https://github.com/redpanda-data/kminion/blob/master/docs/metrics.md#general--cluster-metrics

And my logs below:
{"level":"info","ts":"2022-07-29T07:47:11.491Z","logger":"main","msg":"started kminion","version":"sha-2a62a9d24ca6579cc5dcf30c4f56215430162fd4"}
{"level":"info","ts":"2022-07-29T07:47:11.491Z","logger":"main.kafka_service","msg":"connecting to Kafka seed brokers, trying to fetch cluster metadata","seed_brokers":"bootstrap.*****:9093"}
{"level":"info","ts":"2022-07-29T07:47:11.492Z","logger":"main.kgo_client","msg":"metadata update triggered","why":"client initialization"}
{"level":"info","ts":"2022-07-29T07:47:11.539Z","logger":"main.kafka_service","msg":"successfully connected to kafka cluster","advertised_broker_count":4,"topic_count":0,"controller_id":1,"kafka_version":"v2.7"}
{"level":"info","ts":"2022-07-29T07:47:11.555Z","logger":"main.minion_service","msg":"starting to consume messages from offsets topic"}
{"level":"info","ts":"2022-07-29T07:47:11.555Z","logger":"main","msg":"listening on address","listen_address":":8080"}
{"level":"info","ts":"2022-07-29T07:47:11.765Z","logger":"main.kgo_client","msg":"assigning partitions","why":"new assignments from direct consumer","how":"assigning everything new, keeping current assignment","input":"__consumer_offsets[]"}
{"level":"info","ts":"2022-07-29T07:47:26.560Z","logger":"main.minion_service","msg":"successfully consumed all consumer offsets. consumer group lags will be exported from now on"}

Looks strange log with info "topic_count":0, because it is about 10 topics in test cluster.

[weeco]

Have you tried to GET /metrics before pasting the logs?

What I'm expecting is to see some errors when you open the logs, for example some issues to describe Kafka topics. If KMinion can't describe topics it wouldn't be able to export consumer group lags etc and then everything would make sense.

Looks strange log with info "topic_count":0, because it is about 10 topics in test cluster.

That's a good hint, does the sasl user that kminion uses have the right permissions to see and describe all topics?

[dabelousov]

Yes you right, it was a problem with certificate for test cluster, found error in server side to authorization in topic __consumer_offsets, strange that there was no error in the exporter log.
So after changing certificate i get all metrics.
Thank you for you work!

[weeco]

Thanks for reporting back 👍