JakkuSakura/kafcat

new subcommand: admin api

Closed this issue · 5 comments

neverchanje commented

The kafka's official admin CLI sucks. I don't want to run Docker-based kafka-topics.sh nor run Java program. Would you accept the idea of adding admin commands to this tool?

JakkuSakura commented

Sure! What kind of admin commands do you want to support at top tier? I love the concept of kaf, which is already in the roadmap #8 .

neverchanje commented

At least I'd like to create topics beforehand in order to dump files to kafka. A more natural way of managing topics in the use-case of kafkacat is to directly use the file name as the topic name.

For example, starting with a clean-slate kafka cluster, we can provide an option --file-as-topic that creates a Kafka topic using the file name if it doesn't exist. And after that, the command will ingest the file content into the topic.

kafkacat --create-if-not-exists --file-as-topic -b '127.0.0.1:9002' example.json

Finally it will create a topic named "example" and copy example.json into Kafka.

JakkuSakura commented

It shouldn't be hard to implement. But for now, I have little time to work on this project. Do you mind implementing it on your own?

TODO:

  • inspect.rs: turn information about the cluster and watermark of the topic into rust struct
  • hook.rs: do some operation before performing dumping/copying
  • --create-if-not-exists as a pre-produce hook
  • --file-as-topic in main
neverchanje commented

Yep I'm glad to do that! Thanks for your information!

neverchanje commented

Just realized that Kafka supports automatic creation of topics auto.create.topics.enable, so for now, I no longer need admin API from the client-side. Other people encountering a wider scope of requirements may create another issue.