devshawn/kafka-shell

Add support for `.sh` kafka commands in the PATH

solarmosaic-kflorence opened this issue · 5 comments

solarmosaic-kflorence commented

Is your feature request related to a problem? Please describe.
See discussion in #15

Describe the solution you'd like
Using kafka-shell with a docker image that uses .sh commands (e.g. kafka-topics.sh instead of just `kafka-topics, which seems to be the default behavior provided by Apache Kafka) should not result in an error:

OCI runtime exec failed: exec failed: container_linux.go:346: starting container process caused "exec: \"kafka-topics\": executable file not found in $PATH": unknown
command terminated with exit code 126

Some known images that result in this problem:

Additional context
Note that with this fix in place, running commands against docker deployed kafka should work with the following additional configuration in ~/.kafka-shell/config: command_prefix: kubectl exec svc/kafka1 --

devshawn commented

Thanks for opening this issue, @solarmosaic-kflorence. I'll add support for this and associated documentation. :-)

devshawn commented

I've added support for command_file_extension on a per-cluster basis in PR #19.

This has been released in 0.1.3. Documentation can be found here. I've left the default of kafka-shell to be null, for backwards-compatibility reasons.

Let me know if this helps or if you have any other questions! 😄

solarmosaic-kflorence commented

Great, I will try that out tomorrow @devshawn.

solarmosaic-kflorence commented

It works, thanks!

Note for future users that in my use-case (kafka deployed to local Kubernetes via docker stack deploy), I must also update my config to reference the internal (to docker) addresses, like so:

version: 1
enable:
  history: true
  save_on_exit: true
  auto_complete: true
  auto_suggest: true
  inline_help: true
  fuzzy_search: true
cluster: local
clusters:
  local:
    bootstrap_servers: kafka1:19092
    zookeeper_connect: zookeeper1:2181
    ksql_server_url: http://ksql-server:8088
    command_file_extension: sh
    command_prefix: kubectl exec svc/kafka1 --

And here is the corresponding docker stack configuration file (kafka-stack.yaml):

version: '3.3'
services:
  # https://hub.docker.com/r/bitnami/zookeeper
  zookeeper1:
    image: bitnami/zookeeper:3.4.13
    hostname: zookeeper1
    ports:
      - "2181:2181"
    environment:
      ALLOW_ANONYMOUS_LOGIN: "yes"
    volumes:
      - zookeeper1-data:/bitnami/zookeeper

  # https://hub.docker.com/r/bitnami/kafka
  kafka1:
    image: bitnami/kafka:2.2.1
    hostname: kafka1
    ports:
      - "9092:9092"
    environment:
      ALLOW_ANONYMOUS_LOGIN: "yes"
      ALLOW_PLAINTEXT_LISTENER: "yes"
      KAFKA_CFG_ADVERTISED_LISTENERS: INTERNAL://kafka1:19092,EXTERNAL://${DOCKER_HOST_IP:-127.0.0.1}:9092
      KAFKA_CFG_INTER_BROKER_LISTENER_NAME: INTERNAL
      KAFKA_CFG_LISTENERS: INTERNAL://0.0.0.0:19092,EXTERNAL://0.0.0.0:9092
      KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT
      KAFKA_CFG_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_CFG_ZOOKEEPER_CONNECT: "zookeeper1:2181"
    volumes:
      - kafka1-data:/bitnami/kafka

which can be deployed like so: docker stack deploy --orchestrator kubernetes -c kafka-stack.yaml kafka-stack.

Hopefully this saves someone some time.

devshawn commented

Happy to hear it is working!