/KPA

Kafka-Python-Admin

Primary LanguagePythonOtherNOASSERTION

kpa

python admin tool for kafka(MSK)

requirements:

-[] python3
-[x] install requirements.txt
    -[] kafka-python
    -[] wheel
    -[] PyYaml
    -[] botocore
-[] bootstrap endpoint (broker)
-[] check if *ssl* require or not
install python3
$ sudo yum install -y python3
install pip & required packages
$ sudo python3 -m pip install --upgrade
$ sudo python3 -m pip install -r requirements.txt

to get boostrap endpoint fom MSK

TBD
Help
$ python3 ./kpa.py -h
usage: kpa.py [-h] [-v] -b BOOTSTRAP [-lts] [-dts DESC_TOPICS] [-lcgs]
                      [-dcg DESC_CONSUMER_GROUP] [-ofcg OFFSET_CONSUMER_GROUP]
                      [-c] [-d] [-t TOPIC_NAMES] [-f FILE_NAME] [--ssl]
                      [--dry] [--debug]

Kafka CLi tools (requirement python 3.x)

optional arguments:
  -h, --help            show this help message and exit
  -v                    display app version
  -b BOOTSTRAP          set bootstrap broker host
  -lts, --list_topics   display kafka topics list
  -dts DESC_TOPICS, --desc_topics DESC_TOPICS
                        display topic config description (i.e. all or topic_1,topic2)
  -lcgs, --list_consumer_groups
                        display consumer groups list
  -dcg DESC_CONSUMER_GROUP, --desc_consumer_group DESC_CONSUMER_GROUP
                        describe consumer group
  -ofcg OFFSET_CONSUMER_GROUP, --offset_consumer_group OFFSET_CONSUMER_GROUP
                        consumer group offsets
  -c, --create          create topic (optional)
  -d, --delete          delete topic (optional)
  -t TOPIC_NAMES, --topics TOPIC_NAMES
                        topic name(list)
  -f FILE_NAME          load file (json)
  --ssl                 Enable SSL
  --dry                 dry run
  --debug               turn debug mode on (option)
basic connection to kafka (example)
-- no ssl requirements
$ python3 ./kpa.py -b "bootstrap_server_address:port"

-- with SSL requirements
$ python3 ./kpa.py -b "bootstrap_server_address:port" --ssl
To create topics
-- you may edit the sample-topics.json file before create topics
$ python3 ./kpa.py -b "bootstrap_server_address:port" -c -f topics/<topics>.json --ssl
To delete topics
-- topic name (list), comma delimited
$ python3 ./kpa.py -b "bootstrap_server_address:port" -d -t "topic1,topic2,......"
To list topics
-- list topics
$ python3 ./kpa.py -b "bootstrap_server_address:port" --list_topics
To describe topic
-- topic_name (list), comma delimited
$ python3 ./kpa.py -b "bootstrap_server_address:port" --desc_topic "topic1,topic2,...."
To list consumer groups
-- list conusumer groups
$ python3 ./kpa.py -b "bootstrap_server_address:port" --list_consumer_groups
To enable debug mode
-- to add --debug flag at the end
 $ python3 ./kpa.py -b "bootstrap_server_address:port" --debug