bilalcaliskan/kafka-ansible-role

An Ansible role which installs and configures Apache Kafka on Redhat/Debian based hosts

Kafka Ansible Role

Installs and configures Apache Kafka on Redhat/Debian based hosts.

By default, this role sets up Apache Zookeeper and Apache Kafka on the same server. But you can change this behavior by setting below variables while running the playbook. Check the Examples section for ready to use examples.

seperate_zookeepers: true
zookeepers:
  - zookeepernode01

Requirements

This role has below requirements:

• Python 3.x
• Ansible (min 2.4, suggested 2.9.16)

You can install suggested version with pip3:

$ pip3 install "ansible==2.9.16"

Note that this role requires root access, so either run it in a playbook with a global become: true, or invoke the role in your playbook.

Role Variables

See the default values in defaults/main.yml. You can overwrite them in vars/main.yml if neccessary or you can set them while running playbook.

Also please see the Redhat ansible_os_family specific variables in vars/redhat.yml and Debian ansible_os_family specific variables in vars/debian.yml.

Please note that this role can ensure that firewalld systemd service on your servers are started and enabled by default. If you want to start and enable firewalld service, please modify below variable as true while running playbook:

firewalld_enabled: true

Dependencies

That role requires bilalcaliskan.zookeeper role

Examples

Seperate Zookeeper, Seperate Kafka Installation

inventory.ini:

[zookeepers]
  zookeepernode01

[brokers]
  brokernode02

playbook.yaml:

---

- name: Zookeeper role execution play
  hosts: zookeepers
  become: true
  roles:
    - role: bilalcaliskan.zookeeper
      vars:
        install: true

- name: Kafka role execution play
  hosts: brokers
  become: true
  roles:
    - role: bilalcaliskan.kafka
      vars:
        install_kafka: true
        install_zookeeper: false
        seperate_zookeepers: true
        zookeepers:
          - zookeepernode01

Seperate Zookeeper, Seperate Kafka Uninstallation

inventory.ini:

[zookeepers]
  zookeepernode01

[brokers]
  brokernode02

playbook.yaml:

---

- name: Zookeeper role execution play
  hosts: zookeepers
  become: true
  roles:
    - role: bilalcaliskan.zookeeper
      vars:
        install: false

- name: Kafka role execution play
  hosts: brokers
  become: true
  roles:
    - role: bilalcaliskan.kafka
      vars:
        install_kafka: false
        install_zookeeper: false
        seperate_zookeepers: true
        zookeepers:
          - zookeepernode01

Kafka + Zookeeper on the same server installation

inventory.ini:

[brokers]
  brokernode01
  brokernode02
  brokernode03

playbook.yaml:

---

- name: Kafka role execution play
  hosts: brokers
  become: true
  roles:
    - role: bilalcaliskan.kafka
      vars:
        install_kafka: true
        install_zookeeper: true
        seperate_zookeepers: false

Kafka + Zookeeper on the same server uninstallation

inventory.ini:

[brokers]
  brokernode01
  brokernode02
  brokernode03

playbook.yaml:

---

- name: Kafka role execution play
  hosts: brokers
  become: true
  roles:
    - role: bilalcaliskan.kafka
      vars:
        install_kafka: false
        install_zookeeper: false
        seperate_zookeepers: false

Development

This project requires below tools for development:

• Python 3.x
• Ansible - (min 2.4, suggested 2.9.16)
• pre-commit
• ansible-lint - required by pre-commit
• Bash shell - required by pre-commit

After you install all the tools above, you can simply configure pre-commit by typing:

$ pre-commit install

License

Apache License 2.0