Jikkou streamlines the management of the configurations that live on your data streams platform!

Introduction

Jikkou (jikkō / 実行) is an open-source tool designed to provide an easier way to manage the configurations on your Data Stream platform.

Developed by Kafka ❤️, Jikkou aims to streamline daily operations on Apache Kafka, ensuring that platform governance is no longer a boring and tedious task for both Developers and Administrators.

Jikkou enables a declarative management approach of Topics, ACLs, Quotas, and even more with the use of YAML files called Resource Definitions.

Taking inspiration from kubectl and Kubernetes resource definition files, Jikkou offers an intuitive and user-friendly approach to configuration management.

Motivation

See this blog post for more details.

The main usage scenarios

Create new resource entities on an Apache Kafka cluster (i.e., Topics, ACLs, and Quotas).
Update the configurations of existing resource entities.
Delete resource entities which are not anymore managed.
Describe all the configuration defined for Brokers.
Describe all the configuration defined for Topics, ACLs, and Quotas.
Replicate configurations of a production cluster to another with a few command lines.
Initialize a new cluster for testing purpose.

Core features that make it awesome

Simple command line interface (CLI) for end user.
Simple Java API on top of the Kafka’s Java AdminClient.
Completely stateless and thus does not store any state (Basically: Your kafka cluster is the state of Jikkou).
Pluggable validation rules to ensure that resources meet your requirement before being created or updated ona target cluster.
Pluggable resource manager to extend Jikkou with cloud managed services for Apache Kafka which are supported out-of-the-box.
Simple templating mechanism using Jinja notation.

Quick Installation

Jikkou is available:

As a zip/tar.gz package from GitHub Releases
As a fatJar available from Maven Central
As a docker image available from Docker Hub.
As a Debian package from GitHub Releases

wget https://github.com/streamthoughts/jikkou/releases/download/v0.21.0/jikkou.deb
sudo dpkg -i jikkou.deb

Note	Jikkou will install itself in the directory : `/opt/jikkou`

It is recommended to install the bash/zsh completion script jikkou_completion:

wget https://raw.githubusercontent.com/streamthoughts/jikkou/master/jikkou_completion . jikkou_completion

or alternatively, run the following command for generation the completion script.

source <(jikkou generate-completion)

Overview

It allows you to define the desired current state of your Topics, ACLs, or Quotas, using one or more YAML resource definition files :

kafka-topics.yml

# file:./kafka-topics.yml
apiVersion: 'kafka.jikkou.io/v1beta2'
kind: 'KafkaTopic'
metadata:
  name: 'my-first-topic-with-jikkou'
  labels: {}
  annotations: {}
spec:
  partitions: 12
  replicas: 3
  configs:
    min.insync.replicas: 2

It will then take care of computing and applying the necessary changes directly to your cluster.

$ jikkou apply -f ./kafka-topics.yml

TASK [CREATE] Create a new topic my-first-topic-with-jikkou (partitions=12, replicas=3) - CHANGED **********************

{
  "changed" : true,
  "end" : 1634071489773,
  "resource" : {
    "name" : "my-first-topic-with-jikkou",
    "operation" : "ADD",
    "partitions" : {
      "after" : 12,
      "operation" : "ADD"
    },
    "replicas" : {
      "after" : 3,
      "operation" : "ADD"
    },
    "configs" : {
      "min.insync.replicas" : {
        "after" : "2",
        "operation" : "ADD"
      }
    }
  },
  "failed" : false,
  "status" : "CHANGED"
}

EXECUTION in 2s 661ms (DRY_RUN)
ok : 0, created : 1, altered : 0, deleted : 0 failed : 0

Most of the time, you will use Jikkou part of your CI/CD pipeline allowing you to version and ship your Apache Kafka resources directly from Git (more on GitOps)

Jikkou can be used with self-hosted Kafka, managed Kafka and Confluent Cloud.

Note	you can use the environment variable `JIKKOU_DEFAULT_KAFKA_BOOTSTRAP_SERVERS` for connection to the Kafka cluster (instead of the CLI arg `--bootstrap-servers`)

Documentation

Check the official documentation for further installation and usage instructions.

🏭 Developers

You need to have Java and Docker installed.

Dependencies

Jikkou modules are built with:

Java 17
Apache Kafka Client (3.0.x)

Build project

This project includes Maven Wrapper.

For building distribution files.

$ ./mvnw clean package -Pdist

Build Docker Images (locally)

$ make

Formats

This project uses the Maven plugin Spotless to format all Java classes and to apply some code quality checks.

Bugs

This project uses the Maven plugin SpotBugs and FindSecBugs to run some static analysis to look for bugs in Java code.

Reported bugs can be analysed using SpotBugs GUI:

$ ./mvnw spotbugs:gui

💡 Contributions

Any feedback, bug reports and PRs are greatly appreciated!

Source Code: https://github.com/streamthoughts/jikkou
Issue Tracker: https://github.com/streamthoughts/jikkou/issues

🙏 Show your support

You think this project can help you or your team to manage your Apache Kafka Cluster ? Please ⭐ this repository to support us!

Licence

Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements.See the NOTICE file distributed with this work for additional information regarding copyright ownership.The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License.You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions and limitations under the License

Allsimon/jikkou