OpenKruise/Kruise

What is NEW!
Nov 24th, 2019. A blog about new UnitedDeployment controller is posted in Kruise Blog (link).

Kruise is the core of the OpenKruise project. It is a set of controllers which extends and complements Kubernetes core controllers on workload management.

Today, Kruise offers three workload controllers:

Advanced StatefulSet: An enhanced version of default StatefulSet with extra functionalities such as inplace-update, pasue and MaxUnavailable.
BroadcastJob: A job that runs Pods to completion across all the nodes in the cluster.
SidecarSet: A controller that injects sidecar containers into the Pod spec based on selectors and also is able to upgrade the sidecar containers.

Please see documents for more technical information.

The project roadmap is actively updated in here.

Several tutorials are provided to demonstrate how to use the workload controllers. A video by Lachlan Evenson is also provided for demonstrating the controllers.

Getting started

Check before installation

You should check the cluster before installation via the command-line with either curl or wget.

via curl

sh -c "$(curl -fsSL https://raw.githubusercontent.com/openkruise/kruise/master/scripts/check_for_installation.sh)"

via wget

sh -c "$(wget -O- https://raw.githubusercontent.com/openkruise/kruise/master/scripts/check_for_installation.sh)"

Install with helm charts

wget https://raw.githubusercontent.com/openkruise/kruise/master/hack/auto_generate_charts.sh
chmod +x auto_generate_charts.sh
./auto_generate_charts.sh
helm install kruise charts/

Install with YAML files

Install CRDs

kubectl apply -f https://raw.githubusercontent.com/kruiseio/kruise/master/config/crds/apps_v1alpha1_broadcastjob.yaml
kubectl apply -f https://raw.githubusercontent.com/kruiseio/kruise/master/config/crds/apps_v1alpha1_sidecarset.yaml
kubectl apply -f https://raw.githubusercontent.com/kruiseio/kruise/master/config/crds/apps_v1alpha1_statefulset.yaml

Install kruise-controller-manager

kubectl apply -f https://raw.githubusercontent.com/kruiseio/kruise/master/config/manager/all_in_one.yaml

Or run from the repo root directory:

kustomize build config/default | kubectl apply -f -

To Install Kustomize, check kustomize website.

Note that use Kustomize 1.0.11. Version 2.0.3 has compatibility issues with kube-builder

The official kruise-controller-manager image is hosted under docker hub.

Usage examples

Advanced StatefulSet

apiVersion: apps.kruise.io/v1alpha1
kind: StatefulSet
metadata:
  name: sample
spec:
  replicas: 3
  serviceName: fake-service
  selector:
    matchLabels:
      app: sample
  template:
    metadata:
      labels:
        app: sample
    spec:
      readinessGates:
        # A new condition must be added to ensure the pod remain at NotReady state while the in-place update is happening
      - conditionType: InPlaceUpdateReady
      containers:
      - name: main
        image: nginx:alpine
  podManagementPolicy: Parallel  # allow parallel updates, works together with maxUnavailable
  updateStrategy:
    type: RollingUpdate
    rollingUpdate:
      # Do in-place update if possible, currently only image update is supported for in-place update
      podUpdatePolicy: InPlaceIfPossible
      # Allow parallel updates with max number of unavailable instances equals to 2
      maxUnavailable: 2

Broadcast Job

Run a BroadcastJob that each Pod computes pi, with ttlSecondsAfterFinished set to 30. The job will be deleted in 30 seconds after the job is finished.

apiVersion: apps.kruise.io/v1alpha1
kind: BroadcastJob
metadata:
  name: broadcastjob-ttl
spec:
  template:
    spec:
      containers:
        - name: pi
          image: perl
          command: ["perl",  "-Mbignum=bpi", "-wle", "print bpi(2000)"]
      restartPolicy: Never
  completionPolicy:
    type: Always
    ttlSecondsAfterFinished: 30

SidecarSet

The yaml file below describes a SidecarSet that contains a sidecar container named sidecar1

# sidecarset.yaml
apiVersion: apps.kruise.io/v1alpha1
kind: SidecarSet
metadata:
  name: test-sidecarset
spec:
  selector: # select the pods to be injected with sidecar containers
    matchLabels:
      app: nginx
  containers:
  - name: sidecar1
    image: centos:7
    command: ["sleep", "999d"] # do nothing at all

Developer Guide

There's a Makefile in the root folder which describes the options to build and install. Here are some common ones:

Build the controller manager binary

make manager

Run the tests

make test

Build the docker image, by default the image name is openkruise/kruise-manager:v1alpha1

export IMG=<your_image_name> && make docker-build

Push the image

export IMG=<your_image_name> && make docker-push or just docker push <your_image_name>

Generate manifests e.g. CRD, RBAC YAML files etc.

make manifests

To develop/debug kruise controller manager locally, please check the debug guide.

Uninstall

To uninstall kruise from a Kubernetes cluster:

export KUBECONFIG=PATH_TO_CONFIG
sh -c "$(curl -fsSL https://raw.githubusercontent.com/kruiseio/kruise/master/scripts/uninstall.sh)"

Note that this will lead to all resources created by Kruise, including webhook configurations, services, namespace, CRDs, CR instances and Pods managed by Kruise controller, to be deleted! Please do this ONLY when you fully understand the consequence.

Community

If you have any questions or want to contribute, you are welcome to communicate most things via GitHub issues or pull requests.

Other active communication channels:

Slack: channel address
Mailing List: todo
Dingtalk Group(钉钉讨论群)

Copyright

Certain implementation relies on existing code from Kubernetes and the credit goes to original Kubernetes authors.

wangyapu/kruise