tomflynn/k8s-bench-suite

Simple scripts to benchmark kubernetes cluster features

ShellMIT

k8s-bench-suite

Bash scripts collection to benchmark kubernetes cluster performance

knb : Kubernetes Network Benchmark

knb is a bash script that will start a networking benchmark on a target Kubernetes cluster.

Here are some highlights:

Plain bash script with very few dependencies
Complete benchmark takes only 2 minutes
Ability to select only a subset of benchmark tests to run
Testing both TCP and UDP bandwidth
Automatic detection of CNI MTU
Includes host cpu and ram monitoring in benchmark report
No ssh access required, just an access to the target cluster through standard kubectl
No need for high privileges, the script will just launch very lightweight pods on two nodes.
Based on very lights containers images :
- infrabuilder/bench-iperf3, is used to run benchmark tests
- infrabuilder/bench-custom-monitor, is used to monitor nodes

Requirements

This script needs a valid kubectl setup with an access to the target cluster.

Binaries dependencies for the host that will execute knb :

awk
grep
tail
date
kubectl

Quickstart

Choose two nodes to act as server/client on your cluster (for example node1 and node2) . Then start the knb :

./knb --verbose -client-node node1 -server-node node2

If you omit the --verbose flag, it will also complete, but you will have no output until the end of the benchmark.

Examples

Simple benchmark from "node1" to "node2" in verbose mode :
```
knb -v -cn node1 -sn node2
```
Benchmark from "nA" to "nB" and save data in file mybench.knbdata
```
knb -cn nA -sn nB -o data -f mybench.knbdata
```
Generate report in json from previous benchmark data file mybench.knbdata
```
knb -fd mybench.knbdata -o json
```
To run benchmark from node A to node B, showing only result in yaml format :
```
knb -cn A -sn B -o yaml
```
To run benchmark from node Asterix to node Obelix, with the most verbose output and a result as json in a res.json file :
```
knb --debug -cn Asterix -sn Obelix -o json -f res.json
```
Running benchmark in namespace myns :
```
knb -n myns -cn node1 -sn node2
```

Run only idle and tcp benchmark :

knb -cn clientnode -sn servernode -ot idle,tcp

Usage

To display usage, use the -h flag :

aducastel@infrabuilder:~/k8s-bench-suite$ ./knb -h

knb is a network benchmark tool for Kubernetes CNI

There are two modes :
- benchmark mode : will actually run benchmark on a cluster
- from data mode : read data generated by previous benchmark with "-o data" flag

=====[ Benchmark mode ]====================================================

 Mandatory flags :

    -cn <nodename>
    --client-node <nodename>    : Define kubernetes node name that will host the client part

    -sn <nodename>
    --server-node <nodename>    : Define kubernetes node name that will host the server part

 Optionnal flags :
    -d <time-in-scd>
    --duration <time-in-scd>    : Set the benchmark duration for each test in seconds (Default 10)

    -k
    --keep                      : Keep data directory instead of cleaning it (tmp dir that contains raw benchmark data)

    -n <namespace>
    --namespace <namespace>     : Set the target kubernetes namespace

    --name <name>               : Set the name of this benchmark run

    -ot <testlist>
    --only-tests <testlist>     : Only run a subset of benchmark tests, comma separated (Ex: -ot tcp,idle)
                                  Possible values: all, tcp, udp, p2p, p2s , p2ptcp, p2pudp, p2stcp, p2sudp, idle

    -sbs <size>
    --socket-buffer-size <size> : Set the UDP socket buffer size with unit, or 'auto'. ex: '256K' (Default: auto)

    -t <time-in-scd>
    --timeout <time-in-scd>     : Set the pod ready wait timeout in seconds (Default 30)

=====[ From Data mode ]====================================================

Mandatory flags :
    -fd <path>
    --from-data <path>          : Define the path to the data to read from

=====[ Common optionnal flags ]============================================

    --debug                     : Set the debug level to "debug"

    -dl <level>
    --debug-level <level>       : Set the debug level
                                  Possible values: standard, warn, info, debug

    -f <filepath>
    --file <filepath>           : Set the output file

    -h
    --help                      : Display this help message

    -o <format>
    --output <format>           : Set the output format
                                  Possible values: text, yaml, json, data
    -v
    --verbose                   : Activate the verbose mode by setting debug-level to 'info'

    -V
    --version                   : Show current script version

=====[ Examples ]==========================================================

  Simple benchmark from "node1" to "node2" in verbose mode
  -------------------------------------------------------------------------
  | knb -v -cn node1 -sn node2                                            |
  -------------------------------------------------------------------------

  Benchmark from "nA" to "nB" with data saved in file "mybench.knbdata"
  -------------------------------------------------------------------------
  | knb -cn nA -sn nB -o data -f mybench.knbdata                          |
  -------------------------------------------------------------------------

  Generate report in json from previous benchmark file "mybench.knbdata"
  -------------------------------------------------------------------------
  | knb -fd mybench.knbdata -o json                                       |
  -------------------------------------------------------------------------

  Run only idle and tcp benchmark :
  -------------------------------------------------------------------------
  | knb -cn clientnode -sn servernode -ot idle,tcp                        |
  -------------------------------------------------------------------------