grafana/xk6-disruptor

Warn about, or prevent users from disrupting pods with `hostNetwork`

Closed this issue · 0 comments

roobre commented

In order to perform disruptions, xk6-disruptor installs two simple iptables rules on the pod that a) forward traffic from the original application port to another, and b) reset new and old connections to the original application port. These two rules are cleaned up by the xk6-disruptor when execution finishes.

This does not typically pose a high risk: Rules affect only the pod's network namespace, and the effects of any potential bug that slips our tests can be almost trivially wiped out by deleting the disrupted pods, as doing so will create a new network namespace with no trace of the rules the disruptor injected.

This is, however, not true for pods with hostNetwork: true, as those share the network namespace of the host. In practical terms, this breaks some assumptions and introduces complications to the approach:

  • Injected rules may affect other applications other than the pod being targeted
  • A potential bug or unexpected termination of the disruptor process can leave netfilter rules active on the host
  • A host may have more than one interface, or a main interface not named eth0, rendering the injected rules useless

As a result, xk6-disruptor should not be used with pods that have hostNetwork: true as targets. This, however, is not currently detected or enforced.

Proposal

  • Detect if any target has hostNetwork set to true in the pod spec
  • Abort if any of the targets runs with hostNetwork

Additionally, we can include a flag to override this behavior, e.g. allowHostNetwork. If this flag is passed, the disruptor will not abort if any target runs with hostNetwork, and just emit a warning instead.