FRRouting Based OCP Multiple External Gateway

❗ Red Hat does not provide commercial support for the content of these repos

#############################################################################
DISCLAIMER: THESE ARE UNSUPPORTED COMMUNITY TOOLS.

THE REFERENCES ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
#############################################################################

This repository contains an example lab configuration for using an FRR-based Pod with the OVN Kubernetes Multiple External Gateway capability.

To restrict the external gateway functionality to only work on a pre-selected set of namespaces, an example security profile using Kyverno has been included.

NOTE: At this moment Kyverno is an upstream project without a validated or certified Operator.

Installing Kyverno

oc create -f https://raw.githubusercontent.com/kyverno/kyverno/main/config/release/install.yaml

(pre-requisite) Install NMState Operator 01-nmstate-install.yaml

Update the manifests to match your environment, then apply them in the following order

# Configure NMState Operator
# Note: update to match your environment
oc apply -f 01-nmstate-nodeselector.yaml

# Configure NMState for external and internal NICs
# Note: update to match your environment
oc apply -f 02-nmstate-external-net.yaml
oc apply -f 02-nmstate-internal-net.yaml

# Create "frr" namespace and multus network definitions
# Note: update to match your environment
oc apply -f 03-create-namespace.yaml
oc apply -f 03-network-definition.yaml

# Configure Kyverno security policy
# Note: update to match your environment
oc apply -f 05-kyverno-cluster-policy.yaml

# Create example Namespaces and Pods
# Note: update to match your environment
oc apply -f 07-dummy-ns-foo-bar.yml
oc apply -f 07-dummy-pod-bar.yaml
oc apply -f 07-dummy-pod-foo.yaml

# Create ConfigMap and Pod for external gateway
# Note: update to match your environment
oc apply -f 10-frr-configmap.yaml
oc apply -f 10-frr-pod.yaml

TIPs

Generating the static routes entries for nodes subnets

oc get nodes -o jsonpath='{range .items[*].metadata.annotations}{.k8s\.ovn\.org\/node\-subnets}{.k8s\.ovn\.org\/node\-primary\-ifaddr}{"\n"}{end}' | awk -F'["/]' '{print "ip route " $4"/"$5 " " $9}'
ip route 10.128.2.0/23 198.18.111.12
ip route 10.129.0.0/23 198.18.111.13
ip route 10.128.0.0/23 198.18.111.14
ip route 10.130.0.0/23 198.18.111.15
ip route 10.131.0.0/23 198.18.111.16

A reference configuration for an upstream router is 99-frr-upstream-router.conf

williamcaban/frr-gw

FRRouting Based OCP Multiple External Gateway

TIPs