Spiderpool

Spiderpool is a CNCF Landscape Level Project.

Introduction

Spiderpool is a Kubernetes underlay network solution that enhances the capabilities of Macvlan CNI, ipvlan CNI, SR-IOV CNI.It fulfills various networking needs and enables the utilization of underlay network solutions in bare metal, virtual machine, and public cloud environments. Spiderpool delivers exceptional network performance, particularly benefiting network I/O-intensive and low-latency applications like storage, middleware, and AI.

Why does Spiderpool select macvlan, ipvlan, and SR-IOV as datapaths?

Unlike CNI solutions based on veth virtual interfaces, underlay networks eliminate layer 3 network forwarding on the host, avoiding tunnel encapsulation overhead. This translates to excellent network performance with high throughput, low latency, and reduced CPU utilization for network forwarding.
Connecting seamlessly with underlay layer 2 VLAN networks enables both layer 2 and layer 3 communication for applications. It supports multicast and broadcast communication, while allowing packets to be controlled by firewalls.
Data packages carry the actual IP addresses of Pods, enabling direct north-south communication based on Pod IPs. This connectivity across multi-cloud networks enhances flexibility and ease of use.
Underlay CNI can create virtual interfaces using different parent network interfaces on the host, providing isolated subnets for applications with high network overhead, such as storage and observability.
macvlan, ipvlan, and SR-IOV is crucial for supporting RDMA network acceleration. RDMA significantly enhances performance for latency-sensitive and network I/O-intensive applications, surpassing overlay network solutions in terms of network performance.

What enhancements does Spiderpool provide for macvlan, ipvlan, and SR-IOV CNI?

Simplified installation and usage

Spiderpool simplifies the installation process by eliminating the need for manually installing multiple components like Multus CNI. It provides streamlined installation procedures, encapsulates relevant CRDs, and offers comprehensive documentation for easy setup and management.
CRD-based dual-stack IPAM capability

Spiderpool provides exclusive and shared IP address pools, supporting various affinity settings. It allows configuring specific IP addresses for stateful applications like middleware and kubevirt, while enabling fixed IP address ranges for stateless ones. Spiderpool automates the management of exclusive IP pools, ensuring excellent IP reclamation to avoid IP leakage.
Multiple network interface access for Pods

Spiderpool enables scenarios where Pods can have multiple underlay CNI interfaces or a combination of overlay and underlay CNI interfaces. It ensures proper IP addressing for each CNI interface and effectively manages policy routing to maintain consistent data paths, eliminating packet loss concerns.
Enhanced network connectivity

Spiderpool establishes seamless connectivity between Pods and host machines, ensuring smooth functioning of Pod health checks. It enables Pods to access services through kube-proxy or eBPF-based kube-proxy replacement. Additionally, it supports advanced features like IP conflict detection and gateway reachability checks.
eBPF enhancements

The eBPF-based kube-proxy replacement significantly accelerates service access, while socket short-circuiting technology improves local Pod communication efficiency within the same node.
RDMA support

Spiderpool provides RDMA solutions based on RoCE and InfiniBand technologies.
Dual-stack network support

Spiderpool supports IPv4-only, IPv6-only, and dual-stack environments.
Metrics

Which scenarios can Spiderpool be applied in?

Spiderpool, powered by underlay CNI, offers unparalleled network performance compared to overlay CNI solutions, as evidenced in I/O Performance. It can be effectively applied in various scenarios, including:

Provide a unified underlay CNI solution for bare metal, virtual machine, and public cloud environments.
Traditional host applications.
Network I/O-intensive applications such as middleware, data storage, log observability, and AI training.
Latency-sensitive application.

Spiderpool Architecture

Spiderpool features a well-designed and comprehensive architecture that caters to various application scenarios, including:

Pods have multiple underlay CNI network interfaces to establish connections with underlay networks.
Pods have an underlay CNI and multiple underlay CNI network interfaces, enabling dual-network connectivity.
Underlay CNIs run in public cloud environments or virtual machines.
Leverage RDMA for efficient network transmission.

For more detailed information, please refer to Spiderpool Architecture.

Major Features

Features	macvlan	ipvlan	SR-IOV
Service by kubeproxy	Beta	Beta	Beta
Service by kubeproxy replacement	Alpha	Alpha	Alpha
Network policy	In-plan	In-plan	In-plan
Bandwidth	In-plan	In-plan	In-plan
RDMA	Alpha	Alpha	Alpha
IPAM	Beta	Beta	Beta
Egress policy	Alpha	Alpha	Alpha
Multiple NIC and routing coordination	Beta	Beta	Beta
Applied scenarios	Bare metal	Bare metal and VM	Bare metal

For detailed information about all the planned features, please refer to the roadmap.

Quick start

Refer to Quick start to explore Spiderpool quickly.

Blogs

Refer to Blogs

Governance

The project is governed by a group of Maintainers and Committers. How they are selected and govern is outlined in our governance document.

Adopters

A list of adopters who are deploying Spiderpool in production, and of their use cases, can be found in file.

Contribution

Refer to Contribution to join us for developing Spiderppol.

Contact Us

If you have any questions, please feel free to reach out to us through the following channels:

Slack: join the #Spiderpool channel on CNCF Slack by requesting an invitation from CNCF Slack. Once you have access to CNCF Slack, you can join the Spiderpool channel.
Email: refer to the MAINTAINERS.md to find the email addresses of all maintainers. Feel free to contact them via email to report any issues or ask questions.
WeChat group: scan the QR code below to join the Spiderpool technical discussion group and engage in further conversations with us.

License

Spiderpool is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.

Spiderpool enriches the CNCF Cloud Native Landscape.

yylt/spiderpool