Fabric-TNA

fabric-tna is a P4 program that realizes the data plane of SD-Fabric, an SDN-enabled programmable network fabric tailored for 5G-connected edge clouds, with a focus on enterprise and Industry 4.0 use cases.

fabric-tna is based on the Tofino Native Architecture (TNA), hence it can be used to program any switch based on the Intel Tofino ASIC. The P4 program is specifically designed to run on Stratum-enabled switches controlled by the ONOS SDN controller.

fabric-tna.p4 is not to be confused with fabric.p4, an older P4 program hosted in the ONOS repository originally designed to support Trellis. fabric-tna.p4 follows a similar design to fabric.p4, but has evolved significantly to provide more advanced capabilities for 4G/5G mobile user plane function (UPF), In-band Network Telemetry (INT), slicing, and QoS.

For up-to-date documentation on the architecture, capabilities, and instructions to deploy and use SD-Fabric, please refer to the official documentation:

https://docs.sd-fabric.org/

Content

P4 Program

The directory p4src/tna contains the P4 program for the TNA architecture. p4src/v1model contains an equivalent version of the same program based on the V1Model architecture, to be used for development and testing together with the BMv2 software switch.

ONOS Pipeconf

src/ contains the Java implementation and tests for the ONOS pipeconf, which is essentially a set of drivers enabling ONOS to control different aspects of the switch pipeline defined by the P4 program.

PTF Tests

ptf/ contains an extensive suite of test cases for the P4 program, based on the Packet Test Framework (PTF). These tests are used to validate the forwarding behavior of the switch. We provide scripts and instructions to run tests on multiple targets:

Tofino-Model (ASIC emulator)
HW switch
BMv2 software switch

Requirements

To build and test this project you will need the following software:

Barefoot SDE (a.k.a. Intel P4 Studio) = 9.7.0
ONOS >= 2.5.8
Docker (to run the build scripts without worrying about dependencies)
cURL (to interact with the ONOS REST APIs)

Intel P4 Studio (SDE) Docker Image

To facilitate managing different versions of the Intel P4 Studio (SDE), the build scripts expect the Barefoot SDE to be provided as a Docker image. We do not provide such image, but one can be easily generated by:

executing the regular SDE install instructions inside a Dockerfile
making sure executables such as bf-p4c and tofino-model are on $PATH

Once you obtain such Docker image, set the following environment variables:

export SDE_VERSION=9.7.0
export SDE_DOCKER_IMG=my-docker-repo/bf-sde:9.7.0

Quick steps

To compile the P4 program with a given <profile>:

make <profile>

The available profiles are:

Profile name	Description
`fabric`	Basic L2/L3 forwarding capabilities
`fabric-upf`	With 4G/5G mobile user plane support
`fabric-int`	With INT support
`fabric-upf-int`	WITH UPF and INT support

To run PTF tests on Stratum using Tofino Model:

./ptf/run/tm/run <profile>

For more information about running PTF tests, check ptf/README.md.

If you want to test the pipeline manually with an interactive P4Runtime shell:

./ptf/run/tm/p4rt-shell <profile>

This command will start Tofino Model with Stratum and a P4Runtime Shell attached to it. For more information about using P4Runtime Shell, check p4lang/p4runtime-shell.

Also, check p4rt_shell_snippets.md for some examples of how to program tables in fabric-tna.

To build the ONOS pipeconf .oar package which includes the compiled P4 artifacts for the previously built profile(s):

make pipeconf

To learn more about pipeconfs, keep reading.

Detailed steps to build the fabric-tna pipeconf

ONOS uses "pipeconfs" to deploy and manage a given P4 program on a device. Pipeconfs include mainly two things:

the P4 compiled artifacts (e.g., tofino.bin, context.json, etc.) to deploy on devices.
Java classes implementing ONOS driver behaviors to control capabilities of the particular P4 program.

Pipeconfs are distributed as ONOS applications, hence using the .oar packaging. The following steps provide instructions on how to generate an oar package that includes one or more profiles.

To learn more about pipeconfs and how ONOS supports P4-programmable devices: https://github.com/opennetworkinglab/ngsdn-tutorial

To build fabric-tna.p4 using the Barefoot compiler and to create the pipeconf .oar package in one command:

make build [PROFILES=...]

This command will build the fabric-tna.p4 profiles specified in the PROFILES argument.

To build all profiles, leave PROFILES unset.

To build a subset of the available profiles separate them with spaces: PROFILES="fabric fabric-int"

The P4 compiler outputs to include in the .oar package will be placed under src/main/resources/p4c-out.

When done, the pipeconf .oar package can be found in target/fabric-tna-<VERSION>.oar

Steps to use the fabric-tna pipeconf with ONOS

1 - Get and run ONOS

You can either build from sources or run one the released versions: https://wiki.onosproject.org/display/ONOS/Downloads

Pre-built ONOS Docker images are available here: https://hub.docker.com/r/onosproject/onos/tags

For more information on how to get and run ONOS: https://wiki.onosproject.org/display/ONOS/Guides

2 - Start Stratum on your switch

To run Stratum on a Tofino-based switch, please follow instructions in this document: https://github.com/stratum/stratum/blob/main/stratum/hal/bin/barefoot/README.run.md

Note that fabric-tna uses the @p4runtime_translation annotation for certain P4 tables, you must enable P4Runtime translation with the following flags when starting Stratum:

-experimental_enable_p4runtime_translation
-incompatible_enable_bfrt_legacy_bytestring_responses

3 - Install pipeconf app in ONOS

To install the pipeconf app built in the previous step, assuming ONOS is running on the local machine:

make pipeconf-install ONOS_HOST=localhost

Use the ONOS_HOST argument to specify the hostname/IP address of the machine where ONOS is running.

This command is a wrapper to a curl command that uses the ONOS REST API to upload and activate the .oar package previously built.

You should see the ONOS log updating with messages notifying the registration of new pipeconfs in the system, depending on the profiles compiled before, and the Barefoot SDE version:

New pipeconf registered: org.stratumproject.fabric.mavericks_sde_9_7_0 (fingerprint=...)
New pipeconf registered: org.stratumproject.fabric.montara_sde_9_7_0 (fingerprint=...)
...

NOTE: it might take up to one minute for the pipeconfs to be registered.

To check all pipeconfs registered in the system, use the ONOS CLI:

onos> pipeconfs

4 - Connect ONOS to a Stratum switch

Activate the Barefoot drivers in ONOS:

onos> app activate org.onosproject.drivers.barefoot

This command will register a new driver named stratum-tofino. As the name suggests, this driver allows ONOS to control Tofino switches running Stratum.

For ONOS to be able to discover your switch, you need to push a JSON file, usually referred to as the "netcfg" file. We provide an example of such tofino-netcfg.json file in this repository. Make sure to modify the following values:

managementAddress is expected to contain a valid URI with host and port of the Stratum gRPC server running on the switch;
The device_id URI query parameter is the P4Runtime-internal device_id, also known as the Stratum "Node ID". Usually, you can leave this value set to 1;
Use the pipeconf field to specify which pipeconf/fabric profile to deploy on the switch.

Push the tofino-netcfg.json to ONOS using the command:

make netcfg ONOS_HOST=localhost

Like before, this command is a wrapper to a curl command that uses the ONOS REST API to push the tofino-netcfg.json file.

Check the ONOS log for potential errors.

Using SD-Fabric with Stratum+Tofino switches

Check the official SD-Fabric documentation here: https://docs.sd-fabric.org

In the "Network Configuration" section: https://docs.sd-fabric.org/master/configuration/network.html

Support

To report issues when compiling fabric-tna.p4 for Tofino (i.e., P4 compiler errors), please contact Intel support.

To report any other kind of problem, feel free to open a GitHub Issue or reach out to the project maintainers on the ONF Community Slack.

KaiLv16/fabric-tna