This is a repository to explore some of the dimensions of Time Sensitive Networking (TSN) on Linux machines, interconnected with a TSN-capable switch.
This repository is focused on configuring time synchronization, VLAN configuration for traffic prioritization, and LAN stress tests to verify the two latter points. The READMEs will also include information on how we configure our TSN switch. This codebase is developed as an exercise for understanding of TSN and how it can be used with the ARENA project at Carnegie Mellon to introduce more deterministic network to distributed runtime environments for applications like industrial automation.
We have small network of three endpoint devices running linux, connected to a TSN-capable Ethernet Switch.
Each endpoint is an Intel NUC (NUC11TNHv5) running stock Ubuntu 20.04 LTS. They each have an Intel i225-LM Ethernet NICs (Network Interface Card) for hardware timestamping, queue management for QoS, and a few other TSN capabilities.
The switch is a Belden/Hirschmann BRS40-8TX managed layer 2 switch for industrial applications (which we'll refer to as the 'switch' or 'TSN switch'). Your mileage may vary for other brands of switches as they do not necessarily implement all of the same standards. During the development of this codebase, it was running HiOS 8.7.02-S. To date, we have almost solely used the GUI for modifying switch configurations, although it appears to be missing a few minor features related to TSN's TDMA-style traffic shaping that the CLI does offer.
Please find a few additional notes on the hardware capabilities and constraints in <TODO!>.
- TSN Switch Setup
- PTP Configuration
- VLAN Configuration
- VLAN Latency Testing
- Errata and Other Mentionables
- Setup endpoint devices with Ubuntu 20.04 LTS or newer
- Install switch software (HiOS 8.7 or newer)
- Connect endpoints to switch(s) with ethernet cables
- Connect switch(s) to LAN that your host machine can access..
- Configure and activate gPTP/802.1AS
- Create VLANs
- Setup Gate Control Lists in Switch
- Create and run applications that run on top of configured VLAN(s)
- NB: If using TCP, both sides of the connection should use the same VLAN/priority, e.g., the MQTT broker should bind itself to an IP on the same VLAN as the clients.
As a whole, TSN is composed of 3 elements: time synchronization, traffic shaping / Quality of Service, and stream reservation / enforcement.
These are generally encompassed in the 802.1Q standards from IEEE; TSN is not a standard in-and-of-itself, but an umbrella term for a collection of standards. TSN was borne from Audio-Video Bridging as the standards evolved and industrial domains realized their value in applications like factory automation. Note that time sychronization in TSN is effectively just the Precision Time Protocol, which is governed under IEEE 1588 (and a sister specification for a simplified PTP configuration called generic PTP or 802.1AS). To reiterate, TSN itself is not a standard; it refers to the overall concept of layer 2 networking with time-determinism.
You may find a fairly technical slide deck from Onera that conscisely describes these standards and their purposes here. A white paper for Belden is also a useful read on the motivations and components of TSN, and the AVNU Alliance similarly has helpful resources (albeit geared towards TSN product design) on concepts and best practices.
TSN is exists almost exclusively at the Data Link Layer (more specifcally, Ethernet). This is in large part due to the maturity of NICs that can add or manipulate tags in the Ethernet frame's header, which is a necessity for sub-microsecond time synchronization and data stream reservation.
Yes. In our experience (as of 2021), time synchronization and QoS management is mature in both Linux and the TSN switch. Stream reservation (argueably the most interesting element of TSN for networking with deterministic latency), has little evidence of a working, open-source implementation.
Existing implementations like OpenAVNU and iotg_tsn_ref_sw have very specific hardware and software requirements, including custom linux kernels and out-of-date network drivers. A somewhat dated set of documents for TSN for Linux covers VLANs, traffic shaping, and time synchronization, but does not include any note of stream reservation.
Further, the i225 NIC and TSN switch do not indicate they support this technology (standards 802.1Qat and 802.1Qcc for reservation and centralized coordination, respectively) as of December 2021, although future software/firmware updates might introduce this.
This repository is instead focused on time synchronization, VLAN configuration for traffic prioritization, and LAN stress tests to verify the two latter points.
This codebase is not intended for production level systems. It was developed as an exploration of TSN capabilities both in standard Linux machines and a designated switch.
There is no guarantee this will work on an arbitrary Linux distribution or with an arbitrary TSN-compatible switch.
MIT