The technology standards being developed in the IEEE, 3GPP and IETF aim to deliver solutions that are complementary and can be combined by network operators to deliver end-to-end converged networks supporting both traditional and deterministic services.  The ability to ensure the delivery of traffic for each flow’s different Quality of Service requirement is critical for all deterministic networking technologies.   The remainder of this article highlights the deterministic networking related technologies and activities of these three standards development bodies.

IEEE 802.1 Time-Sensitive Networking (TSN)

Standardization

The IEEE 802.1 Working Group (WG) focuses on standards and recommended practices in the following areas: (i) 802 LAN/MAN architecture, (ii) Internetworking among 802 LANs, MANs and other wide area networks, (iii) 802 Security, (iv) 802 overall network management, and protocol layers above the MAC & LLC layers. The Time-Sensitive Networking (TSN) Task Group (TG) within the IEEE 802.1 WG deals with deterministic services for IEEE 802 networks.

The TSN TG specifies the tools of the TSN toolbox, as well as the use of the tools for various application areas. The TSN TG is chartered to provide deterministic services for IEEE 802 networks with:

• Guaranteed packet transport
• Low packet loss
• Bounded low latency
• Low packet delay variation

The TSN TG evolved from the Audio Video Bridging (AVB) TG and it includes the former Interworking TG. There are three groups of TSN Standards & Projects: (1) Base technology (e.g., 802.1CB, 802.1Qbv, etc.); (2) Configuration (e.g., 802.1Qcp, 802.1Qcc, etc.); and (3) Profiles (e.g., 802.1BA, 802.1CM, IEC/IEEE 60802, etc.).

802.1 TSN components are shown in Figure 2.

TSN Profiles have been defined for Audio-Video Bridging (AVB) networks (802.1BA) and for Fronthaul (for cellular networks) (802.1CM amended by 802.1CMde) to date. There is ongoing work to define TSN profiles for:

• IEC/IEEE 60802 TSN Profile for Industrial Automation
• P802.1DG TSN Profile for Automotive In-Vehicle Ethernet Communications
• P802.1DF TSN Profile for Service Provider Networks
• P802.1DP TSN for Aerospace Onboard Ethernet Communications

TSN Functionalities

IEEE 802.1 defines a TSN Stream as a unidirectional flow of data from a Talker to one or more Listeners. During the forwarding process at a bridge, QoS functions are applied to the frames of a TSN Stream, –e.g.: (i) Filtering and Policing, (ii) Shaping and (iii) Queuing.

A wide range of TSN functions are being defined by the IEEE 802.1 TSN TG. Only some of the functions are discussed here at a high level. There are other documents with further details. The primary media for TSN is IEEE 802.3 Ethernet. Work is also ongoing to involve wireless, e.g., the 5G – TSN integration work in 3GPP.

Scheduled Traffic (802.1Qbv) reduces latency variation for frames with known timing. This is achieved via time-based control and programming of the bridge queues. Each queue is equipped with time-gates (time-gated queues) and a queue can be served only when the gate is open. Gate open/closed states are changed according to a periodically repeated time schedule. This function requires end-to-end time synchronization.

Frame Preemption (802.3br and 802.1Qbu) enables so-called express frames (i.e., critical traffic) to suspend the transmission of preemptable frames (i.e., non-critical traffic). As a result, delay variation for express traffic is decreased and it also increases the available bandwidth for preemptable traffic over subscription-based schemes. Frame preemption is a link local per hop function, i.e., it is not multi-hop like IP fragmentation.

Per-Stream Filtering and Policing (802.1Qci) provides protection against traffic streams that violate their bandwidth allocation, or malfunction, or participate in attacks, etc. Filtering and policing decisions can be made on per-stream, per-priority, etc. basis.

Asynchronous Traffic Shaping (ATS) (802.1Qcr) provides zero congestion loss without time synchronization. ATS is similar to per-flow IETF IntServ shaping, except that: (i) all streams from one input port to the same output port share the same queue and (ii) a shaper state machine is for a set of streams, and ensures the correct shaper is applied to the packet upfront of the queue. The essence of the ATS function is to smoothen traffic patterns by re-shaping at every hop so that urgent traffic is prioritized over less urgent or elastic traffic. Strict priority queueing is used for ATS.

Frame Replication and Elimination for Reliability (FRER) (802.1CB) is designed to avoid frame loss due to equipment failure. It is a per-frame 1+1 (or 1+n) redundancy function. There is no need for failure detection or switchover mechanisms. FRER sends frames on 2 (or more) maximally disjoint paths, then combines the streams and deletes extra frames.

Explicit Trees by IS-IS Path Control & Reservation (802.1Qca, RFC 7813) provides IS-IS control beyond Shortest Path Trees (SPTs) by augmenting IS-IS with non-shortest path or explicit path forwarding. There are no protocol changes, only a couple of new sub-TLVs are defined and it reuses existing ones as much as possible. The concept is a hybrid Software Defined Networking (SDN) approach, where IS-IS provides basic functions, e.g., topology discovery and default paths, and one or more controllers control the Explicit Trees.

Stream Reservation Protocol (SRP) Enhancements and Performance Improvements (802.1Qcc): provides Time-Sensitive Networking (TSN) configuration related attributes. 802.1Qcc describes three models for TSN user and network configuration (fully distributed, centralized network/distributed user and fully centralized model). Each model specification defines the logical flow of user/network configuration information between various entities in the network.

Future Outlook on TSN

TSN standardization is still in progress. The IEC/IEEE 60802 TSN Profile for Industrial Automation is a joint project of IEC SC65C/WG18 and IEEE 802. This joint work will provide a dual logo standard that is both an International Electrotechnical Commission (IEC) and an IEEE standard. The profiles select features, options, configurations, defaults, protocols, and procedures of bridges, end stations, and LANs to build industrial automation networks. Work has been started to provide a conformity assessment specification for IEC/IEEE 60802 in a separate IEC specification.

Open Platform Communications United Architecture (OPC UA) builds upon TSN, DetNet, and 5G. There are multiple OPC UA work items in progress related to TSN. One of them is the Field Level Communications (FLC) Working Group, which heavily builds upon the IEC/IEEE 60802 specification as well as the associated conformity assessment specification. The scope of OPC UA FLC work is on the higher OSI layers and involves conformity assessment for their work.

3GPP support for deterministic transport (URLLC)

Standardization

5G has been designed from the beginning to not only address enhanced mobile broadband services for consumer devices such as smart phones or tablets, but also tailored for future Internet of Things (IoT) communication and connected cyber-physical systems. To this end, two requirement categories have been defined: (i) massive machine-type communication for a large number of connected devices/sensors, and (ii) Ultra-Reliable and Low Latency Communication (URLLC) for connected control systems and critical communication. URLLC capabilities enable 5G to be a suitable candidate for supporting wireless deterministic and time-sensitive communication applications.

Several features have been introduced to 5G in Release 15 that enable the transmission of a message with a one-way latency down to 1ms with reliability of up to 99.999 percent. Further URLLC features have been added to 5G Release 16 to support a one-way latency down to 0.5 ms with a reliability of up to 99.9999 percent. A brief summary of URLLC features is provided here.

URLLC Functionalities

The 5G Radio Access Network (RAN) with its New Radio (NR) radio interface includes several functionalities to achieve low latency for selected data flows. The flexible choice of numerology for Orthogonal Frequency Division Multiplexing (OFDM) makes 5G RAN scalable, so that it can be adapted to a particular use case and deployment. The NR numerology enables shorter slots in a radio subframe, which is beneficial for low latency applications. Furthermore, NR introduces mini-slots when prioritized transmissions can be started without waiting for slot boundaries, which enables fast delivery of data with low latency requirements. As part of enabling priority and faster radio access to URLLC traffic, NR introduced preemption where URLLC data transmission can preempt ongoing non-URLLC transmissions. Additionally, NR applies very fast processing, e.g., for the decoding of received data and transmissions and generation of feedback acknowledgements completing data transmission.

Resource management for ultra-reliability and low latency has several components, e.g., spectrum resources are essential. NR radio resource management allows the configuration of deterministic radio resource allocations for URLLC traffic, thereby avoiding the delay to request for transmission resources as used for dynamic scheduling. Semi-persistent scheduling is used in the downlink direction and configured grants are applied in the uplink direction.

5G defines extra robust transmission modes for increased reliability applying to both data and control radio channels. Reliability is further improved by various techniques, e.g., multi-antenna transmission, the use of multiple carriers, and packet duplication over independent radio links.

Time synchronization has been embedded in cellular radio systems as an essential part of its operation. Devices are time aligned by a base station to compensate for their different propagation delays. The radio network components themselves are also time synchronized, for instance, via the precision time protocol telecom profile. This is a good basis to provide synchronization for time critical applications.

Besides the 5G RAN features, the 5G system also provides solutions in the Core Network (CN) for Ethernet networking and URLLC. The 5G CN supports native Ethernet Protocol Data Unit (PDU) sessions. For user plane redundancy on the 5G system level, 5G supports the establishment of redundant user plane paths through the 5G system including RAN, CN and the transport network. Such redundant paths are possible by using a single User Equipment (UE) with the RAN dual connectivity feature in the end device or by using multiple UEs in the end device. Furthermore, 5G can provide virtual networks (5G-VN) and LAN groups, which can be used for allocating resources to the members of a particular group.

All these new URLLC functionalities of 5G provide a well-designed and solid ground for using 5G in deterministic scenarios even as a stand-alone solution or as a segment of the deterministic network.

Figure 3 shows a 5G system architecture where the 5G system is considered as a TSN bridge. In particular, the fully centralized model of IEEE 802.1Qcc is supported in release 16 of the 3GPP specification. A new translation functionality (noted as DS-TT and NW-TT) is specified which holds and forwards user plane packets for de-jittering where the 5G system (5GS) is integrated as a bridge with the TSN network. The 5GS includes TSN Translator (TT) functionality for adaptation of the 5GS to the TSN domain both for user plane and control plane, making the 5GS specific procedures hidden from the deterministic transport network.

Future Outlook on URLLC

5G URLLC capabilities provide a good match to TSN and DetNet features. Thus, the three technologies can be integrated to provide deterministic connectivity end-to-end, i.e., between input/output (I/O) devices with their controller, for example, residing in an edge cloud providing network management. The integration already includes data plane support for both the necessary base bridging/routing features and the TSN/DetNet add-ons, however the control and management plane needs further standardization work.

IETF Deterministic Networking (DetNet)

Standardization

The IETF DetNet WG belongs to the Routing (RTG) Area, which focuses on routing and signaling protocols. The DetNet Working Group focuses on deterministic data paths that operate over Layer 2 bridged and Layer 3 routed segments, where such paths can provide bounds on latency, loss, and packet delay variation (jitter), and high reliability. The scope of the DetNet WG includes: (i) overall architecture, (ii) data plane specification, (iii) data flow information model and (iv) related YANG models.

There is close cooperation between the IETF DetNet WG and the IEEE 802.1 TSN TG.

DetNet operates at the IP/MPLS layer and its initial scope is for networks that are under a single administrative control or within a closed group of administrative control.

Solution documents specify procedures and behaviors required of nodes supporting DetNet with a specification focus on interoperable implementations. There are two data planes being defined:

• IP: uses IP and Transport Protocol header information to support DetNet [RFC 8939]
• MPLS: uses Labels to support DetNet [RFC 8964]

Note that IETF DetNet documents are very focused; therefore, multiple documents explain the details of the DetNet data plane, an overview is provided by the data plane framework [RFC 8938].

Forwarding characteristics are accomplished by dedicating network resources such as link bandwidth and buffer space to DetNet flows and/or classes of DetNet flows, and by protecting packets (e.g., by replicating them along multiple maximally disjoint paths). Unused reserved resources are available to non-DetNet flows as long as all guarantees are fulfilled.

The following forwarding parameters from source to destination are defined:

• Minimum and maximum end-to-end latency: timely delivery, and bounded jitter (packet delay variation) derived from these constraints
• Packet loss ratio: loss during transport, extreme low loss values may apply
• Upper bound on out-of-order packet delivery: some DetNet applications are unable to tolerate any out-of-order delivery

Note: DetNet has the distinction (and TSN is similar) that it is concerned solely with worst-case values for the end-to-end latency, delay variation, and misordering. Average, mean, or typical values are not considered significant, because they do not affect the ability of a real-time system to perform its tasks. A priority-based queuing scheme may give better average latency to a data flow than DetNet; however, it may not be a suitable option for deterministic services because of its worst-case latency.

DetNet Functionalities

DetNet characteristics are ensured using the following building blocks:

• Congestion protection
• Service protection
• Explicit routes

Congestion protection means allocating resources along the path of a DetNet flow, e.g., buffer space or link bandwidth. It addresses two of the DetNet QoS requirements: latency and packet loss.

Congestion protection eliminates congestion related loss by utilizing properly designed queuing, so no packets are dropped due to a lack of buffer storage. It serves also as a tool to reduce delay variation, which enables, for example, the convergence of sensitive non-IP networks onto a common IP network infrastructure. Congestion protection, provided via feedback mechanisms such as congestion detection and notification, was explicitly excluded from consideration in DetNet. Many functions of congestion protection require time synchronization of DetNet nodes; however, time synchronization is out-of-scope in DetNet discussions as it does not impact interoperability. Time synchronization is expected to be provided by appropriate solutions, e.g., by lower layers.

Service protection addresses random media errors and equipment failures, e.g., packet replication and elimination (protects against failures), packet encoding (protects against media errors), re-ordering (ensures in-order delivery). Service protection can be ensured by (i) Packet Replication, Elimination, and Ordering Functions (PREOF) or (ii) packet encoding techniques. PREOF defined by DetNet are: (i)  packet replication function (PRF: sends copies of the same packets with sequencing information over multiple paths), (ii) packet elimination function (PEF: discards duplicates based on sequencing information and history of received packets), and (iii) packet ordering function (POF: restore original packet order as out-of-order delivery impacts the amount of buffering at the destination to properly process received data). Packet replication and elimination does not react to and correct for failures. These functions are entirely passive. Packet encoding (also called Network Coding) encodes the information into multiple transmission units, sends them using multiple paths, and combines the units on the other end.

Explicit routes (a.k.a. nailed down paths) can be utilized to address the impact of the convergence of routing or bridging protocols (i.e., temporary interruptions).

DetNet functionality is implemented in two adjacent sub-layers in the protocol stack:

• DetNet service sub-layer: provides DetNet service (e.g., service protection) to higher layers in the protocol stack and applications
• DetNet forwarding sub-layer: supports DetNet service in the underlying network (e.g., by providing explicit routes and congestion protection) to DetNet flows

The Layer-3 equivalent of a TSN Stream is called as DetNet flow. A DetNet flow is a sequence of packets that conform uniquely to a flow identifier, and to which the DetNet service is to be provided. It includes any DetNet headers added to support the DetNet service and forwarding sub-layers.

DetNet related mechanisms require two attributes:

• Flow-ID: to identify the flow the packet belongs to
• Sequence number: to recognize duplicate packets and re-order packets

Future Outlook on DetNet

DetNet standardization is still in progress. There is a close cooperation between IETF DetNet and IEEE TSN in order to ensure interoperability and to simplify implementation of deterministic functions working for both Layer-2 and Layer-3. For example, IEEE P802.1CBdb (FRER Extended Stream Identification Functions) focuses on extending the fields used for the Stream identification function to arbitrary mask-and-match, which is essential for combined TSN and DetNet network scenarios. Control and Management plane related work is the next focus of the DetNet WG.

Conclusions

Networks for applications requiring deterministic transport are increasingly demanding to gain from the use of the lower cost and higher capacities of asynchronous packet-based technologies.  Packet-based networks in the past were aimed to carry all but the very delay-sensitive/real-time application traffic. Over time these networks have leveraged speed upscaling, instead of rigid engineering, for  delay, delay variation, and loss constraints.  This focus ensured their success for a large range of applications at a global scale. Utilizing deterministic technology developments, packet-based networks are evolving to incorporate support for demanding applications as well.

TSN, DetNet, and 5G URLLC can meet the networking requirements of deterministic applications, providing ultra-reliable and low-latency connectivity via a converged network. TSN and DetNet (for wireline) and 5G (for wireless) technologies are a perfect match to complement each other in deterministic transport networks. A level of overall integration of these technologies is required to provide end-to-end connectivity meeting the deterministic requirements. For example, time synchronization over both a wireless 5G domain and a wired TSN/DetNet domain is a requirement because a common reference time is essential for deterministic end points, irrespective of the networking technology interconnecting them. Providing bounded low latency may also require integration between TSN, DetNet, and 5G depending on which deterministic tools are used in a deployment. End-to-end ultra-reliability will require alignment of the characteristics for the necessary disjoint forwarding paths. The first steps supporting overall integration is being accomplished using an SDN-based approach. The base TSN, DetNet and URLLC technologies are ready and their combined deployment is imminent.