The 400ZR Implementation Agreement was released in March 2020. It standardized the transport of 400 Gigabit Ethernet (GbE) traffic over a coherent DWDM wavelength in a pluggable QSFP transceiver, and is intended for use in routers and switches for metro DCI applications. This was soon followed by the Open ZR+ MSA, which extends the optical reach and supports additional line rates between 100G and 400G. These new pluggable modules enable DWDM directly on the router, avoiding the need for a separate transponder.
Meanwhile, transponder options are also increasing in numbers. The OpenROADM ZR+ pluggable module for transponders provides OTN encapsulation, Flex-Grid flexibility, and higher launch power in a CFP2-DCO pluggable form factor. For vendor-specific discrete transponder implementations, line rates continue to increase beyond 400G. Transponder speeds up to 1.2T, with the ability to transport 3 x 400 GbE links on a single wavelength, are already on the horizon.
Comparing 400G Transport Options
With 100G transponders having attained mainstream status, 400G is the next major upgrade cycle for optical transport. How does the DWDM-on-a-router deployment model compare with the transponder-based architecture? This blog examines the pros and cons of each and identifies scenarios where one or the other may be preferable.
[2] Cost Model Comparison of ZR/ZR+ Modules Against Traditional WDM Transponders for 400G IP/WDM Core Networks, presented at ECOC (European Conference on Optical Communications) 2020
For a short point-to-point link between two routers over dark fiber, 400ZR is clearly the simplest and least expensive option. So why does 400ZR become the most expensive option when considering the total network cost? Cost modelling for a nation-wide network indicates that Day 1 deployment costs are 2-4X higher – and that this ratio increases in subsequent years as the network grows.
Let’s take a closer look at this counterintuitive result.
Multi-hop Router Traffic Flows
400ZR incorporates lower cost optics and is designed for short point-to-point links. This means it is well suited for point-to-point and hub-and-spoke router topologies. While it is claimed to reach 80-120km theoretically, practically speaking, a 40-60km reach is a good rule-of-thumb to use to ensure sufficient optical link budget. Hence direct connections between switches and routers will be distance-limited to fiber spans within the metro less than 60km.
Traffic patterns are changing as more investment is poured into the edge of the network. Locating application servers closer to users and devices keeps latency low, which is a requirement for time-sensitive 5G use cases requiring URLLC (Ultra-Reliable Low-Latency Communications) such as connected cars, and industry automation. Hence, traffic flows are expected to evolve from a centralized hub-and-spoke model to a more distributed mesh model.
Mesh traffic is the weakness of 400ZR. Routers must terminate every point-to-point link (see figure 1 below) so if the source and destination router are not directly connected, packets must pass through multiple router hops. This requires terminating the 400ZR wavelength at every intermediate routers and performing a forwarding decision through the router’s switch fabric. This consumes expensive router ports, processing power, and adds to the end-to-end packet latency. Transiting two routers will require four router ports and four 400ZR modules. The router cost quickly adds up and then there is also the cost of the external amplifiers and Mux/Demux DWDM modules. The cost of the circuit becomes a linear function of the number of sites transited.
Figure 1: Router traffic from site A to D is a single hop using a ROADM, but requires multiple hops when using 400ZR interfaces on the route
By comparison, with a transponder/ROADM architecture, intermediate sites can be bypassed by provisioning a passthrough wavelength connection directly through the ROADM. So long as the wavelength does not require regeneration, the deployment cost of the circuit is completely independent of the number of sites it is required to traverse.
Creating Mesh Connectivity for the Cloud Edge
Creating a mesh network is possible with a 400ZR/FOADM architecture. This entails making manual fiber cross-connects for transit wavelengths at the intermediate nodes. Signal regeneration is still required but this need not be done by consuming router ports. Instead, 400ZR modules connected back-to-back can perform the optical-electrical-optical (OEO) conversion function. This helps alleviate the impact on router costs, but the price to pay is a hardwired, inflexible network that is not remotely configurable and requires a site visit (potentially to multiple transit sites) for circuit moves, adds and changes.
Mixed Network Management of the Optical Layer
Traditionally the routing layer has been managed separately from the optical layer. The introduction of 400ZR/OpenZR+ modules into the optical network requires the routers’ network management system to monitor and manage those links. Optical layer operational processes must be adapted and extended to the router NMS, which will likely require API development and integration between the EMS, NMS, and/or the optical domain SDN controller (see figure 2 below). For smaller networks with a limited number of such links, using the two management platforms separately without integration could be a viable solution. However, this might represent a challenge for larger carriers with parallel switching and optical transport organizations, processes and operational models.
The overlap of router/optical domains affects a service provider’s demarcation point as well. Traditionally the demarcation point has been at the interface into the transponder. Now that the transponder is a module inside a customer’s router, the traditional handoff point no longer exists and must be redefined.
Summary
Optical network planners now have additional tools in their toolbox for metropolitan networks: 400ZR for short fiber spans and Open ZR+ for longer spans. Where they should be deployed depends on router traffic patterns as well as the distance between sites. If the traffic flows over a short distance, centralized hub-and-spoke network topology, then 400ZR router-based architectures is the lowest cost option. If traffic is distributed and a meshed router topology is in order, costs can escalate quickly and the traditional transponder/ROADM architecture is a better fit. Finally, 400ZR pluggables are designed for short-haul applications so transponder-based options are needed for longer fiber spans.
Interested in 400G? The Fujitsu 1FINITY™ T700 transponder provides a 400 GbE interface to the router and 400G DWDM for transport across the optical network. A range of spectral widths are supported for optimizing optical reach, and the 1FINITY™ T700 also features an inverse multiplexing option that splits 400 GbE client traffic into two wavelengths of 200G, both which can then be transported by fixed grid networks before being recombined at the other end of the circuit.
Interested in 400G?
The Fujitsu 1FINITY™ T700 transponder provides a 400 GbE interface to the router and 400G DWDM for transport across the optical network. A range of spectral widths are supported for optimizing optical reach, and the 1FINITY™ T700 also features an inverse multiplexing option that splits 400 GbE client traffic into two wavelengths of 200G, both of which can then be transported by fixed grid networks before being recombined at the other end of the circuit.
