#### by Olga Vassilieva

For transmission over optical fiber, a digital signal must first be translated into an analog signal. This is known as modulation. 10G wavelengths use simple modulation techniques such as NRZ (non-return to zero) and OOK (on-off keying), but coherent systems utilize QAM (quadrature amplitude modulation), where the digital signal is encoded in both the phase and amplitude of the analog signal. QAM can squeeze in more bits/hertz, so it is more spectrally efficient and used for coherent 100G+ wavelengths. Today I take a deep dive into QAM, and share some advanced QAM techniques – geometric and probabilistic shaping – that can increase capacity, extend reach, or even both.

## All About QAM

The spectral efficiency or the number of transmitted bits per symbol is defined by the modulation order *m* of the MQAM systems, where *M* is the number of the constellation points on the constellation diagram and *m* is related to *M* as *m=log _{2}M.* For example, QPSK (also known as 4QAM) transmits 2 bits per symbol and has total 4 points on the constellation diagram. Higher spectral efficiency can be achieved by increasing modulation order

*m*such that 16QAM transmits 4 bits per symbol, 64QAM transmits 6 bits per symbol as shown on Fig. 1.

However, every time we increase modulation order *m* by one bit the required SNR increases by approximately 3 dB. This is because the larger number of constellation points must be located closer to each other to satisfy the same average power constraint. This additional 3 dB raises the threshold and means that the transmission reach will reduce by half. The trade-off between the spectral efficiency and the transmission distance is schematically illustrated on Fig. 2 solid blue line.

## Constellation Shaping

Constellation Shaping is a technique that can reduce the required SNR by up to 1.5 dB by making the constellation more Gaussian-like and, thus, operating close to Shannon Capacity. This reduced SNR is called shaping gain and is beneficial for extending reach. There are two types of constellation shaping techniques: geometric shaping and probabilistic shaping. Both can provide SNR gains up to 1.5 dB. The geometric shaping modifies the location of the constellation points to approach Gaussian-like distribution, while keeping the constellation points with the same probabilities. The example of geometric shaping is shown on Fig. 3 (left), where inner constellation points are located more densely on the constellation diagram compared to the outer constellation points. Probabilistic shaping modifies the probability of occurrence of the constellation points via Maxwell–Boltzmann (MB) distribution, while keeping the location of constellation points the same as QAM format (Fig. 3 (right)).

These constellations are shaped to maximize tolerance to the linear noise. However, optical signals can be affected by Kerr nonlinearities during transmission. Thus, the amount of the shaping gain, obtained from the linear noise contribution, can be reduced by the nonlinear noise. Therefore, shaping for both linear and non-linear channels is highly desirable.

## Geometric shaping (GS)

With GS we can design the constellation specifically for improved tolerance to both linear and non-linear noise, which can enhance reach significantly. GS doesn’t allow rate adaptivity because of the fixed constellation design. Thus, it can be an ideal candidate for niche applications where rate adaptivity is not required (for example, long haul applications).

One of the examples of such constellations is 4D-2A8PSK, which is 4-dimensional (4D) constant modulus modulation format ^{[1]}. The 4D formats optimize 4 dimensions of the optical field: in-phase and quadrature (2D) in two orthogonal polarizations (2D). In 4D-2A8PSK, each X- and Y-polarization has eight phases (8PSK modulation) and two levels of amplitudes (2A). Two polarizations have complementary amplitudes, such that if smaller amplitude is chosen in X-polarization then larger amplitude is selected in Y-polarization. In this case, the power of combined X and Y polarizations is constant in each time slot. This constant power constraint is known to suppress fiber Kerr nonlinear effects such as self-phase modulation (SPM) and cross-phase modulation (XPM).

4D-2A8PSK has been shown to be superior to DP-8QAM format, while carrying the same number of bits per symbol per polarization. The 4D-2A8PSK has simpler configuration and lower complexity compared to probabilistic shaping because it doesn’t require distribution matching and, thus, doesn’t need a new DSP chip. This modulation format can be an ideal candidate for long haul systems.

## Probabilistic shaping (PS)

PS modifies the probability of occurrence of constellation points in square QAM constellation rather than their locations to approximate Gaussian distribution. The distribution matcher changes the occurrence of symbols such that the low-energy inner constellation symbols are transmitted more frequently than high-energy outer constellation symbols. These probabilities can be optimized through a single parameter called shaping parameter.

The main advantage of PS is the fine granularity in both capacity and reach. This makes PS a good candidate for universal transponder technology, where we can design a certain modulation order *m* QAM transponder (for example 64QAM) and apply probabilistic shaping to change the number of transmitted bits per symbol all the way from m = 6 to m = 2 (which corresponds to QPSK) with fine granularity, as shown in Figure 2 with dashed green line. However, PS requires more advanced DSP chip with distribution matcher and de-matcher.

One disadvantage of PS is its lower tolerance to fiber Kerr non-linearity compared to uniform QAM due to modified constellation. This impairment can reduce shaping gain. Contrary to GS, where constellation itself can be designed for improved tolerance to fiber nonlinearity, a distribution matcher must be designed for improved tolerance to both linear and non-linear noise.This is not a simple task. There are limited reports with the design of non-linearity tolerant distribution matcher and more research is needed in this area ^{[2-3]}.

## Summary

Constellation shaping is an attractive technique to improve SNR of MQAM systems. The benefit of constellation shaping is two-fold: first, it can operate close to Shannon capacity and reduce SNR by up to 1.5 dB, which can result in extended reach. The geometric shaping can optimize reach by designing nonlinearity tolerant constellation for a specific target application and is an ideal candidate for a niche application. The second benefit is fine granularity in capacity and reach, which is useful for better network utilization. Probabilistic shaping can optimize both *reach* and *capacity* due to rate adaptivity. Thus, it can cover a wider range of applications compared to geometric shaping.

The Fujitsu 1FINITY™ T700 transponder for 100/400 GbE services supports 4D-2A8PSK geometric shaping.

References:

[1] K. Kojima et al., “Constant Modulus 4D Optimized Constellation Alternative for DP-8QAM”, ECOC 2014

[2] A. Amari et al., “Introducing Enumerative Sphere Shaping for Optical Communication Systems with Short Blocklengths”, JLT 2019.

[3] Tobis Fehenberger et al., “Mitigating Fiber Nonlinearities by Short-length Probabilistic Shaping”, OFC 2020, paper Th1I.2.