Probabilistic constellation shaping

Probabilistic constellation shaping (PCS, or probabilistic shaping) changes how often each point of a QAM constellation is used instead of changing the constellation itself. Points near the center, which need little energy, are sent more often; corner points are sent rarely. For the same average power the points can then be spread farther apart, and the transmitted signal approaches the Gaussian distribution that achieves channel capacity on a noise-limited link. The benefit is a reduction of the required signal-to-noise ratio of the order of 1 dB at the rates used in fiber links, bounded by 1.53 dB, and a continuously adjustable information rate. The modulation format entry places PCS among the other formats.

Maxwell-Boltzmann distribution

The standard choice assigns point $x$ the probability

$$P(x) \propto \exp\left(-\nu |x|^2\right),$$

a sampled Gaussian. The parameter $\nu \ge 0$ sets the entropy $H$, the information carried per symbol: $\nu$ = 0 gives the uniform constellation with $H = \log_2 M$, and larger $\nu$ concentrates probability in the center and lowers $H$. For square QAM the distribution factorizes into the same one-dimensional distribution on each quadrature, so shaping is applied per dimension.

Worked example. For 64-QAM on an additive white Gaussian noise channel, a mutual information of 5 bits per symbol requires an SNR of 16.14 dB with uniform signaling. The Maxwell-Boltzmann distribution that minimizes the requirement at that rate (entropy 5.70 bits) reaches the same 5 bits at 15.42 dB, a gain of 0.72 dB. The Shannon limit for 5 bits per two-dimensional symbol is $10\log_{10}(2^5 - 1)$ = 14.91 dB, so shaping closes about 60% of the 1.23 dB gap. In this distribution each innermost point is used with probability 0.038 and each corner point with 0.0026, compared with 1/64 ≈ 0.016 for every point in the uniform case.

The 1.53 dB limit

At high SNR a uniform distribution over a square (or cube) region needs more energy than a Gaussian with the same entropy. The ratio of the two average energies is $\pi e/6$, so the largest possible shaping gain is

$$10\log_{10}\frac{\pi e}{6} = 1.53\ \mathrm{dB}.$$

This limit is approached only with large constellations and ideal distributions. Practical PCS systems recover around 1 dB, and less at lower spectral efficiency, where the uniform constellation is already closer to capacity.

Rate adaptation

The widely used probabilistic amplitude shaping (PAS) architecture puts a distribution matcher in front of a systematic forward error correction encoder. The matcher maps uniform data bits into amplitudes with the target distribution; the FEC parity bits, which are close to uniform, select the signs, which leaves the amplitude distribution intact. With $m$ bits per QAM symbol and FEC code rate $R_c$, the net information rate per symbol is

$$R = H - (1 - R_c)\, m.$$

With 64-QAM ($m$ = 6) and $R_c$ = 5/6, an entropy of 5.0 bits gives 4.0 net bits per symbol, and 5.5 bits gives 4.5. On a dual-polarization carrier at 130 GBd that is 1040 and 1170 Gbit/s. The rate is set by changing $\nu$ in the digital signal processing, with the same FEC, symbol rate and optical hardware, so a coherent transceiver can match its rate to the margin of each route in steps much finer than switching between 16-QAM and 64-QAM.

Practical considerations

• Implementation. The distribution matcher and its inverse add latency and logic to the coherent DSP; constant-composition and other matchers trade rate loss against block length.
• Fiber nonlinearity. Shaped constellations have a higher peak-to-average power and higher fourth moment, which slightly raises Kerr nonlinear interference, so part of the linear-channel gain is lost at high launch power. The shaping that is optimal for an AWGN channel is not exactly optimal for a long nonlinear link.
• Receiver impairments. Carrier phase recovery and equalizer adaptation must work with a nonuniform symbol distribution, in which the outer points that carry phase information most clearly are rare.
• Comparisons. Gains should be stated at a fixed net rate, with the FEC and the measure of throughput (mutual information or generalized mutual information) specified.

Common questions

What is the difference between probabilistic and geometric shaping?

Geometric shaping moves the constellation points into a nonuniform, more Gaussian-like arrangement while keeping equal probabilities; probabilistic shaping keeps the square grid and changes the probabilities. Both aim at the same 1.53 dB limit, and probabilistic shaping is more common in fiber systems because it also gives fine rate control with a fixed grid.

Why is the Maxwell-Boltzmann distribution used?

It maximizes entropy for a given average energy on a fixed set of points, the discrete counterpart of the Gaussian, and it is described by a single parameter that sets the rate.

References: G. Böcherer, F. Steiner and P. Schulte, "Bandwidth efficient and rate-matched low-density parity-check coded modulation," IEEE Trans. Commun. 63, 4651 (2015); J. Cho and P. J. Winzer, "Probabilistic constellation shaping for optical fiber communications," J. Lightwave Technol. 37, 1590 (2019); T. M. Cover and J. A. Thomas, Elements of Information Theory, 2nd ed. (Wiley, 2006).