Equalization (FFE, DFE, CTLE)

Equalization is the filtering a high-speed link applies to reverse the distortion of its channel. A board trace, cable, optical modulator or photodiode attenuates high frequencies more than low ones, so each transmitted symbol spreads into the time slots of its neighbors, which is intersymbol interference (ISI). An equalizer applies the approximate inverse of that response, restoring the eye at the sampling instant. Electrical lanes at 53 and 106 GBd PAM4 routinely face 20–40 dB of loss at the Nyquist frequency, which the equalizers of a SerDes must undo. Three structures do most of the work: the feed-forward equalizer (FFE), the continuous-time linear equalizer (CTLE) and the decision-feedback equalizer (DFE).

Feed-forward equalizer

An FFE is a finite-impulse-response filter: the output is a weighted sum of the current and neighboring input samples,

$$y_n = \sum_{k} c_k\,x_{n-k},$$

with taps $c_k$ usually spaced by one symbol period. The largest tap is the cursor; taps before it are pre-cursors and taps after it post-cursors. Negative neighbors subtract scaled copies of adjacent symbols, which cancels the tails that the channel adds.

A transmit FFE with taps $(-0.10,\ 0.75,\ -0.15)$, normalized so that the absolute values sum to one as a peak-limited driver requires, has a DC gain of 0.50 (the sum of the taps) and a Nyquist gain of magnitude 1.00 (the sum with alternating signs): a 6.0 dB high-frequency boost, obtained by lowering the low-frequency swing. Transmit FFEs have a few taps; receive FFEs in a DSP after an ADC commonly have tens, adapted by a least-mean-squares rule that moves each tap in proportion to the decision error times the corresponding input sample.

A linear equalizer amplifies noise along with the signal. A receive FFE with taps $(-0.25,\ 1.50,\ -0.25)$ has unit DC gain and a 6.0 dB Nyquist boost, but it multiplies white input noise by the tap norm $\sqrt{\sum c_k^2} = 1.54$, a noise enhancement of 3.8 dB. This is the quantity that TDECQ charges a transmitter for: its reference receiver contains a fixed-length FFE (5 taps for the 50G and 100G-per-lane PAM4 classes), and a waveform that forces large taps pays the noise penalty even after the eye is reopened.

Continuous-time linear equalizer

A CTLE is an analog high-pass stage at the receiver input, typically a differential pair with source degeneration whose zero sits below the Nyquist frequency and whose poles sit above it. It provides a peaking gain, the ratio of its gain near Nyquist to its gain at DC, of a few dB up to roughly 10–20 dB in programmable steps. Acting before sampling, it relaxes the dynamic range demanded of the ADC, at little cost in power. It amplifies high-frequency noise and crosstalk, and its coarse shape cannot match a channel exactly, so a digital FFE, a DFE or both usually follow it.

Decision-feedback equalizer

A DFE removes post-cursor ISI using symbols already decided. If the channel leaves a fraction $b_1$ of each symbol in the next slot, the DFE subtracts $b_1 \hat{d}_{n-1}$ from the current sample, where $\hat{d}_{n-1}$ is the previous decision. Since the decisions are noise-free levels, the subtraction adds no noise, which is the DFE's advantage over a linear equalizer of the same reach. It cannot touch pre-cursor ISI, which the FFE handles.

The cost is error propagation. When a decision is wrong, the DFE subtracts the wrong value from the next sample, which raises the probability that the next decision is also wrong; with a large first tap, a single error can grow into a burst. Burst errors matter because forward error correction codes are designed against a limit on symbol errors per codeword. High-rate PAM4 links therefore limit DFE tap magnitudes, use precoding so that a burst of decision errors maps to only two output symbol errors, and interleave codewords. The feedback must also settle within one symbol period, a constraint that tightens with each increase in symbol rate.

Where equalization sits in optical links

In a conventional pluggable module the retimer DSP equalizes the electrical and optical segments separately. In linear pluggable optics the host SerDes equalizes the full chain of electrical traces, driver, modulator, fiber, photodiode and TIA as one channel. Coherent receivers use much longer equalizers: a static filter of hundreds to thousands of taps for chromatic dispersion and an adaptive 2×2 butterfly FFE that separates polarizations, as described in the coherent DSP entry. Equalization can correct linear distortion only; it cannot repair the level compression that a nonlinear modulator or driver applies to a PAM4 eye diagram.

Common questions

What is the difference between FFE and DFE?

An FFE filters the received samples and can cancel ISI from both earlier and later symbols, at the cost of amplifying noise. A DFE subtracts ISI computed from earlier decisions, adds no noise, and corrects post-cursor ISI only, but it can propagate errors.

How many taps does an equalizer need?

Enough to span the channel's significant pulse response. A short optical module channel may need 3–5 symbol-spaced taps; a long lossy electrical channel needs tens of taps, and the dispersion filter of a long coherent link hundreds to thousands. Each added tap also adds noise enhancement and power.

References: J. G. Proakis and M. Salehi, Digital Communications, 5th ed. (McGraw-Hill, 2008); S. U. H. Qureshi, "Adaptive equalization," Proceedings of the IEEE 73, 1349 (1985); S. J. Savory, "Digital coherent optical receivers: algorithms and subsystems," IEEE Journal of Selected Topics in Quantum Electronics 16, 1164 (2010).