Photonic tensor core

A photonic tensor core is the processing unit of optical computing: a circuit whose job is the multiply-accumulate operation, a matrix times a vector, that makes up most of the arithmetic in neural-network inference. The name borrows from the tensor cores in GPUs, which perform the same operation electronically. The input vector is encoded on optical signals, the matrix is stored in the circuit, and the products are summed either by interference or by photodetection.

There are two main architectures. Coherent designs use a mesh of Mach-Zehnder interferometers, each with phase shifters, arranged so that any unitary $N \times N$ matrix can be set with $N(N-1)/2$ interferometers, 2016 of them for $N = 64$; an arbitrary matrix is built from two such meshes and a column of attenuators, following its singular-value decomposition. All inputs share one wavelength, and the answer appears in the output amplitudes. Incoherent designs put each input on its own wavelength and weight it by the transmission of a microring or of a phase-change cell in a crossbar; a photodetector then sums the power of all the wavelengths it receives, which performs the accumulate. Because transmission cannot be negative, signed weights use pairs of detectors in balanced detection. A crossbar of phase-change cells fed by a microcomb has been used to run convolutions this way.

The appeal is in the scaling. An $N \times N$ core updated at a symbol rate $f$ performs $N^2 f$ multiply-accumulates per second, 41 TMAC/s for $N = 64$ at 10 GS/s. The data converters around it scale only with $N$: $N$ conversions in and $N$ out per vector, so if each conversion costs 1 pJ, the conversion energy per multiply-accumulate is $2/N$ pJ, 31 fJ at $N = 64$, and falls as the array grows. Against that, optical loss grows with the mesh depth, so the laser power needed to keep a given signal-to-noise ratio grows too, and the signal-to-noise ratio sets the precision: an effective resolution of $(\mathrm{SNR} - 1.76)/6.02$ bits gives 4.7 bits at 30 dB and 6.4 bits at 40 dB, below the 8-bit integer formats common in inference.

Holding the weights is the other practical constraint. Thermo-optic phase shifters draw power continuously and drift, so the matrix must be calibrated and trimmed; phase-change cells hold their state without power but switch slowly and offer a limited number of levels. Why these costs have so far kept photonic tensor cores in research, while optics has succeeded in moving data between electronic processors, is argued in Why light moves data and electrons still compute.

References: M. Reck, A. Zeilinger, H. J. Bernstein, P. Bertani, Phys. Rev. Lett. 73, 58 (1994); W. R. Clements et al., Optica 3, 1460 (2016); J. Feldmann et al., Nature 589, 52 (2021); M. A. Nahmias et al., IEEE J. Sel. Top. Quantum Electron. 26, 7701518 (2020).