Hyperspectral imaging

Hyperspectral imaging combines a camera and a spectrometer: for every pixel of a scene it records a spectrum sampled in many narrow, contiguous bands. The result is a datacube with two spatial axes and one spectral axis. A typical visible and near-infrared (VNIR) instrument covers 400–1000 nm in 100–300 bands of a few nanometers each; the airborne AVIRIS instrument records 224 channels from 400 to 2500 nm, about 10 nm per channel. Multispectral imaging, by contrast, records a handful of broader, often non-contiguous bands; Landsat 8's Operational Land Imager has nine. The spectrum at each pixel identifies materials through their absorption and reflectance features, which a color camera with three broad channels cannot resolve.

How the datacube is acquired

A detector array is two-dimensional, so any instrument must trade one of the three datacube axes against time or against sensor area.

• Pushbroom (line scan). A slit selects one line of the scene; a diffraction grating or prism disperses it across the second axis of an area sensor. Each frame is one spatial line by all wavelengths, and the second spatial axis is built up by moving the scene, the platform or a scan mirror. This is the most common design for aircraft, satellites and conveyor-belt sorting.
• Whiskbroom. A single point is dispersed onto a line detector; a mirror scans it across the track and the platform's motion supplies the second spatial axis. AVIRIS uses this approach.
• Tunable filter (staring). A liquid-crystal or acousto-optic tunable filter selects one band at a time, and the camera records full two-dimensional frames band by band. Suited to static scenes and microscopy.
• Snapshot. The whole cube is captured in one exposure, for example with a mosaic of filters over the pixels, much like a Bayer pattern. A 2048 × 2048 sensor with a 4 × 4 mosaic gives 16 bands at 512 × 512 pixels; spatial and spectral sampling share the same pixels. Snapshot designs suit moving scenes but have fewer bands.

Detectors and spectral ranges

Silicon CCD and CMOS sensors (see CCD vs CMOS) cover the VNIR range, about 400–1000 nm, with quantum efficiency falling steeply above 900 nm. The shortwave infrared, 1000–2500 nm, requires InGaAs arrays (standard material to about 1.7 µm, extended-composition material further) or HgCdTe arrays cooled to reduce dark current; the SWIR camera entry covers these sensors. Many mineral, water and organic absorption features lie in the SWIR, so remote sensing instruments usually pair a VNIR and a SWIR spectrometer. Longer infrared bands use cooled HgCdTe or other thermal detectors.

Data volume

A VNIR pushbroom camera with 1024 spatial pixels per line, 224 bands and 16-bit storage produces

$$1024 \times 224 \times 2\ \text{bytes} \approx 459\ \text{kB}$$

per line. At 100 lines per second that is about 46 MB/s, and a 1000-line scene is 459 MB, about 75 times the size of a 16-bit RGB image of the same scene (6.1 MB). Storage, downlink and processing are therefore design constraints, and dimensionality reduction (principal component analysis, band selection) is a routine first step.

Calibration and processing

Raw counts are converted to radiance using the dark frame and a calibrated source, then to reflectance with a white reference panel imaged under the same illumination, or, for airborne data, with an atmospheric correction model. Classification compares each pixel spectrum with library spectra (spectral angle mapping is a common measure) or uses trained classifiers. Spectral unmixing estimates the fractional abundance of several materials within a pixel that is larger than any one of them.

Applications

Mineral mapping and geology, where the method originated; vegetation health and precision agriculture through chlorophyll and water features; food and pharmaceutical inspection on production lines; plastic sorting for recycling in the SWIR; and medical imaging of tissue oxygenation, which depends on the different absorption spectra of oxygenated and deoxygenated hemoglobin. Raman and fluorescence microscopes that record a full spectrum at each scan position produce the same kind of datacube; see Raman spectroscopy.

Pitfalls

Pushbroom instruments suffer from spectral "smile" (the center wavelength varies along the slit) and "keystone" (spatial misregistration that varies with wavelength); both distort pixel spectra if uncorrected. Narrow bands collect little light, so signal-to-noise ratio per band is lower than for a broadband camera at the same exposure. Illumination matters: halogen lamps are smooth across VNIR and SWIR, while white LEDs have little output beyond 700 nm.

Common questions

What is the difference between hyperspectral and multispectral imaging?

Multispectral imaging records a few (typically 3–15) discrete bands, often tens of nanometers wide. Hyperspectral imaging records many contiguous narrow bands, enough to reconstruct a continuous spectrum at each pixel. There is no strict band-count threshold; the distinction is contiguity and spectral resolution.

What wavelength range does a hyperspectral camera cover?

VNIR cameras with silicon sensors cover roughly 400–1000 nm. SWIR cameras with InGaAs or HgCdTe cover about 900–1700 nm or 1000–2500 nm. Combined instruments span 400–2500 nm.

What is a hyperspectral datacube?

A three-dimensional array with two spatial dimensions and one spectral dimension. Each spatial position holds a spectrum, and each spectral slice is a monochrome image at one wavelength.

References: A. F. H. Goetz, G. Vane, J. E. Solomon, B. N. Rock, Science 228, 1147 (1985); R. O. Green et al., Remote Sensing of Environment 65, 227 (1998); G. Lu, B. Fei, Journal of Biomedical Optics 19, 010901 (2014).