This Tiny Video-Capable "Hyperspectral Camera" Could Give Your Next Smartphone Superpowers

Researchers at the University of Utah and the US Naval Research Laboratory, working with Lumos Imaging, have developed a high-speed high-definition camera that captures more than just visible light — splitting the incoming image into no fewer than 25 distinct spectra.

"When we started out on this research," explains joint project lead Rajesh Menon, a professor in the department of electrical and computer engineering at the University of Utah, of the team's work, "our intention was to demonstrate a compact, fast, megapixel resolution hyperspectral camera, able to record highly compressed spatial-spectral information from scenes at video-rates, which did not exist."

That statement is very deliberately in the past tense, as that is exactly what the team has now created: a camera sensor, small enough that it could be integrated into a future smartphone, which doesn't capture a mere three channels of color information but an impressive 25.

The trick to creating the hyperspectral camera lies in front of the sensor itself: a diffractive filter element made from repeating nanoscale patterns which splits the incoming light so the sensor can record both spatial and spectral information at each pixel. Performance is pushed to video-capable frame rates, meanwhile, by encoding data as a series of individual two-dimensional images — a simpler and faster approach to the three-dimensional "data cubes" created by rival sensors.

"Our camera costs many times less, is very compact and captures data much faster than most available commercial hyperspectral cameras," claims project co-lead and first author Apratim Majumder, an associate professor working alongside Menon. "We have also shown the ability to post-process the data as per the need of the application and implement different classifiers suited to different fields such as agriculture, astronomy, and bio-imaging."

To prove a real-world application for the camera, the team set up three experiments: differentiating between different tissue types in a surgical scene, predicting the age of decaying strawberries, and mimicking the spectral filters used in astrophotography. The researchers also say the sensor could have a place in satellite imagery, being able to beam down much less data than existing hyperspectral cameras yet in a way that the full 3D data cube can be generated in post-processing.

The current prototype captures 25 wavelengths at a resolution of one megapixel; the researchers' next task is to boost both the resolution and the number of distinct spectra captured at each frame. "This work demonstrates a first snapshot megapixel hyperspectral camera," Majumder explains. "Next, we are developing a more improved version of the camera that will allow us to capture images at a larger image size and increased number of wavelength channels, while also making the nano-structured diffractive element much simpler in design."

The team's work has been published under open-access terms in the journal Optica.