A Lens-Free Camera, Passed Through a Vision Transformer, Could Unlock New Forms of Photography
Despite its lack of lens, this camera can capture clear images — and could lay the foundation for one-shot 3D and post-capture focus.
A team at the Tokyo Institute of Technology has come up with a way to take the blurred mess captured when using an image sensor without an attached lens and turn it into a discernible image, potentially opening the path to ultra-compact cameras as well as offering 3D and post-focus capabilities.
"Without the limitations of a lens," explains Masahiro Yamaguchi, professor at Toykyo Tech and co-author of the potential held in the paper revealing the team's work, "the lens-less camera could be ultra-miniature, which could allow new applications that are beyond our imagination."
Typically, an image sensor works behind a lens: Light is captured and focused on the sensor, which can then provide a pixel-by-pixel image to the connected system. Yamaguchi and colleagues, however, are using the sensor behind a thin mask — an approach that usually produces nothing more than a colorful smudge.
That smudge, the researchers have found, has enough information in it to recreate the original image as though a lens had been in place. They're not the first to attempt it: Previous work has leaned on convolutional neural networks, with mixed results — where the Tokyo Tech team has instead employed a vision transformer (ViT) creating far clearer images than previous approaches and in a fraction of the time, allowing for real-time image capture and processing.
"We realize that miniaturization should not be the only advantage of the lens-less camera," says lead author Xiuxi Pan, of the potential for the team's work. "The lens-less camera can be applied to invisible light imaging, in which the use of a lens is impractical or even impossible.
"In addition, the underlying dimensionality of captured optical information by the lens-less camera is greater than two, which makes one-shot 3D imaging and post-capture refocusing possible.
"We are exploring more features of the lens-less camera. The ultimate goal of a lens-less camera is being miniature-yet-mighty. We are excited to be leading in this new direction for next-generation imaging and sensing solutions."
The team's work has been published under closed access terms in the journal Optics Letters.