An Eye-Catching Innovation

Bio-inspired sensors have emerged as a cutting-edge technological advancement, revolutionizing various industries and propelling innovation to new heights. Taking inspiration from the intricate sensory systems found in nature, such as the olfactory capabilities of animals or the visual perception of insects, scientists have developed sensors that mimic these natural mechanisms. By incorporating the key principles behind these biological systems, researchers have been able to enhance the performance of sensing devices, making them more accurate, responsive, and versatile.

Despite these many advancements, computer vision notably lags far behind the human visual system in terms of performance and efficiency. Image sensors used in computer vision systems capture millions of pixels, resulting in vast amounts of data that require complex processing algorithms to interpret. In contrast, the human visual system is composed of specialized cells that are sensitive to certain wavelengths of light that is preprocessed even before the information is transmitted to the brain. This system is capable of efficiently encoding a high-fidelity representation of the world.

Researchers at Penn State University recognized that while we may not be able to top nature, we would do well to borrow some of its tricks. Towards this end, they have developed a new type of camera that is inspired by the human eye. Specialized sensors mimic the light-sensitive cells of the human eye, and a neural network was developed that mimics the preprocessing of data that occurs in the eye.

The device contains a trio of perovskite photodetector types, which convert light into electrical energy. These perovskite photodetectors were fabricated using a novel process that rendered them sensitive only to a narrow range of light wavelengths. Different types of photodetectors were created to be sensitive to red, green, and blue light, much as the eye contains different types of cone cells, each sensitive to these same wavelengths of light.

In much the same way that solar cells convert light into electricity, so do perovskite photodetectors. This feature of the technology may one day enable the development of battery-free cameras that power themselves as they capture images.

The raw signal data is forwarded into a three-sublayer neuromorphic algorithm that was designed to mimic the eye’s own intermediate image processing capabilities performed by bipolar, horizontal, and amacrine cells. Informed by the electrical current values produced by the sensors, this neural network predicts pixel values.

To prove the concept, a 32 x 32 grid of perovskite sensors was created, which captures images with 1,024 total pixels. When attempting to image a scene with the raw sensors, the researchers found the result to be of poor quality. But after applying the neuromorphic algorithm to the data, the system was then able to create a very close representation of the actual scene it was imaging.

It is hoped that in the future this work will lead us to a better understanding of how the human visual system operates. Looking further down the road, the team hopes that a future technology based on their methods might replace dead or damaged cells in the eyes of the visually impaired or blind.