Sony Launches "World's First" Vision Sensor with On-Board Edge AI Processing Capabilities
Designed for edge AI, Sony's new IMX500 and IMX501 can run computer vision tasks entirely locally — and quickly, too.
Sony has announced a pair of new vision sensors which, it claims, are the first to feature on-board processing capabilities for edge artificial intelligence (edge AI) — and it has its eyes firmly on the Internet of Things (IoT).
Launching in two variants — the IMX500 bare sensor and IMX501 package — Sony's latest image sensor appears at first glance to be relatively simple: It's a 1/2.3-type image sensor capturing 4056x3040 imagery at up to 60 frames per second with MIPI D-PHY 1.2 four-lane and SPI outputs. The packaged variant is in a ceramic land grid array (LGA) package measuring 12.5x15mm (around 0.5x0.6").
There's more than just the sensor on-board, however: There's a processing engine which, the company says, can perform various artificial intelligence tasks without the need for external processing. Built into a digital signal processor with dedicated memory, located under the sensor chip itself in a stacked-chip format, the logic processor is capable of running incoming imagery through a deep learning network with impressive speed: The company claims that live video can be processed directly on-device at a rate of 3.1 milliseconds per frame — based on the MobileNet V1 network.
Data from deep learning networks can then be used in a variety of ways: The sensor can output metadata, such as text classifying the image, it can output only the region of interest (ROI) defined by the network, or it can output an image with overlays. It can also be programmed by the user: Sony suggests the same sensor could be put to use in the retail industry for footfall counting, consumer behaviour monitoring, stock checking, congestion monitoring, and heat mapping.
While Sony's sensor, sampling now in chip-only form with the packaged variant due to sample in June, seems likely to be the first image sensor with embedded AI to be launched commercially, it's not the first to be announced: Earlier this year researchers at the Vienna University of Technology unveiled their own combined image sensor and neural network device, capable of being powered by nothing more than the very photons it's imaging.
Sony has confirmed sample pricing for the new sensor at 10,000 JPY bare-chip and 20,000 JPY packaged (around $93 and $187 respectively), exclusive of tax, with no word yet on volume pricing and availability. More information is available on the company website.