You Little SCAMP!
Image classification at 17,500 FPS without a CPU or GPU? Yes, please!
Artificial intelligence systems need to process image data for many applications. To do so, a large volume of data must be transferred from an image sensor to a GPU or CPU for processing. Much of this data is likely irrelevant to the task at hand — consider, for example, the details of leaves on roadside trees that are imaged by a self-driving car. Transferring and processing this irrelevant data along with the relevant data increases processing time and power consumption.
A research collaboration between the University of Manchester and University of Bristol has proposed a new path forward in which the camera itself can run a convolutional neural network. This method is able to process thousands of frames per second, and send higher level information (e.g. a classification) directly to the next processing step. In addition to greatly increasing efficiency, privacy is also enhanced in that image data need not be stored.
This new SCAMP architecture has been built into a camera-processor chip which serves as a Pixel Processor Array (PPA). Each pixel in a PPA contains a processor and is able to communicate with every other pixel. Devices implementing the SCAMP architecture are similar in size to existing image sensors.
SCAMP still relies on traditional means of training machine learning models, after which a general purpose PPA is configured to run the model. The team demonstrated their SCAMP camera doing hand gesture recognition and plankton classification at speeds of 2,000 to 17,500 frames per second, all while consuming less than 1.5 watts of power.
The researchers note that as the model size increases, the performance of PPAs begins to decrease. They suggest that this is due to the relatively dated 180 nanometer CMOS silicon technology available to them in academia. Using state-of-the-art manufacturing processes would help to remove this limitation.
SCAMP appears to be a very significant advancement for machine learning inference tasks, and has the potential to provide a giant leap forward for tiny machine learning applications. We hope this will prove to perform in the real world as well as it has in the lab.