A Little Data Goes a Long Way

The latest wave of artificial intelligence (AI) applications has certainly wowed the masses, but in most cases, their capabilities have come at the expense of efficiency. Researchers have learned to give these algorithms very advanced perception capabilities, sometimes rivaling those of humans, but the resemblance is only superficial — they work in completely different ways. Whereas people can make sense of the world around them with just a little bit of information, AI requires large amounts of data to achieve any reasonable level of understanding.

Consider, for instance, visual perception. A person could recognize a lion hiding in the bushes from a quick and heavily-obscured glimpse. A neural network, on the other hand, has to analyze each and every pixel in a high-resolution image to achieve a similar result. Given that Internet of Things devices alone are now generating tens of trillions of gigabytes of data each year, this is a big problem. As this volume of data grows, the costs involved in analyzing it will become astronomical.

If we are going to keep up with this growth in available data, more efficient AI algorithms will need to be developed to help us make sense of it. That is exactly the challenge a team of researchers from Pennsylvania State University and MIT have taken on. Their newly developed Shift-Invariant Spectrally Stable Undersampled Network (SIUN) promises to drastically reduce the amount of sensor data needed for AI-driven tasks while maintaining accuracy. Their research introduces a selective learning approach where the data collected is tailored to the specific problem at hand.

Traditional AI models, particularly those used in industrial sensing and scientific computing, rely on the Shannon-Nyquist sampling theorem. This principle, formulated in the 1940s, states that to avoid losing information, a signal must be sampled at a rate at least twice its bandwidth. However, this leads to massive amounts of redundant data being collected and processed, straining computational resources.

The SIUN approach challenges this notion by introducing selective sampling, which was inspired by human perception. Unlike traditional methods that capture and process all available sensor data, SIUN intelligently samples a fraction of the data at Nyquist rates, ensuring that only the most relevant portions are used for analysis. The architecture maintains shift invariance through localized windowing and ensures spectral stability by preserving relative positions of data points rather than absolute values. Using a neural network-based approach, SIUN adapts to different sensing tasks — such as classification and regression — while drastically reducing computational overhead, memory requirements, and latency compared to conventional deep learning models like convolutional neural networks.

In tests involving industrial sensor data, SIUN was able to correctly classify faulty machinery with 96% accuracy while sampling only 30% of the raw data. In contrast, a traditional convolutional neural network achieved slightly higher accuracy (99.77%) but required the full dataset, making it computationally expensive. In other cases, SIUN maintained 80-90% accuracy with just 20% of the data.

Since AI systems using SIUN can function effectively with far less computational power, they are ideal for edge computing applications where data storage and processing resources are limited. This could be particularly useful for applications in remote or extreme environments such as deep-sea exploration or space missions.

To drive this point home, the researchers ran SIUN on the tiny, $4 Raspberry Pi Pico microcontroller. Despite its severely limited hardware resources, the system successfully performed AI inference tasks, proving that SIUN could bring advanced AI capabilities to even the most resource-limited devices.