Adapt or Perish

If you have spent any time at all training neural networks, either in a professional capacity or as a hobbyist, then you know that these models do not always operate as expected in practice. Consider an image classification or object detection model, for instance. It may perform with near-perfect accuracy on the test dataset, but when taken out into the field for real-world validation, that accuracy level could very easily fall right off a cliff.

This is generally because there is some type of noise in the data — which could be as simple as the low-light conditions present on a cloudy day — that the model did not encounter in the training dataset. Since the algorithm never learned how to deal with this type of data, accuracy is greatly impacted. In theory, these issues could be resolved by training the model on a more diverse dataset, but it is virtually impossible to account for every potential situation.

An emerging paradigm called test-time adaptation (TTA) seeks to deal with this problem by using unlabeled test data to adapt models to distributional shifts in inputs in real time. Unfortunately, existing TTA implementations hurt the performance of a model when conditions change frequently. They are also unaware of when domain shifts occur, so the adjustment process is continuously happening even when it is not necessary. These issues increase both computational complexity and energy consumption, which in turn makes TTA-based approaches impractical for many edge AI applications.

Researchers at Northeastern University and the Air Force Research Laboratory have proposed a framework called Domain-Aware Real-Time Dynamic Adaptation (DARDA) to deal with these shortcomings of current TTA approaches. DARDA works by detecting sudden data distribution changes in real-time, extracting the corruption characteristics, and generating a latent representation called a “corruption signature.” This signature is matched to a pre-learned corruption centroid, which corresponds to a known corruption type. Each centroid is linked to a specialized sub-network within the main neural network, which is dynamically selected and activated to handle the specific corruption.

As new corruptions occur, DARDA adapts by moving the sub-network closer to the ongoing corruption in real time. This allows the neural network to maintain performance even when encountering unforeseen corruptions. DARDA’s design reduces unnecessary computational overhead by avoiding continuous adaptation when the data distribution remains stable, making it highly efficient in terms of energy consumption and memory usage.

The researchers tested DARDA on both a Raspberry Pi 5 and an NVIDIA Jetson Nano to understand how well it works on edge computing platforms. When compared with a state of the art TTA algorithm, the new approach was shown to be 1.74 times more energy efficient and 7.3 times faster while using 2.64 times less cache memory. Given the unique characteristics of DARDA, it is particularly well-suited for applications like autonomous vehicles, where environmental conditions frequently change.