Never Stop Learning

Large, cutting-edge artificial intelligence (AI) algorithms may get most of the spotlight, but the majority of today’s practical AI applications run on portable computing devices like smartphones and wearables. When stepping down from a massive cloud computing infrastructure loaded with powerful GPUs to a tiny embedded microcontroller, developers have to cut some corners to make things work. One of the key optimizations involves reducing the size of the model and the amount of data that it is trained on.

There is no doubt that reducing the computational complexity of the algorithm will allow it to run on less powerful systems. But of course this can also have a negative impact on its accuracy. These factors can result in data drift, for example, which is a major challenge in edge AI. Data drift rears its ugly head when a model is trained on a limited dataset, and the data that is encountered in the real world looks significantly different from that dataset.

Human activity recognition, which can be used for device control, fitness, and many other applications is frequently built into wearable and portable devices. But the way that people move and perform activities varies widely, so these systems commonly need to be fine-tuned to each individual user via a supervised on-device learning approach.

How to do this efficiently is still an unsettled question. In particular, assigning labels to the training data before the learning process kicks off presents developers with many difficulties. A pair of researchers at Keio University and the University of Texas at Austin have recently put forth a solution that could help.

Traditional methods of updating the device's learning model require constant communication with a more powerful "teacher" device (e.g., a nearby computer), which can be inefficient and drain the device's battery. To solve this problem, the researchers developed a low-cost, low-power on-device learning system that allows these edge devices to learn and adapt in real-time without needing to constantly rely on the teacher device. The system uses a specialized neural network algorithm called OS-ELM, which is lightweight and efficient enough to run on these small devices. The key innovation is the use of automatic data pruning, which reduces unnecessary communication with the teacher device by ensuring that the edge device only asks for help (in the form of activity labels) when it is truly needed.

The system decides when to skip asking the teacher based on three conditions: sufficient training has already been done, no significant data drift is detected, and the device is confident in its predictions. This confidence is measured by the difference between the top two possible activity labels, and the system dynamically adjusts how confident it needs to be to operate independently. If the device is consistently accurate, it becomes more autonomous, reducing the need for teacher queries and saving power.

To demonstrate the power of their approach, the team designed a custom on-device learning core using a 45 nanometer CMOS process technology that supports automatic data pruning. It was found that this setup used less memory and power than conventional computational resources, and also reduced communication with the teacher system by over 55 percent. A reduction in accuracy of only 0.9 percent was observed using these methods. These findings suggest that the combination of data pruning and the custom hardware could make personalization of AI-powered wearables more practical in the future.