Researchers Turn to Smart Orchestration and Task-Specific Models to Get the Most Out of TinyML
Team shows interference-correcting models successfully running on an ultra-wideband tracker with just 4kB of RAM.
A team of researchers from the Graz University of Technology (TU Graz), Pro²Future, and the University of St. Gallen is looking to take tiny machine learning (tinyML) to its absolute limit — deploying artificial intelligence (AI) models on Internet of Things (IoT) devices with as little as 4kB of RAM.
"Of course, these small devices do not run large language models, but rather models with very specific tasks, for example to estimate distances," explains project lead Michael Krisper, a scientist at TU Graz' Institute of Technical Informatics. "But you also have to get these models small enough first. This requires a few tricks and it is precisely these tricks that we have been working on as part of the project."
Machine learning is not a new technology, but an explosion of interest in large language models (LLMs) — the technology powering chatbots like OpenAI's ChatGPT and Google's Gemini — has spilled over into other, more lightweight implementations. In the case of these researchers, it's tinyML: embedded machine learning designed for resource-constrained devices, including microcontrollers.
The project saw a "specialized AI model" run on-device on an ultra-wideband (UWB) location tracker with just 4kB of memory, pinpointing and correcting for sources of interference in location data — without having to transmit said data to a more powerful external system. The secret: splitting the task down into application-specific models, rather than one single model that does everything. If there is interference from metal walls, for example, it is handled by one model; interference from shelves another; and interference from people is handled by a third model.
Deciding which model to use is the job of an orchestration system that, the team found, was able to determine the type of interference present and to load the required model within 100 milliseconds. The same approach, the team says, could be used elsewhere — in a car keyfob, for example, to block keyless relay attacks, or to boost battery life in smart home remote control systems.
More information on the project is available on the TU Graz website.