TinyML is rapidly growing in popularity, with new applications appearing at a blistering pace. But there are still many use cases for which it is difficult to develop solutions using current methods. In particular, it is a challenge to develop solutions where continual refinement of models is needed, and data privacy must be ensured.
A duo of researchers at Harvard University has recently built a framework called TinyFedTL to enable on-device retraining of machine learning models that preserves the privacy of training data. TinyFedTL was designed to work with devices that are highly resource-constrained, such as the Arduino Nano 33 BLE Sense (with 1 MB of flash memory and 256 KB of SRAM) that was used to demonstrate the framework.
The popular deep learning framework TensorFlow Lite Micro generates static models that are used for running inferences, but does not support training models on-device. So, to add support for on-device training, the team designed their own custom fully connected layer inference and backpropogation update algorithms in C++. This allows TinyFedTL models to continually learn, but protecting data privacy requires further work.
Towards this end, the researchers created a federated learning implementation. With federated learning, all data remains on the device where training occurs. For other devices to benefit from a larger body of training data, changes to models are summarized and sent to the devices (through a centralized server) where they can be incorporated into their own models through the gold standard Federated Averaging algorithm. The data itself can remain private.
One particularly interesting area where TinyFedTL could be implemented in the future is in healthcare. This is an area where very serious privacy concerns exist, which has slowed the adoption of tinyML techniques. It may be possible to use this framework to create a network of patient wearable sensors that can continually learn and improve their ability to, for example, detect health problems.
The team is currently working on enhancements to both reduce the computational resources needed by the framework, and also to speed up communication during model updates — this presently takes upwards of one minute. There is certainly room for refinement of the method, but this work presents some intriguing early steps towards a network of smarter, and more secure, devices.