The Cloud in Your Pocket

Large deep learning models are dramatically reshaping people’s opinions of artificial intelligence (AI), and are finding many useful applications in industry. But so far, we have only seen the tip of the iceberg. These technologies promise to be far more transformative when we move beyond the general, one-size-fits-all models that largely dominate the landscape today, and move into the era of personalization. Consider an AI application that is finely-tuned to who you are, knowing your preferences, personality, and so on. Such an application could revolutionize the way we interact with technology on a daily basis.

Imagine a virtual assistant that not only understands your voice commands but also anticipates your needs based on your past behavior and preferences. This level of personalization could extend to various aspects of your life, from suggesting personalized fitness routines and dietary plans to curating news feeds tailored to your interests. As we delve deeper into personalization, these AI systems could become indispensable companions, seamlessly integrating into our routines and enhancing our overall efficiency and well-being.

The impact of personalized AI extends far beyond personal assistants. In the realm of healthcare, for instance, personalized medical assistants could analyze vast amounts of patient data to offer tailored treatment plans, taking into account individual genetic factors, lifestyle choices, personal response patterns, and more.

However, personalizing large models to individual users requires a great deal of computational power, often far more than what standard devices can provide. This necessitates transferring personal data to cloud servers, where the processing and customization can be done. However, this raises privacy concerns as sensitive information is being transmitted over the internet. This opens the door to data breaches or other unauthorized access that many individuals find unacceptable.

A team led by researchers at MIT is seeking to put the power of personalization in your pocket with their recently published technique called PockEngine. Most portable consumer electronics, like smartphones, do not have the computational horsepower or memory needed to fine-tune a large machine learning model. This is true, in large part, because the way in which modern AI algorithms are trained requires that the full model, with all of its parameters, be loaded into memory at the same time. PockEngine gets around this requirement through some clever tricks that allow it to select specific portions of a larger model for retraining.

Initially, PockEngine fine-tunes each layer of a model, one at a time to understand how each segment contributes to the model’s overall accuracy. The system then determines which layers, or pieces of layers are the most important. These segments are extracted from the full network, then can be fully loaded into memory for additional training on new data for personalization. This process only needs to be done once, so the training process will not take a performance hit for using PockEngine.

The researchers tested their methods on a wide range of systems, ranging from computers with Apple M1 processors to Raspberry Pis and NVIDIA Jetson edge AI computers. It was discovered that on-device training was sped up by as much as a factor of fifteen, and that speed was not met with any decreases in model accuracy. And importantly for these edge platforms, PockEngine also dramatically reduced the amount of memory that was required for retraining.

Experiments were conducted in which popular models, like the Llama-V2 large language model, were retrained using PockEngine. In addition to the aforementioned benefits of the technique, it was also demonstrated that these models could be effectively personalized for individual users.

Moving forward, the researchers intend to further refine their methods, such that it may eventually be possible to retrain even larger models on edge hardware. They plan, for example, to enable retraining models that incorporate both image and text inputs in the near future.