METIER Model Combines Wearable Sensor Activity and User Recognition Tasks to Excel at Both

A trio of scientists at Zhejiang University have released details of a deep learning system designed to combine activity recognition and user recognition into a single system based on wearable sensors: METIER, the deep multi-task learning based activity and user recognition model.

"Activity recognition (AR) and user recognition (UR) using wearable sensors are two key tasks in ubiquitous and mobile computing," the researchers explain of the problem they attacked. "Currently, they still face some challenging problems. For one thing, due to the variations in how users perform activities, the performance of a well-trained AR model typically drops on new users. For another, existing UR models are powerless to activity changes, as there are significant differences between the sensor data in different activity scenarios."

"To address these problems, we propose METIER (deep multi-task learning based activity and user recognition) model, which solves AR and UR tasks jointly and transfers knowledge across them. User-related knowledge from UR task helps AR task to model user characteristics, and activity-related knowledge from AR task guides UR task to handle activity changes. METIER softly shares parameters between AR and UR networks, and optimizes these two networks jointly."

In METIER, the user is equipped with wearable sensors — a smartphone attached to the waist for the first dataset, a smartphone in the pocket for the second dataset, and nine inertial measurement units (IMUs) distributed around the body for the third dataset — and the captured data processed using TensorFlow. By cross-training for AR and UR tasks, METIER performs well at both — something the researchers point out isn't the case for existing models.

The team's paper has been published under open access terms as part of the ACM International Joint Conference on Pervasive and Ubiquitous Computing 2020 (UbiComp '20), while the source code for METIER has been published on GitHub under an unspecified license.