Did I Miss Something?

Where there are wearables, there are gaps in data collection. It is an immutable law of the universe. Whether this is due to device removal for charging, an ill-fitting strap, or motion artifacts, sensor data is inevitably missed from time to time. And that is a pretty big problem for wearable devices — especially those designed for health monitoring applications. Missing data points make it difficult to reliably find patterns and uncover insights that might otherwise help to diagnose a medical condition.

The reason that missing data is such a big issue is due to the way that this sort of data is typically analyzed. Asking users to annotate their own data is impractical in most cases, so self-supervised learning (SSL) methods are commonly employed such that unlabeled data can be used by the learning algorithm. So far so good, but SSL algorithms have a problem of their own — they assume the data set they are learning from is complete. When that is not the case, imperfect methods are used to preprocess the data in an effort to clean it up.

These methods either try to guess the missing values to fill them in, or they simply filter out imperfect stretches of data. Either way, the accuracy of the resulting model suffers. Fortunately, these are no longer the only reasonable options, thanks to the work of a team led by researchers at Google Research. They have developed a Large Sensor Model called LSM-2 that utilizes a novel training framework to learn directly from incomplete data sets.

LSM-2 builds upon its predecessor, LSM-1, by adopting a technique called Adaptive and Inherited Masking (AIM). Rather than treating data gaps as errors that need fixing, AIM views them as inherent features of wearable sensor streams. This shift in perspective allows the model to learn patterns not just in the data that exists, but in the way data is missing.

AIM is an extension of the masked autoencoder training approach. It handles the unpredictability of missing sensor data through a two-part strategy involving token dropout and attention masking. During model training, AIM masks both naturally missing data and artificially masked sections used for reconstruction learning. It first drops a fixed number of tokens to keep training efficient, then uses attention masking to handle any remaining gaps, including those introduced by sensor failure or noise.

To train LSM-2, the team used a dataset of 40 million hours of sensor data collected from over 60,000 participants wearing various Fitbit and Google Pixel devices. The model was then evaluated on tasks like blood pressure and anxiety classification, activity recognition across 20 classes, and health metrics prediction such as BMI and age.

LSM-2 was found to outperform previous models across classification, regression, and generative tasks. And unlike its predecessors, its performance does not degrade significantly when entire sensor streams or chunks of time are missing. Furthermore, it continues to improve as more data and compute are added, suggesting it has not hit its performance ceiling yet. That could signal that brighter times are ahead for wearable health monitors.