Have You Got the Time?

Miniaturization and reduction of manufacturing costs is allowing modern devices to be loaded with high precision sensors that we can carry around with us on a daily basis. As prevalent as the sensors themselves are ideas for their uses, and we have reported on many such ideas. But building a device that makes meaningful sense of sensor input, say an inertial measurement unit designed to detect when the wearer is smoking, typically means first training a machine learning model.

To do so, you will need to collect a lot of data — commonly in the form of sensor measurements paired with labeled video. These measurements and labels need to be very closely synchronized in time so that actions and measurements can be accurately associated. Any mismatches in time will introduce noise in the resulting model. However, commercially-available wearable cameras are not designed for synchronization with other sensors, and the high bandwidth and power requirements for transmitting video wirelessly, along with sensor data, to a central collection point for time stamping make such an approach infeasible.

A collaboration between the Georgia Institute of Technology and Northwestern University has taken this problem on in developing a method called Window Induced Shift Estimation for Synchronization of video and accelerometry (SyncWISE). SyncWISE is a fully automated tool that accepts a video clip and accelerometry data as input and returns a time offset for synchronization.

The algorithm first preprocesses the data to remove any low quality accelerometry data that would otherwise skew the results. This often results from missing data points due to problems with wireless transmissions. Next, estimated acceleration measurements are calculated from the video clips. These estimates are then compared with the actual sensor measurements using a sliding window to find the time offset.

In a study conducted by the researchers, SyncWISE outperformed current state-of-the-art methods by achieving 90% synchronization accuracy. While the current study focused on synchronizing accelerometer and video data, the authors believe the algorithm can be further adapted to accommodate other types of sensors in the future. SyncWISE is freely available for download.