Cornell Tracks Appliances, Home Activities Using a Single Laser-Based VibroSense Sensor

A team of researchers at Cornell University have come up with a way to keep an eye on a range of appliances around the home, from washing machines to toilets — by listening out for the vibrations they create in walls and ceilings.

"Recognizing home activities can help computers better understand human behaviors and needs, with the hope of developing a better human-machine interface," says Assistant Professor Cheng Zhang, senior author of the paper on what the team has dubbed VibroSense. "In order to have a smart home at this point, you’d need each device to be smart, which is not realistic; or you’d need to install separate sensors on each device or in each area. Our system is the first that can monitor devices across different floors, in different rooms, using one single device."

"VibroSense [...] is able to recognize 18 different types of activities throughout a house by observing structural vibration patterns on a wall or ceiling using a laser Doppler vibrometer," the paper's abstract explains. "The received vibration data is processed and sent to a deep neural network which is trained to distinguish between 18 activities."

To prove the concept, a single VibroSense system was installed in a series of five homes for two days each — and proved up to 96.6 accurate in tracking usage of washing machines, microwaves, toilets, and 15 other appliances and activities, at least on the first day. After setting the devices back up on the second day, however, average recognition accuracy dropped to a still useful 89.4 percent.

"Since our system can detect both the occurrence of an indoor event, as well as the time of an event, it could be used to estimate electricity and water-usage rates, and provide energy-saving advice for homeowners," claims Zhang. "It could also prevent water and electrical waste, as well as electrical failures such as short circuits in home appliances."

There are, of course, privacy concerns — particularly issues with using the device in multi-occupancy properties, especially those which share walls or ceilings with neighbors. "It would definitely require collaboration between researchers, industry practitioners, and government to make sure this was used for the right purposes," Zhang admits.

The team's work has been published in the Proceedings of the ACM on Interactive, Mobile, Wearable, and Ubiquitous Technologies under closed-access terms, ahead of a presentation at the ACM International Joint Conference on Pervasive and Ubiquitous Computing next week.