CSAIL Engineers Build a System for Tracking Home Appliances — Without Manual Intervention

A team from the Massachusetts Institute of Technology's (MIT's) Computer Science and Artificial Intelligence Lab (CSAIL) have published a paper detailing how to detect when residents are using particular electrical appliances in a house — using just two sensors.

"Learning home appliance usage is important for understanding people’s activities and optimising energy consumption," the team explains of its focus. "The problem is modelled as an event detection task, where the objective is to learn when a user turns an appliance on, and which appliance it is (microwave, hair dryer, etc.). Ideally, we would like to solve the problem in an unsupervised way so that the method can be applied to new homes and new appliances without any labels."

"To this end, we introduce a new deep learning model that takes input from two home sensors: 1) A smart electricity meter that outputs the total energy consumed by the home as a function of time, and 2) a motion sensor that outputs the locations of the residents over time. The model learns the distribution of the residents' locations conditioned on the home energy signal. We show that this cross-modal prediction task allows us to detect when a particular appliance is used, and the location of the appliance in the home, all in a self-supervised manner, without any labeled data."

The team's model is based on four core concepts: That location data alone can't determine whether someone is simply stood next to an appliance without using it or is actively using it; that energy data may include activity not related to active use, such as a refrigerator cycling on and off; that there may be more than one inhabitant of a given space; and that the two data streams can't be merged into a shared space, as they're unrelated most of the time.

The solution is a deep learning model which is able to predict when an appliance is likely to be used, and where they are in the home, without any user-labeled data. "We introduced a self-supervised solution for learning appliance usage patterns in homes," the team concludes. "We infer appliance usage by learning from data streams of two modalities: the total energy consumed by the home and the residents’ location data. Our approach improves on unsupervised appliance event detection significantly, and learns appliance locations and usage patterns without any supervision."

More information on the work, though not yet the data from the study nor the code for the model, can be found on the CSAIL website along with a link to the paper under open-access terms.