Learning appliance usage in homes

Problem: detect when different appliances are turned on.

Why learning appliance usage?

  • Useful for energy saving and improving energy efficiency
    (consumer, industry, and policy benefits are discussed in Armel et al 2013).
  • Health sensing by understanding user habits and activities at home (e.g., for seniors living alone)
  • Provide behavioral analytics

Past work has looked at the problem using energy data from the utility meter. These meters sample the total energy consumed by a home over time. The challenge is that the input data is a mix of multiple patterns.


Sapple

In this work, we stay in the unsupervised domain -- we do NOT assume knowing the energy patterns of the appliances, or any human labels. However, we take a new perspective: our idea is to use in-door location data to provide self-supervising information. Our solution combines utility meter's total energy data with residents' location data from a passive radio sensor.

We show that our design allows us to detect when a particular appliance is used, and the location of the appliance in the home, all in a self-supervised manner, without any labeled data.



Challenge: designing a model for weakly-related data streams.

  • Location data does not imply appliance usage (people can be next to an appliance without using it, and we do not assume knowing the locations of appliances).
  • For energy data, many background cycling events (e.g., fridge, HVAC) are unrelated to residents' locations.
  • For location data, there could be multiple people in the environment, and not all are related to appliance activation.
  • Cannot learn to encode the two modalities into a shared space (as common in past work), because the two streams are unrelated most of the time.

Our solution: learning cross-modal predictability

The intuition is that when an appliance is activated, location data becomes predictable (because someone is likely to be near the appliance). Therefore, our design:

  1. learns to predict locations given the energy event
  2. clusters activation events with the learned predictability and cross-modal relation


Paper

Self-Supervised Learning of Appliance Usage
Chen-Yu Hsu, Abbas Zeitoun, Guang-He Lee, Dina Katabi, Tommi Jaakkola
International Conference on Learning Representations (ICLR) 2020
[PAPER] [TALK (5 minutes)]


Code & Data

Coming soon ... Stay tuned! Email cyhsu@csail.mit.edu if you are interested.


Press

Sapple was covered by: Tech Crunch, Engadget, DailyMail, MIT news, and other media outlets.


Also check out

Enabling Identification and Behavioral Sensing in Homes using Radio Reflections
Chen-Yu Hsu, Rumen Hristov, Guang-He Lee, Mingmin Zhao, Dina Katabi
CHI 2019
[WEBSITE] [PAPER] [VIDEO]

Zero-Effort In-Home Sleep and Insomnia Monitoring using Radio Signals
Chen-Yu Hsu, Aayush Ahuja, Shichao Yue, Rumen Hristov, Zachary Kabalec, Dina Katabi
Ubicomp 2017
[WEBSITE] [PAPER]

Extracting Gait Velocity and Stride Length from Surrounding Radio Signals
Chen-Yu Hsu, Yuchen Liu, Zachary Kabelac, Rumen Hristov, Dina Katabi, Christin Liu
CHI 2017
[PAPER] [VIDEO]

Capturing the Human Figure Through a Wall
Fadel Adib, Chen-Yu Hsu, Hongzi Mao, Dina Katabi, Fredo Durand
SIGGRAPH Asia 2015
[WEBSITE] [PAPER] [VIDEO]

Smart Homes that Monitor Breathing and Heart Rate
Fadel Adib, Hongzi Mao, Zachary Kabelac, Dina Katabi, Robert C. Miller
ACM CHI 2015
[WEBSITE] [PAPER] [VIDEO1] [VIDEO2]

3D Tracking via Body Radio Reflections
Fadel Adib, Zachary Kabelac, Dina Katabi, Robert C. Miller
Usenix NSDI 2014
[WEBSITE] [PAPER] [VIDEO] [SLIDES]