Self-Supervised Learning of Appliance Usage

Challenge: designing a model for weakly-related data streams.

• Location data does not imply appliance usage (people can be next to an appliance without using it, and we do not assume knowing the locations of appliances).
• For energy data, many background cycling events (e.g., fridge, HVAC) are unrelated to residents' locations.
• For location data, there could be multiple people in the environment, and not all are related to appliance activation.
• Cannot learn to encode the two modalities into a shared space (as common in past work), because the two streams are unrelated most of the time.

Our solution: learning cross-modal predictability

The intuition is that when an appliance is activated, location data becomes predictable (because someone is likely to be near the appliance). Therefore, our design:

1. learns to predict locations given the energy event
2. clusters activation events with the learned predictability and cross-modal relation