Unsupervised Spatio-Temporal Anomalous Thermal Behavior Monitoring of Inside-Built Environments

Continual wavering of outside weather degrades the efficiency of inside building envelope over time and leads to additional energy consumption, various structural damages, etc. Frequent monitoring of the indoor built environment with thermal images can assist in identifying the energy-leaking and potentially damage-prone areas. Although in recent years different researches performed deep learning and computer vision based thermal anomaly detection in built environment, several issues related to conducting strategic non-intrusive indoor thermal inspection using temporal thermal images, are still unresolved in uncontrolled environment of residential buildings. In this work, we propose a scalable thermal image-based monitoring approach for building envelopes combining the visual knowledge of structural joint information among different building components and their corresponding temporal thermal status. We collected longitudinal thermal images from indoor scenes of different building components (e.g., door, window, wall) and employed a high-level spatio-temporal graph (st-graph) to represent the structural connection among different building components and their temporal self-changes. Our proposed novel unsupervised spatio-temporal clustering framework assigns the cluster label to nodes in st-graph, combining its structural (the self and neighboring component) and temporal features which achieves better performance in identifying thermal variation compared to other clustering based approaches. We demonstrate thermal variation across the spots which indicates the potential energy leakage areas inside the built environment. The cluster patterns obtained from our proposed model assist in understanding the thermal characteristics of various surfaces at certain conditions, such as sun reflection and airflow in the inside built environment.