AD2: Improving Quality of IoT Data through Compressive Anomaly Detection

With recent technological advances in sensor nodes, IoT enabled applications have great potential in many domains. However, sensing data may be inaccurate due to not only faults or failures in the sensor and network but also the limited resources and transmission capability available in sensor nodes. In this paper, we first model streams of IoT data as a handful of sampled data in the transformed domain while assuming the information attained by those sampled data reveal different sparsity profiles between normal and abnormal. We then present a novel approach called AD² (Anomaly Detection using Approximated Data) that applies a transformation on the original data, samples top k-dominant components, and detects data anomalies based on the disparity in k values. To demonstrate the effectiveness of AD², we use IoT datasets (temperature, humidity, and CO) collected from real-world wireless sensor nodes. Our experimental evaluation demonstrates that AD² can approximate and successfully detect 64%-94% of anomalies using only 1.9% of the original data and minimize false positive rates, which would otherwise require the entire dataset to achieve the same level of accuracy.

I. Introduction

Rapid advances in wireless sensor nodes have been key enablers to discover actionable knowledge from raw data and ultimately make smart decisions [1]. This discovery paradigm where knowledge is extracted from a steady stream of data collected from IoT sensor nodes is increasingly adopted by various domains [2], [3]. For instance, IoT-enabled agriculture has transformed traditional farms, which have been relying on human expertise, thereby enabling more precise and productive operations such as the reduction of non-essential pesticides use [4], [5]. A similar IoT platform called Waggle [6] collects real-time urban environmental data, which can be used for enabling a smart city.