VT-Scanner: Layout Similarity on Smartphone and Its Application for Robust Scene Recognition

Mobile applications are becoming attractive targets for revealing usage habits and trends. However, most research studies mainly focus on network traffic classification, which corresponds only to the characteristics of data transfer. Besides, encrypted traffic has limited the attention to robustness as well as intrinsic relationships between apps. In this paper, we introduce VT-Scanner, a robust method for recognizing usage scenarios from user interface layouts. Unlike existing works on UI representation, we first construct a histogram feature for View Tree to explore cross-app similarity with interpretability. We further employ a tree-fitting encoder Tree-LSTM, incorporating multi-view prior knowledge for reconstruction mechanism. VT-Scanner learns a more general representation from View Tree than pixel or layout visualization. Our experiments on scene recognition and app recognition show the high robustness of VT-Scanner to zero-day applications, version differences, and device variations, while ensuring privacy.