The Dynamic Paradox: How Layer-skipping DNNs Amplify Cache Side-Channel Leakages

Traditional neural networks are static: their network structure and parameters remain fixed regardless of the input samples after the networks are trained. In contrast, dynamic neural networks are capable of dynamically adapting their structures and parameters when processing different input samples, e.g., with early exiting and layer skipping, to achieve more efficient use of resources and more effective feature representation. These capabilities make dynamic neural networks a promising area of development for deep learning, with the potential to improve both the efficiency and effectiveness of neural network models.Previous research has demonstrated that the search space of static neural network structures can be significantly reduced by analyzing cache access traces. To date, the side channel risks associated with dynamic neural networks remain unexplored. In this paper, we present the first study on the side-channel leakages of dynamic neural networks which are used to infer the network structure. We conduct both theoretical and experimental validation on a typical dynamic Layer-skipping DNNs structure, SkipNet, and show that the adaptive nature of dynamic DNNs leads to greater side-channel information leakage compared to traditional neural network structures. Specifically, our findings reveal that the network structure space of dynamic DNNs can be significantly reduced, from 366080
⁵⁷
× 2
¹²
× (2
⁶⁴
− 1)
²
to 11286.