RCIF: Towards Robust Distributed DNN Collaborative Inference Under Highly Lossy Networks

Collaborative Inference is a prospective paradigm for accelerating Deep Neural Network (DNN) inference by harnessing the computational resources of multiple devices. However, in highly lossy network environments, such as those encountered in wireless communication systems, the transmission loss of intermediate feature maps between devices can result in significant degradation of co-inference accuracy. In this paper, we first conduct a comprehensive investigation into the impact of intermediate feature map loss in real-world wireless scenarios and provide an in-depth analysis of loss patterns under UDP transmission. Motivated by these observations, we introduce Robust Co-inference Framework (RCIF), a novel framework that employs a hierarchical mask strategy to selectively drop activations at two different scales of feature maps. This approach enhances the robustness of DNN co-inference in the presence of network losses. Our evaluation on a variety of datasets and network architectures demonstrates that RCIF significantly enhances the accuracy and robustness of distributed DNN co-inference under highly lossy network conditions. Specifically, our results show that RCIF can achieve up to a 659% increase in accuracy compared to the original model under particularly poor network conditions.