BPNet: Branch-pruned Conditional Neural Network for Systematic Time-accuracy Tradeoff

Recently, there have been attempts to execute the neural network conditionally with auxiliary classifiers allowing early termination depending on the difficulty of the input, which can reduce the execution time or energy consumption without any or with negligible accuracy decrease. However, previous studies do not consider how many or where the auxiliary classifiers, or branches, should be added in a systematic fashion. In this paper, we propose Branch-pruned Conditional Neural Network (BPNet) and its methodology in which the time-accuracy tradeoff for the conditional neural network can be found systematically. We applied BPNet to SqueezeNet, ResNet-20, and VGG-16 with CIFAR-10 and 100. BPNet achieves up to 3.15× of speedup without any accuracy drop compared to the base networks.