General Purpose Deep Learning Accelerator Based on Bit Interleaving

Along with the rapid evolution of deep neural networks, the ever-increasing complexity imposes formidable computation intensity on the hardware accelerator. In this article, we propose a novel computing philosophy called “bit interleaving” and the associate accelerator couple called “Bitlet” and Bitlet-X to maximally exploit the bit-level sparsity. Apart from the existing bit-serial/parallel accelerators, Bitlet leverages the abundant “sparsity parallelism” in the parameters to enforce the inference acceleration. Bitlet is versatile by supporting diverse precisions on a single platform, including floating-point 32 and fixed-point from
$1b$
to
$24b$
. The versatility enables Bitlet feasible for both efficient inference and training. Besides, by updating the key compute engine in the accelerator, Bitlet-X could furthermore improve the peak power consumption and efficiency for the inference-only scenario, with competitive accuracy. Empirical studies on 12 domain-specific deep learning applications highlight the following results: 1) up to
$81\times /21\times $
energy efficiency improvement for training/inference over recent high-performance GPUs; 2) up to
$15\times /8\times $
higher speedup/efficiency over state-of-the-art fixed-point accelerators; 3)
$1.5 {\mathrm{ mm}}^{2}$
area and scalable power consumption from 570 mW
$(fp32)$
to 432 mW
$(16b)$
and 365 mW
$(8b) \text{@}28$
-nm TSMC; 4)
$1.3\times $
improvement of the peak power efficiency for the Bitlet-X over Bitlet; and 5) highly configurable justified by the ablation and sensitivity studies.