SLIM-Net: Rethinking how neural networks use systolic arrays

Systolic arrays of processing elements are widely used to massively parallelise neural network layers. However, the execution of traditional convolutional and fully-connected layers on such hardware typically requires a non-negligible latency to distribute data over the array before each operation - data is not immediately in-place. This arises from the fundamental incompatibility between the physical spatial nature of a systolic array and the un-physical form of existing neural networks. We propose the systolic lateral mixer network (SLIM-Net) in an effort to reconcile this mismatch. The architecture of SLIM-Net maps directly onto the physical structure of a systolic array such that, after evaluating one layer, data immediately finds itself where it needs to be to begin the next. To evaluate the potential of SLIM-Net we compare it to a UNet model on a COCO segmentation task and find that, for models of equivalent size, SLIM-Net not only achieves a slightly better performance but requires almost an order of magnitude fewer MAC operations. Furthermore, we implement a lateral mixing layer on a systolic smart imager chip which executes seven times faster than similar convolutional layers on the same hardware and provides encouraging initial insights into the practicality of this new neuromorphic approach.