Scattering-Based Hybrid Networks: An Evaluation and Design Guide

Hybrid networks combine fixed and learnable filters to address the limitations of fully trained CNNs such as poor interpretability, high computational complexity and a need for large training sets. Many hybrid designs were proposed, utilising different filter types, backbone CNNs and different approaches to learning. They were evaluated on different (and often simplistic) datasets, making it difficult to understand their relative performance, their strengths and weaknesses, also there are no design guides on building a hybrid application for the problem at hand. We present and benchmark a collection of 27 networks, some new learnable extensions to existing designs, all within a framework that allows an assessment of a wide range of scattering types and their effects on the system performance. Also, we outline application scenarios most suitable for hybrid networks, identify previously unnoticed trends and provide guidance in building hybrids.