A Multi-branch Deep Learning Architecture for Microwave-Ultrasound Breast Imaging

A novel deep convolutional fusion architecture with multiple branches is proposed to address the challenges posed by electromagnetic inverse scattering problems. Traditional inverse scattering techniques suffer from various difficulties, including strong ill-conditioning, high computational cost, and intrinsic nonlinearity. To overcome these challenges, a multi-branch convolutional neural network (CNN) is proposed for reconstructing two-dimensional (2D) images of breast permittivity in order to detect tumors. The proposed architecture, which is influenced by objective-function methods, inputs scattered-field data and ultrasound-derived breast mask as prior information to reconstruct images of permittivity. The architecture consists of two branches, a decoder-only branch and a convolutional branch, to handle inputs of different formats. The final reconstructed 2D image is obtained by fusing the outputs of these two branches. The CNN is trained using eight numerical MRI-based breast models. Results demonstrate that the proposed approach provides high-quality imaging for high-contrast objects of interest. This work opens up a new avenue for real-time quantitative microwave imaging using deep learning.