Adaptive Context Aggregation Network With Prediction-Aware Decoding for Multimodal Brain Tumor Segmentation

Precise segmentation of brain tumor from magnetic resonance images (MRIs) is of vital importance for lesion measurement, surgical planning decision, and clinical treatment evaluation. In the context of rapid development of deep neural networks (DNNs), an increasing number of DNN-based brain tumor segmentation methods have been recently reported. However, most methods usually ignore the interaction among multimodal data and multiscale context features, resulting in poor segmentation accuracy on challenging cases, e.g., tumors with complex interactions between subregions. This article proposes a novel adaptive context aggregation network (ACANet) for multimodal brain tumor segmentation. ACANet follows a parallel encoder-decoder framework. Specifically, to bridge the gaps between different modalities, we propose an adaptive context aggregation (ACA) module that adaptively fuses multimodal features from the encoders of two input branches to explore their complementary information and aggregates multiscale context information extracted by a set of dilated convolutions. To cope with the complex boundary interactions, we propose a prediction-aware decoding strategy (PDS) at the decoder to help the network focus more on error-prone regions through the guidance of two temporal predictions when incorporating the features from the encoder. Extensive experiments on two brain tumor segmentation datasets demonstrate the superiority of ACANet over eight mainstream methods. Additionally, a series of ablation studies supports the effectiveness of ACANet’s underlying concepts.