Uncertainty-Aware Multi-Dimensional Mutual Learning for Brain and Brain Tumor Segmentation

Existing segmentation methods for brain MRI data usually leverage 3D CNNs on 3D volumes or employ 2D CNNs on 2D image slices. We discovered that while volume-based approaches well respect spatial relationships across slices, slice-based methods typically excel at capturing fine local features. Furthermore, there is a wealth of complementary information between their segmentation predictions. Inspired by this observation, we develop an Uncertainty-aware Multi-dimensional Mutual learning framework to learn different dimensional networks simultaneously, each of which provides useful soft labels as supervision to the others, thus effectively improving the generalization ability. Specifically, our framework builds upon a 2D-CNN, a 2.5D-CNN, and a 3D-CNN, while an uncertainty gating mechanism is leveraged to facilitate the selection of qualified soft labels, so as to ensure the reliability of shared information. The proposed method is a general framework and can be applied to varying backbones. The experimental results on three datasets demonstrate that our method can significantly enhance the performance of the backbone network by notable margins, achieving a Dice metric improvement of 2.8% on MeniSeg, 1.4% on IBSR, and 1.3% on BraTS2020.

I. Introduction

Brain disease is one of the most common major causes of the increase in world's mortality [1]. Brain and brain tumor segmentation together with subsequent quantitative assessments provide critical information in the study of neuropathology [2], essential for the planning of treatment strategies, monitoring of disease progression, and prediction of patient outcomes [3], [4], [5]. However, manual segmentation is tedious, time-consuming, and easily leads to human biases and mistakes [6]. Computer-Aided Detection (CAD) can assist radiologists in interpreting medical images with dedicated automatic algorithms. In the diagnosis of brain pathologies, magnetic resonance imaging (MRI) plays an important role by providing typical volumetric medical image data.