Multi-Loss Topology-Aware Deep Learning Network for Segmentation of Vessels in Microscopy Images

Accurate segmentation of microvasculature is vital for the analysis of vascular networks. Common deep learning networks using pixel-level loss functions like binary cross-entropy (BCE) or dice are effective for general segmentation but face challenges with thin curvilinear structures that are prone to topology errors. Accurate segmentation of the retinal vessels is essential for precise morphological and structural characterization of the underlying microvascular system. Minor pixel segmentation errors, despite resulting in low BCE or dice errors, can significantly alter the resulting vascular graphs, affecting blood flow analysis, vessel tortuosity, bifurcation numbers and angles, etc. In this paper, we propose a novel deep segmentation network that incorporates two complementary loss functions, one local and one global, computed in spatial and frequency domains respectively to preserve the topological features of the analyzed microvascular network. The proposed global orientation loss can be used both in supervised or unsupervised fashions, further increasing its applicability in scenarios where labeled data is unavailable, scarce, or imprecise. Preliminary results with the proposed approach show improved topology preservation during the segmentation of thin curvilinear structures.