Domain-Incremental Cardiac Image Segmentation With Style-Oriented Replay and Domain-Sensitive Feature Whitening

Contemporary methods have shown promising results on cardiac image segmentation, but merely in static learning, i.e., optimizing the network once for all, ignoring potential needs for model updating. In real-world scenarios, new data continues to be gathered from multiple institutions over time and new demands keep growing to pursue more satisfying performance. The desired model should incrementally learn from each incoming dataset and progressively update with improved functionality as time goes by. As the datasets sequentially delivered from multiple sites are normally heterogenous with domain discrepancy, each updated model should not catastrophically forget previously learned domains while well generalizing to currently arrived domains or even unseen domains. In medical scenarios, this is particularly challenging as accessing or storing past data is commonly not allowed due to data privacy. To this end, we propose a novel domain-incremental learning framework to recover past domain inputs first and then regularly replay them during model optimization. Particularly, we first present a style-oriented replay module to enable structure-realistic and memory-efficient reproduction of past data, and then incorporate the replayed past data to jointly optimize the model with current data to alleviate catastrophic forgetting. During optimization, we additionally perform domain-sensitive feature whitening to suppress model’s dependency on features that are sensitive to domain changes (e.g., domain-distinctive style features) to assist domain-invariant feature exploration and gradually improve the generalization performance of the network. We have extensively evaluated our approach with the M&Ms Dataset in single-domain and compound-domain incremental learning settings. Our approach outperforms other comparison methods with less forgetting on past domains and better generalization on current domains and unseen domains.

I. Introduction

According to recent studies, cardiovascular diseases have become the leading cause of death. The mortality rate of cardiovascular diseases increases progressively year after year [1]. One fundamental task for computer-aided cardiovascular disease diagnosis is cardiac image segmentation, which delineates heart structures for many non-invasive volumetric quantifications, including stroke volumes, ejection fraction, myocardium thickness, and etc [1], [2]. With recent advances in deep learning, considerable deep learning-based approaches have achieved promising performance in this task [1], [3], [4]. However, most of them adopt the static learning setup, where they assume model training and data delivery as a one-step process, but ignore the potential need for model upgrading [5], [6], [7]. In real-world clinical scenarios, new patients keep turning up day by day and induce new data samples to gather over time across institutions. This provides sufficient possibility to promote model functionality towards perfection. To reach this goal, incremental learning (IL) is essential to investigate, which encourages the model to consecutively update upon the previously learnt one to progressively improve itself by exploiting each incoming dataset over time [3], [5], [8].