DBVC: An End-to-End 3-D Deep Biomedical Video Coding Framework

Biomedical videos require tremendous storage space and transmission bandwidth, so efficient coding methods are urgently required. Existing methods can be roughly divided into motion-based methods and wavelet-based methods. Motion-based methods use motion estimation designed for natural videos and independently optimize prediction, transform, and entropy coding modules. Wavelet-based methods treat the more redundant time dimension exactly the same as other spatial dimensions. They are both unable to completely remove the redundant spatial-temporal information in biomedical videos. In this paper, to address these problems, we build an end-to-end framework named DBVC with 3-D motion estimation, MV coding, 3-D motion compensation, and residual coding networks for efficient 3-D biomedical video coding. First, we propose a simple yet efficient 3-D motion estimation network to extract motion information. Specifically, we obtain the region with the most intense motion by a segmentation network and then perform unsupervised motion estimation exclusively on this region. After that, to encode and decode the estimated motion vectors, we apply a 3-D autoencoder-based MV coding network. Moreover, we use a lossless learnable wavelet transform for residual coding, which makes lossless coding possible. To the best of our knowledge, this is the first end-to-end video coding framework that supports both lossy and lossless coding, thus meeting the requirements of 3-D biomedical video coding. Extensive experiments demonstrate that our framework achieves state-of-the-art performance on both 3-D biological videos and 3-D medical videos.

I. Introduction

Biomedical videos describe the change of 3-D biomedical images along the fourth dimension, the time. Redundancies in the time dimension are normally higher than that in spatial dimensions. Among all kinds of biomedical videos, on the one hand, medical videos are widely studied and used for clinical diagnosis. The most representative one is functional magnetic resonance imaging (fMRI) [1], [2], which is a powerful tool for measuring changes in neuronal activity caused by hemodynamics. On the other hand, biological videos are adopted to study structural changes in organisms. For example, 3-D biological videos captured by optical microscopy imaging techniques facilitate observing activities such as cell division [3], [4]. These captured 3-D biomedical videos usually have hundreds of frames along time dimension and pose great challenges on storage and transmission. To address these challenges, effective biomedical video coding methods are urgently needed. These methods are required to satisfy high-bitrate lossy or even lossless coding to preserve all important information, which is beneficial for precise diagnosis and research.