Iterative Bi-Directional Optical Flow for Decoder Side Motion Vector Refinement

Multi-pass Decoder Side Motion Vector Refinement (DMVR) based on a Bilateral Matching (BM) concept is a very important tool being used in video codecs to improve the Motion Vector (MV) accuracy on the decoder side. In VVC, a single stage DMVR is applied on each 16x16 subblock to refine its MV. Exploring Coding Model (ECM), which is being developed for beyond VVC activity, adopts multi-pass DMVR. First, a BM approach is used to adjust the MV for each Prediction Unit (PU). Next, another BM approach is used to adjust the MVs for each 16x16 subblock. Finally, a Bi-directional Optical Flow (BDOF) based approach is used to adjust the MVs for each 8x8 subblock. In this paper, an iterative BDOF to improve multi-pass DMVR is proposed, where an additional pass of BDOF is added as the 4th stage of the multi-pass DMVR. To balance the gain and the complexity, the granularity of subblocks is adjusted adaptively in the third and forth stages of multi-pass DMVR. Finally, a regularization factor is added to avoid unnecessary over-tuning of BDOF adjustment. The proposed method is implemented on top of ECM-8.0. Experimental results show that the proposed method provides more than -1% bitrate saving for some sequences compared to ECM-8.0 under random access configurations. This method is adopted into ECM-10.0.