Abstract:
This paper overviews the motion vector coding and block merging techniques in the Versatile Video Coding (VVC) standard developed by the Joint Video Experts Team (JVET). ...Show MoreMetadata
Abstract:
This paper overviews the motion vector coding and block merging techniques in the Versatile Video Coding (VVC) standard developed by the Joint Video Experts Team (JVET). In general, inter-prediction techniques in VVC can be classified into two major groups: “whole block-based inter prediction” and “subblock-based inter prediction”. In this paper, we focus on techniques for whole block-based inter prediction. As in its predecessor, High Efficiency Video Coding (HEVC), whole block-based inter prediction in VVC is represented by adaptive motion vector prediction (AMVP) mode or merge mode. Newly introduced features purely for AMVP mode include symmetric motion vector difference and adaptive motion vector resolution. The features purely for merge mode include pairwise average merge, merge with motion vector difference, combined inter-intra prediction and geometric partitioning mode. Coding tools such as history-based motion vector prediction and bidirectional prediction with coding unit weights can be applied on both AMVP mode and merge mode. This paper discusses the design principles and the implementation of the new inter-prediction methods. Using objective metrics, simulation results show that the methods overviewed in the paper can jointly achieve 6.2% and 4.7% BD-rate savings on average with the random access and low-delay configurations, respectively. Significant subjective picture quality improvements of some tools are also reported when comparing the resulting pictures at same bitrates.
Published in: IEEE Transactions on Circuits and Systems for Video Technology ( Volume: 31, Issue: 10, October 2021)
Citations are not available for this document.
Cites in Papers - |
Cites in Papers - IEEE (31)
Select All
1.
Yoshitaka Kidani, Haruhisa Kato, Kei Kawamura, "Chained Motion Vector Prediction for Video Coding", ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.1-5, 2025.
2.
Longtao Feng, Qian Yin, Siwei Ma, "Content-Adaptive Rate Control Method for User-Generated Content Videos", IEEE Transactions on Circuits and Systems for Video Technology, vol.35, no.3, pp.2806-2819, 2025.
3.
Chengzhuo Gui, Yuantong Zhang, Weijie Bao, Zhenzhong Chen, Huairui Wang, Shan Liu, "Deep Reference Frame for Versatile Video Coding with Structural Re-parameterization", 2024 IEEE International Conference on Visual Communications and Image Processing (VCIP), pp.1-5, 2024.
4.
Zhuoyuan Li, Yao Li, Chuanbo Tang, Li Li, Dong Liu, Feng Wu, "Uniformly Accelerated Motion Model for Inter Prediction", 2024 IEEE International Conference on Visual Communications and Image Processing (VCIP), pp.1-5, 2024.
5.
Philipp Merkle, Martin Winken, Jonathan Pfaff, Heiko Schwarz, Detlev Marpe, Thomas Wiegand, "Spatio-Temporal Convolutional Neural Network for Enhanced Inter Prediction in Video Coding", IEEE Transactions on Image Processing, vol.33, pp.4738-4752, 2024.
6.
Zhuoyuan Li, Zikun Yuan, Li Li, Dong Liu, Xiaohu Tang, Feng Wu, "Object Segmentation-Assisted Inter Prediction for Versatile Video Coding", IEEE Transactions on Broadcasting, vol.70, no.4, pp.1236-1253, 2024.
7.
Anastasia Henkel, Benjamin Bross, Jens Brandenburg, Adam Wieckowski, Detlev Marpe, Andoni Morales, Sergio Sanchez, "Optimizing an Open VVC Encoder for Low Delay Remote Desktop Applications", 2024 IEEE International Conference on Multimedia and Expo Workshops (ICMEW), pp.1-6, 2024.
8.
Rana Jassem, Taheni Damak, Mohamed Ali Ben Ayed, Nouri Masmoudi, "Gop Size Effect on VVC Encoder Performance with MRF Varation", 2024 IEEE 7th International Conference on Advanced Technologies, Signal and Image Processing (ATSIP), vol.1, pp.218-222, 2024.
9.
Xi Xie, Meng Wang, Junru Li, Kai Zhang, Li Zhang, Shiqi Wang, "Enhanced Motion Compensated Temporal Filter for VVenC", IEEE Transactions on Circuits and Systems for Video Technology, vol.34, no.11, pp.12086-12091, 2024.
10.
Fabian Brand, Jürgen Seiler, Johannes Sauer, Elena Alshina, André Kaup, "Adaptive Variance-Threshold-Based Skip Modes for Learned Video Compression Using a Motion Complexity Criterion", 2024 Picture Coding Symposium (PCS), pp.1-5, 2024.
11.
Sophie Pientka, Jonathan Pfaff, Heiko Schwarz, Detlev Marpe, Thomas Wiegand, "Deep Video Compression with Conditional Feature Coding", 2024 Picture Coding Symposium (PCS), pp.1-5, 2024.
12.
Yasmin Camargo, Matheus Isquierdo, Renira Soares, Daniel Palomino, Bruno Zatt, Felipe Sampaio, "Error-Resilience Profiling of Inter-Frame Prediction at VVC Encoders for Approximate Storage", 2024 IEEE 15th Latin America Symposium on Circuits and Systems (LASCAS), pp.1-5, 2024.
13.
Cunhui Dong, Haichuan Ma, Zhuoyuan Li, Li Li, Dong Liu, "Temporal Wavelet Transform-Based Low-Complexity Perceptual Quality Enhancement of Compressed Video", IEEE Transactions on Circuits and Systems for Video Technology, vol.34, no.5, pp.4040-4053, 2024.
14.
Chuanmin Jia, Feng Ye, Fanke Dong, Kai Lin, Leonardo Chiariglione, Siwei Ma, Huifang Sun, Wen Gao, "MPAI-EEV: Standardization Efforts of Artificial Intelligence Based End-to-End Video Coding", IEEE Transactions on Circuits and Systems for Video Technology, vol.34, no.5, pp.3096-3110, 2024.
15.
Mingyi Yang, Junyan Huo, Xile Zhou, Wenhan Qiao, Shuai Wan, Hao Wang, Fuzheng Yang, "Joint Rate-Distortion Optimization for Video Coding and Learning-Based In-Loop Filtering", IEEE Transactions on Multimedia, vol.26, pp.2851-2865, 2024.
16.
Wenhao Wang, Shiyu Jin, Yanfu Jiang, Ling Tian, Yimin Zhou, "Encoding Block Pixel Compensation With Motion Vector on DNN", 2023 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC), pp.234-238, 2023.
17.
Rana Jassem, Taheni Damak, Mohamed Ali Ben Ayed, Nouri Masmoudi, "Quantization effect on MRF performance for VVC encoder", 2023 IEEE International Conference on Design, Test and Technology of Integrated Systems (DTTIS), pp.1-4, 2023.
18.
Andy Regensky, Christian Herglotz, André Kaup, "Multi-Model Motion Prediction for 360-Degree Video Compression", IEEE Access, vol.11, pp.117004-117017, 2023.
19.
Sophie Pientka, Michael Schäfer, Jonathan Pfaff, Heiko Schwarz, Detlev Marpe, Thomas Wiegand, "Block-Based Motion Estimation for Deep-Learned Video Coding", 2023 IEEE International Conference on Image Processing (ICIP), pp.3444-3448, 2023.
20.
Quang Nhat Tran, Shih-Hsuan Yang, "Attention-Based Inter-Prediction for Versatile Video Coding", IEEE Access, vol.11, pp.84313-84322, 2023.
21.
Ru-Ling Liao, Yan Ye, Jie Chen, Xinwei Li, "An Improvement to Merge Mode in ECM With Template Matching", 2023 Data Compression Conference (DCC), pp.32-41, 2023.
22.
Xizhong Zhu, Guoqing Xiang, Xiaofeng Huang, Yunyao Yan, Huizhu Jia, Xiaodong Xie, "A Hardware-friendly CTU-level IME Algorithm for VVC", 2023 Data Compression Conference (DCC), pp.110-119, 2023.
23.
Sophie Pientka, Michael Schäfer, Jonathan Pfaff, Heiko Schwarz, Detlev Marpe, Thomas Wiegand, "Deep video coding with gradient-descent optimized motion compensation and Lanczos filtering", 2022 Picture Coding Symposium (PCS), pp.169-173, 2022.
24.
Chen Wang, Xin Jin, Huanbang Chen, Haitao Yang, Kedeng Tong, Yuqing Yang, "Piecewise Linear Model Based Local Illumination Compensation Inter Prediction for Video Coding", 2022 Picture Coding Symposium (PCS), pp.385-389, 2022.
25.
Robert Skupin, Yago Sánchez, Cornelius Hellge, Thomas Schierl, Moncef Gabbouj, "Region-Of-Interest Coding Schemes For Http Adaptive Streaming With VVC", 2022 IEEE International Conference on Image Processing (ICIP), pp.3196-3200, 2022.
26.
Philipp Merkle, Martin Winken, Jonathan Pfaff, Heiko Schwarz, Detlev Marpe, Thomas Wiegand, "Intra-Inter Prediction for Versatile Video Coding Using a Residual Convolutional Neural Network", 2022 IEEE International Conference on Image Processing (ICIP), pp.1711-1715, 2022.
27.
Benjamin Bross, Ye-Kui Wang, Yan Ye, Shan Liu, Jianle Chen, Gary J. Sullivan, Jens-Rainer Ohm, "Overview of the Versatile Video Coding (VVC) Standard and its Applications", IEEE Transactions on Circuits and Systems for Video Technology, vol.31, no.10, pp.3736-3764, 2021.
28.
Haitao Yang, Huanbang Chen, Jianle Chen, Semih Esenlik, Sriram Sethuraman, Xiaoyu Xiu, Elena Alshina, Jiancong Luo, "Subblock-Based Motion Derivation and Inter Prediction Refinement in the Versatile Video Coding Standard", IEEE Transactions on Circuits and Systems for Video Technology, vol.31, no.10, pp.3862-3877, 2021.
29.
Robert Skupin, Christian Bartnik, Adam Wieckowski, Yago Sanchez, Benjamin Bross, Cornelius Hellge, Thomas Schierl, "Open GOP Resolution Switching in HTTP Adaptive Streaming with VVC", 2021 Picture Coding Symposium (PCS), pp.1-5, 2021.
30.
Tung Nguyen, Xiaozhong Xu, Felix Henry, Ru-Ling Liao, Mohammed Golam Sarwer, Marta Karczewicz, Yung-Hsuan Chao, Jizheng Xu, Shan Liu, Detlev Marpe, Gary J. Sullivan, "Overview of the Screen Content Support in VVC: Applications, Coding Tools, and Performance", IEEE Transactions on Circuits and Systems for Video Technology, vol.31, no.10, pp.3801-3817, 2021.
Cites in Papers - Other Publishers (13)
1.
Seori Park, Hyunki Jung, Bumyoon Kim, Byeungwoo Jeon, "Weighted pairwise MV candidate addition with zero motion vector reduction for merge candidate", International Workshop on Advanced Imaging Technology (IWAIT) 2024, pp.97, 2024.
2.
Alberto González-Ruiz, Antonio Jesús Díaz-Honrubia, Santiago Tapia-Fernández, David García-Lucas, Gabriel Cebrián-Márquez, Luis Mengual-Galán, "Adaptive quadtree splitting parallelization (AQSP) algorithm for the VVC standard", The Journal of Supercomputing, 2024.
3.
Mingkai Chen, Xinmeng Tan, Huiyan Han, Lei Wang, "Tactile Codec with Visual Assistance in Multi-modal Communication for Digital Health", Mobile Networks and Applications, 2024.
4.
Yoshitaka Kidani, Kyohei Unno, Kei Kawamura, "Development of 4K/8K Real-time Codec Compliant with H.266 | VVC", The Journal of The Institute of Image Information and Television Engineers, vol.78, no.1, pp.115, 2024.
5.
Young-Ju Choi, Young-Woon Lee, Jongho Kim, Se Yoon Jeong, Jin Soo Choi, Byung-Gyu Kim, "Attention-Based Bi-Prediction Network for Versatile Video Coding (VVC) over 5G Network", Sensors, vol.23, no.5, pp.2631, 2023.
6.
Dohyeon Park , Gihwa Moon , Byung Tae Oh , Jae-Gon Kim , " Coarse-to-Fine Network-Based Intra Prediction in Versatile Video Coding ", Sensors , vol. 23 , no. 23 , pp. 9452 , 2023 .
7.
Rana Jassem, Taheni Damak, Mohamed Ali Ben Ayed, Nouri Masmoudi, "Inter prediction multiple reference frames impact on H266-VVC encoder", Multimedia Tools and Applications, 2023.
8.
T Nithyoosha, Prayline Rajabai Christopher, "A survey on Motion Estimation and De-hazing Algorithms and Architectures", Digital Signal Processing, pp.104130, 2023.
9.
Su-Yeon Lim, Min-Kyeong Choi, Yung-Lyul Lee, "Frequency-Based Adaptive Interpolation Filter in Intra Prediction", Applied Sciences, vol.13, no.3, pp.1475, 2023.
10.
Xiao Xu, Hao Yan, Xiaolei Wang, "Research on Postproduction of Film and Television Based on Computer Multimedia Technology", Scientific Programming, vol.2022, pp.1, 2022.
11.
Yoshitaka KIDANI, Haruhisa KATO, Kei KAWAMURA, Hiroshi WATANABE, "Geometric Partitioning Mode with Inter and Intra Prediction for Beyond Versatile Video Coding", IEICE Transactions on Information and Systems, vol.E105.D, no.10, pp.1691, 2022.
12.
Mario Saldanha, Gustavo Sanchez, Cesar Marcon, Luciano Agostini, "Versatile Video Coding (VVC)", Versatile Video Coding (VVC), pp.7, 2022.
13.
Karim El Khoury, Jonathan Samelson, Benoit Macq, "Deep Learning-Based Object Tracking via Compressed Domain Residual Frames", Frontiers in Signal Processing, vol.1, 2021.