Loading [MathJax]/extensions/MathMenu.js
Deep SR-ITM: Joint Learning of Super-Resolution and Inverse Tone-Mapping for 4K UHD HDR Applications | IEEE Conference Publication | IEEE Xplore
Access provided by:

MIT Libraries

Sign Out
Access provided by:

MIT Libraries

Sign Out
ADVANCED SEARCH
Conferences >2019 IEEE/CVF International C...

Deep SR-ITM: Joint Learning of Super-Resolution and Inverse Tone-Mapping for 4K UHD HDR Applications

PDF
Soo Ye Kim; Jihyong Oh; Munchurl Kim
All Authors

Abstract:

Recent modern displays are now able to render high dynamic range (HDR), high resolution (HR) videos of up to 8K UHD (Ultra High Definition). Consequently, UHD HDR broadca...Show More

Metadata

Abstract:

Recent modern displays are now able to render high dynamic range (HDR), high resolution (HR) videos of up to 8K UHD (Ultra High Definition). Consequently, UHD HDR broadcasting and streaming have emerged as high quality premium services. However, due to the lack of original UHD HDR video content, appropriate conversion technologies are urgently needed to transform the legacy low resolution (LR) standard dynamic range (SDR) videos into UHD HDR versions. In this paper, we propose a joint super-resolution (SR) and inverse tone-mapping (ITM) framework, called Deep SR-ITM, which learns the direct mapping from LR SDR video to their HR HDR version. Joint SR and ITM is an intricate task, where high frequency details must be restored for SR, jointly with the local contrast, for ITM. Our network is able to restore fine details by decomposing the input image and focusing on the separate base (low frequency) and detail (high frequency) layers. Moreover, the proposed modulation blocks apply location-variant operations to enhance local contrast. The Deep SR-ITM shows good subjective quality with increased contrast and details, outperforming the previous joint SR-ITM method.
Published in: 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
Date of Conference: 27 October 2019 - 02 November 2019
Date Added to IEEE Xplore: 27 February 2020
ISBN Information:

ISSN Information:

DOI: 10.1109/ICCV.2019.00321
Conference Location: Seoul, Korea (South)
Citations are not available for this document.
Contents

1. Introduction

Modern TVs come with 4K/8K UHD (Ultra High Definition) displays and high dynamic range (HDR) capabilities. Nevertheless, the current Digital TV and Internet TV (IPTV) services still provide the legacy video contents of Full HD (FHD) resolution and standard dynamic range (SDR), which then, must be rendered on the premium TV displays that support 4K/8K UHD and HDR videos. Therefore at the terminal end, it is necessary to convert the FHD SDR videos to 4K/8K UHD HDR in order to display them on the premium displays. Furthermore, the new media services of high quality suffer from the lack of original 4K/8K UHD and HDR visual content. Thus, it is also essential to convert the legacy contents of Full HD and SDR video to 4K/8K UHD and HDR videos at the content production end.

Cites in Papers - |

Cites in Papers - IEEE (47)

Select All
1.
Jianru Zhao, Hua Zhang, "LANet: Luminance-Aware Network for SDRTV-to-HDRTV Translation", 2024 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp.2354-2359, 2024.
Show Article
Google Scholar
2.
Yuyao Ye, Ning Zhang, Yang Zhao, Hongbin Cao, Ronggang Wang, "Deep Video Inverse Tone Mapping Based on Temporal Clues", 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.25995-26004, 2024.
Show Article
Google Scholar
3.
Fei Zhou, Zikang Zheng, Guoping Qiu, "Removing Banding Artifacts in HDR Videos Generated From Inverse Tone Mapping", IEEE Transactions on Broadcasting, vol.70, no.2, pp.753-762, 2024.
Show Article
Google Scholar
4.
Hengsheng Zhang, Xinning Chai, Yuhong Zhang, Rong Xie, Li Song, "Hdrtvformer: Efficient Sdrtv-to-Hdrtv via Affine Transformation and Spatial-Aware Transformer", ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.2785-2789, 2024.
Show Article
Google Scholar
5.
Congrui Fu, Hui Yuan, Liquan Shen, Raouf Hamzaoui, Hao Zhang, "3DAttGAN: A 3D Attention-Based Generative Adversarial Network for Joint Space-Time Video Super-Resolution", IEEE Transactions on Emerging Topics in Computational Intelligence, vol.8, no.4, pp.3117-3128, 2024.
Show Article
Google Scholar
6.
Hengsheng Zhang, Xueyi Zou, Guo Lu, Li Chen, Li Song, Wenjun Zhang, "EffiHDR: An Efficient Framework for HDRTV Reconstruction and Enhancement in UHD Systems", IEEE Transactions on Broadcasting, vol.70, no.2, pp.620-636, 2024.
Show Article
Google Scholar
7.
Shreshth Saini, Avinab Saha, Alan C. Bovik, "HIDRO-VQA: High Dynamic Range Oracle for Video Quality Assessment", 2024 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW), pp.469-479, 2024.
Show Article
Google Scholar
8.
Hu Wang, Mao Ye, Xiatian Zhu, Shuai Li, Xue Li, Ce Zhu, "Compressed-SDR to HDR Video Reconstruction", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.46, no.5, pp.3679-3691, 2024.
Show Article
Google Scholar
9.
Zhiyong Huo, Jian Yu, "Reconstructing HDR Image from a Single Exposure LDR Image with Global Feature Fusion", 2023 16th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), pp.1-6, 2023.
Show Article
Google Scholar
10.
Chengzhi Luo, Bing Li, Feng Liu, "Iterative Back Projection Network Based on Deformable 3D Convolution", IEEE Access, vol.11, pp.122586-122597, 2023.
Show Article
Google Scholar
11.
Joonsoo Kim, Kamal Jnawali, Chenguang Liu, "Efficient-HDRTV: Efficient SDR to HDR Conversion for HDR TV", 2023 IEEE International Conference on Image Processing (ICIP), pp.1065-1069, 2023.
Show Article
Google Scholar
12.
Mingde Yao, Jie Huang, Xin Jin, Ruikang Xu, Shenglong Zhou, Man Zhou, Zhiwei Xiong, "Generalized Lightness Adaptation with Channel Selective Normalization", 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pp.10634-10645, 2023.
Show Article
Google Scholar
13.
Chao Wang, Ana Serrano, Xingang Pan, Bin Chen, Karol Myszkowski, Hans-Peter Seidel, Christian Theobalt, Thomas Leimkühler, "GlowGAN: Unsupervised Learning of HDR Images from LDR Images in the Wild", 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pp.10475-10485, 2023.
Show Article
Google Scholar
14.
Yunhao Zou, Chenggang Yan, Ying Fu, "RawHDR: High Dynamic Range Image Reconstruction from a Single Raw Image", 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pp.12300-12310, 2023.
Show Article
Google Scholar
15.
Chao Xu, Huamin Yang, Cheng Han, Chao Zhang, "Single Image HDR Reconstruction via A Feature Modulation Network", 2023 4th International Conference on Intelligent Computing and Human-Computer Interaction (ICHCI), pp.194-198, 2023.
Show Article
Google Scholar
16.
Haesoo Chung, Nam Ik Cho, "Pyramidal Feature Adjustment Networks for High Dynamic Range Imaging of Dynamic Scenes", IEEE Access, vol.11, pp.66882-66894, 2023.
Show Article
Google Scholar
17.
Cheng Guo, Leidong Fan, Ziyu Xue, Xiuhua Jiang, "Learning a Practical SDR-to-HDRTV Up-conversion using New Dataset and Degradation Models", 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.22231-22241, 2023.
Show Article
Google Scholar
18.
Hong-Xing Yu, Samir Agarwala, Charles Herrmann, Richard Szeliski, Noah Snavely, Jiajun Wu, Deqing Sun, "Accidental Light Probes", 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.12521-12530, 2023.
Show Article
Google Scholar
19.
Eduard Zamfir, Marcos V. Conde, Radu Timofte, "Towards Real-Time 4K Image Super-Resolution", 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp.1522-1532, 2023.
Show Article
Google Scholar
20.
Peilin Chen, Wenhan Yang, Shiqi Wang, "Reviving Standard-Dynamic-Range Videos for High-Dynamic-Range Devices: A Learning Paradigm With Hybrid Attention Mechanisms", IEEE MultiMedia, vol.30, no.3, pp.110-118, 2023.
Show Article
Google Scholar
21.
Mingde Yao, Dongliang He, Xin Li, Zhihong Pan, Zhiwei Xiong, "Bidirectional Translation Between UHD-HDR and HD-SDR Videos", IEEE Transactions on Multimedia, vol.25, pp.8672-8686, 2023.
Show Article
Google Scholar
22.
Xiangyu Hu, Liquan Shen, Mingxing Jiang, Ran Ma, Ping An, "LA-HDR: Light Adaptive HDR Reconstruction Framework for Single LDR Image Considering Varied Light Conditions", IEEE Transactions on Multimedia, vol.25, pp.4814-4829, 2023.
Show Article
Google Scholar
23.
Xiao Tan, Huaian Chen, Kai Xu, Yi Jin, Changan Zhu, "Deep SR-HDR: Joint Learning of Super-Resolution and High Dynamic Range Imaging for Dynamic Scenes", IEEE Transactions on Multimedia, vol.25, pp.750-763, 2023.
Show Article
Google Scholar
24.
Joon-Ki Bae, Subin Yang, Sung-Ho Bae, "DenSE SwinHDR: SDRTV to HDRTV Conversion Using Densely Connected Swin Transformer With Squeeze and Excitation Module", IEEE Access, vol.10, pp.133969-133980, 2022.
Show Article
Google Scholar
25.
Rui Tang, Fang Meng, Lefu Bai, "Zoned Mapping Network from SDR Video to HDR Video", 2022 IEEE 5th Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC), vol.5, pp.1466-1472, 2022.
Show Article
Google Scholar
26.
Caiyong Wang, Tianhao Lu, Gaosheng Wu, Yunlong Wang, Zhenan Sun, "D-ESRGAN: A Dual-Encoder GAN with Residual CNN and Vision Transformer for Iris Image Super-Resolution", 2022 IEEE International Joint Conference on Biometrics (IJCB), pp.1-8, 2022.
Show Article
Google Scholar
27.
Zeeshan Khan, Parth Shettiwar, Mukul Khanna, Shanmuganathan Raman, "DeepHS-HDRVideo: Deep High Speed High Dynamic Range Video Reconstruction", 2022 26th International Conference on Pattern Recognition (ICPR), pp.1959-1966, 2022.
Show Article
Google Scholar
28.
Ngoc Long Nguyen, Jérémy Anger, Axel Davy, Pablo Arias, Gabriele Facciolo, "Self-Supervised Super-Resolution for Multi-Exposure Push-Frame Satellites", 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.1848-1858, 2022.
Show Article
Google Scholar
29.
Y. Wang, H. R. Tohidypour, M. T. Pourazad, P. Nasiopoulos, V. C.M. Leung, "Deep Learning-Based HDR Image Upscaling Approach for 8K UHD Applications", 2022 IEEE International Conference on Consumer Electronics (ICCE), pp.1-2, 2022.
Show Article
Google Scholar
30.
Kai Xu, Huaian Chen, Chunmei Xu, Yi Jin, Changan Zhu, "Structure-Texture Aware Network for Low-Light Image Enhancement", IEEE Transactions on Circuits and Systems for Video Technology, vol.32, no.8, pp.4983-4996, 2022.
Show Article
Google Scholar

Cites in Papers - Other Publishers (22)

1.
Zheng Wang, Gang He, "Multi-Frame Content-Aware Mapping Network for Standard-Dynamic-Range to High-Dynamic-Range Television Artifact Removal", Sensors, vol.24, no.1, pp.299, 2024.
CrossRef Google Scholar
2.
Huei-Yung Lin, Yi-Rung Lin, Wen-Chieh Lin, Chin-Chen Chang, "Reconstructing High Dynamic Range Image from a Single Low Dynamic Range Image Using Histogram Learning", Applied Sciences, vol.14, no.21, pp.9847, 2024.
CrossRef Google Scholar
3.
Gonzalo Luzardo, Asli Kumcu, Jan Aelterman, Hiep Luong, Daniel Ochoa, Wilfried Philips, "A Display-Adaptive Pipeline for Dynamic Range Expansion of Standard Dynamic Range Video Content", Applied Sciences, vol.14, no.10, pp.4081, 2024.
CrossRef Google Scholar
4.
Liqi Xue, Tianyi Xu, Yongbao Song, Yan Liu, Lei Zhang, Xiantong Zhen, Jun Xu, "Lightweight improved residual network for efficient inverse tone mapping", Multimedia Tools and Applications, 2024.
CrossRef Google Scholar
5.
Xiaonan He, Yukun Xia, Yuansong Qiao, Brian Lee, Yuhang Ye, "An Efficient and\xa0Lightweight Structure for\xa0Spatial-Temporal Feature Extraction in\xa0Video Super Resolution", Advances in Computer Graphics, vol.14495, pp.362, 2024.
CrossRef Google Scholar
6.
Gang Xu, Qibin Hou, Ming-Ming Cheng, "Dual Frequency Transformer for Efficient SDR-to-HDR Translation", Machine Intelligence Research, 2024.
CrossRef Google Scholar
7.
Yi-Rung Lin , Huei-Yung Lin , Wen-Chieh Lin , " Single Image HDR Synthesis with\xa0Histogram Learning ", Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications , vol. 14469 , pp. 108 , 2024 .
CrossRef Google Scholar
8.
Leidong Fan, Ao Yang, Cheng Guo, Xiuhua Jiang, Yixuan Gao, Xiongkuo Min, "Inverse-tone-mapped HDR video quality assessment: A new dataset and benchmark", Displays, pp.102510, 2023.
CrossRef Google Scholar
9.
Wenqi Ren, Senyou Deng, Kaihao Zhang, Fenglong Song, Xiaochun Cao, Ming-Hsuan Yang, "Fast Ultra High-Definition Video Deblurring via Multi-scale Separable Network", International Journal of Computer Vision, 2023.
CrossRef Google Scholar
10.
Zhengnan Ye , Haisong Xu , Yiming Huang , Yuemin Li , Minhang Yang , " Constructing high-dynamic-range spectral images for natural scenes ", Optical Engineering , vol. 62 , no. 11 , 2023 .
CrossRef Google Scholar
11.
Gang He, Shaoyi Long, Li Xu, Chang Wu, Wenxin Yu, Jinjia Zhou, "Global priors guided modulation network for joint super-resolution and SDRTV-to-HDRTV", Neurocomputing, pp.126590, 2023.
CrossRef Google Scholar
12.
Guoping Li, Zhenting Zhou, Xiwu Shang, Guozhong Wang, "Super-resolution network with dynamic cleanup and temporal?spatial attention for compressed videos", Engineering Applications of Artificial Intelligence, vol.123, pp.106216, 2023.
CrossRef Google Scholar
13.
Kai-Fu Yang, Cheng Cheng, Shi-Xuan Zhao, Hong-Mei Yan, Xian-Shi Zhang, Yong-Jie Li, "Learning to Adapt to Light", International Journal of Computer Vision, 2023.
CrossRef Google Scholar
14.
Wei Sun, Xianguang Kong, Yanning Zhang, "Attention-guided video super-resolution with recurrent multi-scale spatial?temporal transformer", Complex & Intelligent Systems, 2022.
CrossRef Google Scholar
15.
Shupei Zhang, Kangkang Hu, Zhenkun Zhou, Anup Basu, "Lighting Enhancement Using Self-attention Guided HDR Reconstruction", Smart Multimedia, vol.13497, pp.409, 2022.
CrossRef Google Scholar
16.
Bruno Lecouat, Thomas Eboli, Jean Ponce, Julien Mairal, "High dynamic range and super-resolution from raw image bursts", ACM Transactions on Graphics, vol.41, no.4, pp.1, 2022.
CrossRef Google Scholar
17.
Zhen Cheng, Tao Wang, Yong Li, Fenglong Song, Chang Chen, Zhiwei Xiong, "Towards Real-World HDRTV Reconstruction: A Data Synthesis-Based Approach", Computer Vision ? ECCV 2022, vol.13679, pp.199, 2022.
CrossRef Google Scholar
18.
Yongqiang Wang, Xue Li, Fangzhe Nan, Feng Liu, Hua Li, Haitao Wang, Yurong Qian, "Image super-resolution reconstruction based on generative adversarial network model with feedback and attention mechanisms", Multimedia Tools and Applications, vol.81, no.5, pp.6633, 2022.
CrossRef Google Scholar
19.
Kejie Lyu, Sicheng Pan, Yingming Li, Zhongfei Zhang, "JSENet: A deep convolutional neural network for joint image super-resolution and enhancement", Neurocomputing, 2022.
CrossRef Google Scholar
20.
Fei Zhou, Guangsen Liao, Jiang Duan, Bozhi Liu, Guoping Qiu, "Tone mapping high dynamic range images based on region-adaptive self-supervised deep learning", Signal Processing: Image Communication, vol.102, pp.116595, 2022.
CrossRef Google Scholar
21.
Hanning Yu, Wentao Liu, Chengjiang Long, Bo Dong, Qin Zou, Chunxia Xiao, "Luminance Attentive Networks for HDR Image and Panorama Reconstruction", Computer Graphics Forum, vol.40, no.7, pp.181, 2021.
CrossRef Google Scholar
22.
Abdul Muqeet, Sung-Ho Bae, "Effective Utilization of Hybrid Residual Modules in Deep Neural Networks for Super Resolution", MultiMedia Modeling, vol.11962, pp.745, 2020.
CrossRef Google Scholar
Contact IEEE to Subscribe
More Like This
Proposal for Broadcast High Dynamic Range Content Transmission

2016 IEEE International Symposium on Consumer Electronics (ISCE)

Published: 2016

Temporal Aspect of Perceived Quality in Mobile Video Broadcasting

IEEE Transactions on Broadcasting

Published: 2008

Show More

References

References is not available for this document.

IEEE Account

Purchase Details

Profile Information

Need Help?

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity.
© Copyright 2025 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.