RA360SR: A Real-time Acceleration-adaptive 360-degree Video Super-resolution System

With the development of virtual reality headsets, panoramic videos have started to gain increased popularity, and 360-degree video streaming technologies and devices have achieved significant success in the market. However, the video resolution provided by existing mainstream panoramic video cameras is not high enough to give users a viewing experience close to that of a conventional display, and the network bandwidth is another bottleneck for high-resolution 360-degree video streaming. Researchers have introduced numerous approaches to resolve these issues, including viewport prediction and regional super-resolution. Nevertheless, existing methods can-not ensure that users always have high-resolution viewport content because of the inevitable prediction error and the heavy server-side pre-process consumption. In this paper, we present RA360SR, a real-time acceleration-adaptive 360-degree video super-resolution system. We develop a dual-camera system with Unity3D post-processing to implement the real-time super-resolution model processing. Additionally, to obtain a more stable frame rate, we propose an acceleration-adaptive approach to switch the super-resolution model processing status based on the acceleration of users' head movements. Our results show that RA360SR can deliver a sharper video viewing experience for users while providing an acceptable frame rate.