1. Introduction
360° video gives a viewer immersive experiences through displaying full surroundings of a camera in a spherical canvas, which differentiates itself from traditional multimedia. As consumer- and production-grade 360° cameras become readily available, 360° videos are captured every minute. Moreover, the promotion of 360° videos by social media giants including YouTube and Facebook further boosts their fast adoption. It is expected that 360° videos will become a major video format in the near future. Studying how to display 360° videos to a human viewer, who has a limited field of visual attention, emerges as an increasingly important problem.
Panel (a) Overlaps three panoramic frames sampled from a 360° skateboarding video with two skateboarders. One skateboarder is more active than the other in this example. For each frame, the proposed “deep 360 pilot” selects a view — a viewing angle, where a natural field of view (NFoV) (cyan box) is centered at. It first extracts candidate objects (yellow boxes), and then selects a main object (green dash boxes) in order to determine a view (just like a human agent). Panel (b) Shows the NFoV from a viewer's perspective.