Video-Rate Panorama for Free-Viewpoint TV

Free Viewpoint Video and TV are technologies regarded as the future of digital entertainment allowing users to navigate through multiple video streams of an event to select novel viewpoints. In this paper, we propose an automatic real-time video stitching algorithm that is capable of producing spherical panoramas for FTV applications. The algorithm works by splitting automatic image stitching into one-time and recurring computations. We utilize GPUs for accelerating the recurring computations and achieving real-time performance. By effectively caching the repeated transform maps, we improve the stitching performance by two orders of magnitude compare to CPU implementations. The multiple stitching steps are pipelined to distribute the panorama stitching computations between multiple processing nodes to produce visually appealing panoramas at 30Hz without any noticeable blurring or stitching artifacts for five HD synchronized cameras.