Cartoon Animation Outpainting With Region-Guided Motion Inference

Cartoon animation video is a popular visual entertainment form worldwide, however many classic animations were produced in a 4:3 aspect ratio that is incompatible with modern widescreen displays. Existing methods like cropping lead to information loss while retargeting causes distortion. Animation companies still rely on manual labor to renovate classic cartoon animations, which is tedious and labor-intensive, but can yield higher-quality videos. Conventional extrapolation or inpainting methods tailored for natural videos struggle with cartoon animations due to the lack of textures in anime, which affects the motion estimation of the objects. In this article, we propose a novel framework designed to automatically outpaint 4:3 anime to 16:9 via region-guided motion inference. Our core concept is to identify the motion correspondences between frames within a sequence in order to reconstruct missing pixels. Initially, we estimate optical flow guided by region information to address challenges posed by exaggerated movements and solid-color regions in cartoon animations. Subsequently, frames are stitched to produce a pre-filled guide frame, offering structural clues for the extension of optical flow maps. Finally, a voting and fusion scheme utilizes learned fusion weights to blend the aligned neighboring reference frames, resulting in the final outpainting frame. Extensive experiments confirm the superiority of our approach over existing methods.

I. Introduction

Anime is a worldwide popular visual entertainment and art form with significant market demand. Though many new titles have been created, there are still a large amount of legacy and classic animations popular and enjoyed by audiences. However, the old cartoon animations were usually produced in 4:3 aspect ratio resolutions, which do not match the commonly used 16:9 aspect ratio or even wider screens at present. To display a classic 4:3 cartoon animation video on a wider screen, the animation is typically placed at the center of the screen, with two wide black stripes on the left and right sides, respectively(Fig. 1(a)). This significantly harms the visual experiences of the audience.