Adaptive Filtering for Video Coding with Focus Change

We propose an adaptive filtering approach for coding video that exhibits localized camera focus changes, e.g., such that different portions of a video frame can undergo different blurriness/sharpness changes with respect to corresponding areas in frames used for prediction. First we obtain, for each macroblock, the filter that provides maximum reduction in residual energy, when applied to the reference macroblock before to motion compensation. Then, starting from the block-wise filter parameters, we divide the macroblocks into classes, by clustering macroblocks that have been assigned "similar" filters. Finally, for each class, a two-dimensional filter is designed to minimize the average residual energy for all macroblocks in the class. The resulting filters are applied to the reference frames to generate better matches for motion compensation. Simulation results shows that the proposed method provides up to 1 dB gain over current H.264 for certain sequences. As compared to H.264 with multiple reference frames, the coding gain is about 0.5 dB.

1. INTRODUCTION

In most current video coding standards, hybrid coding techniques with block-based motion compensation and transform coding are employed. Motion compensation is applied to exploit temporal redundancy from frame to frame while the spatial redundancy within a frame is exploited by transform coding. When the objects in the video scene undergo only displacement with no other transformation, motion compensation will provides excellent coding efficiency, which can be further improved when variable block sizes and multiple reference frames are used, as in H.264 [1]. However, in situations where the pure displacement assumption is violated, the coding efficiency gains provided by motion compensation can be significantly lower.