Hybrid Decomposition Network for Light Field Super-Resolution

abstract-Light field cameras are considered emerging 3D world sensing instruments capturing dense images. High-resolution images would provide richer semantic cues than low-resolution ones to facilitate various computer vision-based tasks. The use of spatial-angular information plays a key role in light field image super-resolution (LFSR), which is an effective way to increase image resolution. A considerable amount of recent works have employed Transformer-based models in LFSR tasks and achieved performance improvement. However, most of them enforced the self-attention mechanism equally on both the light field’s spatial and angular domains, without taking into account the distinct characteristics of each domain. On the one hand, the angular inter-view correlation tends to be low-frequency, where self-attention works more effectively as a low-pass filter. On the other hand, the spatial intra-view information is of high-frequency, which is more suitable for being processed by convolution layers as high-pass filters. With this insight, we propose a Hybrid Decomposition Network for LFSR (LF-HDNet) that combines convolution and self-attention but performs them separately on the two domains to extract spatial-angular features for LFSR in this paper. Experimental results demonstrate the superior performance of our method over the state-of-the-art methods in LFSR tasks.