Reparameterization for Improved Training and Weight Optimization in Single Image Super-Resolution Networks

In the field of Single Image Super-Resolution, convolutional neural networks have demonstrated notable advancements. While the integration of batch normalization, dropout, and other layers, which were previously detrimental, has led to larger, more proficient models, the efficacy of extant training pipelines for Super-Resolution models is still inadequate, even for smaller networks. We propose that the state-of-the-art training procedure is insufficient in identifying optimal weights for even a single convolutional layer. Our research reveals that the weights generated by the baseline training pipeline for all trained models conform to a specific space. We utilize the reparameterization technique to derive improved weights outside this designated space, leading to significantly enhanced performance and stability in prominent models, particularly in large kernel convolutions. We further highlight the importance of reparameterization in SR models and advocate its further exploration in future research.