Detailed and Controllable Old Photo Restoration with Diffusion Priors

Restoring old photos that contain numerous unknown and complex defects is a challenging and ill-posed problem. Traditional methods often struggle to address both structured and unstructured defects in real old photos, frequently leading to over-smoothed and uncompleted results. In this paper, we exploit powerful diffusion priors to construct a novel solution for the restoration of old photos. Our framework begins with a coarse stage model aimed at eliminating unstructured defects, followed by a conditional diffusion model that further refines the content and enriches details. Specifically, we introduce an edge control module, designed to encode the restored edge map into the denoising network. This integration effectively guides the restoration process, allowing the edge conditional diffusion to achieve more precise and controllable results. Additionally, we incorporate a feature fusion VAE to ensure the fidelity of the final outputs. Through extensive qualitative, quantitative, and ablative experiments, we demonstrate the innovation and effectiveness of the proposed method, which offers a complete, detailed, and personalized restoration of old photos.

I. Introduction

Old photo restoration involves a collection of processes including denoising, deblurring, inpainting, and tonal adjustment. This task is complex and inherently ill-posed due to the variety of defects that old photos may suffer. Defects in old photos can be broadly categorized into two types: structured defects, such as scratches, mold spots, creases, and stains, and unstructured defects, such as noise, fading, and blurring. To address these challenges, image inpainting and image restoration techniques have been developed to deal with structured and unstructured defects respectively.