Color-SD: Stable Diffusion Model Already has a Color Style Noisy Latent Space

We present Color-SD, a comprehensive color style transfer framework that utilizes either image or text references. Built on the pretrained Stable Diffusion Model, Color-SD exploits an existing color style space, enabling a training-free and tuning-free zero-shot color style transfer method without introducing new parameters. For image references, we first invert the source and reference images to the noisy latent space, followed by parallel sampling. During this process, we execute distribution transformation in the noisy latent space, effectively completing the color style transfer and generating the stylized result. For text references, we capitalize on the Stable Diffusion model’s inherent text-to-image capability. We only invert the source image to the noisy latent, and the given text reference prompt is utilized during the parallel sampling. This approach eliminates the need for training or tuning, yet produces impressive open-set transfer results. Comprehensive experiments validate the effectiveness of our method, demonstrating significant superiority over existing methods in both qualitative and quantitative evaluations.

1. INTRODUCTION

There is a growing demand among users to tailor the color style of their photos and videos. However, the predefined filters and LUTs [1], [2] might not align with the heterogeneous aesthetic preferences of all users, potentially constraining user adaptability. Image-reference color style transfer represents one of the prevalent tasks in color style transfer. It involves aligning the resulting image with the reference image in the context of global color style coherence, while retaining the original image’s content and texture fidelity. Nevertheless, several contemporary approaches remain insufficient in meeting the demands of this task.