Hollowed-Out Icon Colorization with Controllable Diffusion Model

Icons are indispensable elements for websites and smartphone applications. In design support, some methods utilizing deep learning for the coloring of line-drawn icons have been proposed. Yet, coloring icons with hollow configurations, such as a donut shape, without unintended color spillage remains a challenge. To address this, we have integrated textual annotations to specify the desired coloring area. Specifically, we painted line drawings using a Stable Diffusion-based image generation model, Uni-ControlNet, which operates on multiple conditions to generate images. To facilitate the training of this model, we assembled a dataset comprising icons, along with their corresponding captions. We proposed a quantitative index to evaluate the similarity of gradients between reference images and colored line drawings. Our method was superior to existing methods in several evaluations. In the user study focused on evaluating coloring errors in icons with hollow structures, our method outperformed existing methods. Furthermore, it was shown to be able to color icons with hollow structures, which was difficult with existing methods.