Spectral Sensitivity Estimation Without a Camera

A number of problems in computer vision and related fields would be mitigated if camera spectral sensitivities were known. As consumer cameras are not designed for high-precision visual tasks, manufacturers do not disclose spectral sensitivities. Their estimation requires a costly optical setup, which triggered researchers to come up with numerous indirect methods that aim to lower cost and complexity by using color targets. However, the use of color targets gives rise to new complications that make the estimation more difficult, and consequently, there currently exists no simple, low-cost, robust go-to method for spectral sensitivity estimation that non-specialized research labs can adopt. Furthermore, even if not limited by hardware or cost, researchers frequently work with imagery from multiple cameras that they do not have in their possession. To provide a practical solution to this problem, we propose a framework for spectral sensitivity estimation that not only does not require any hardware (including a color target), but also does not require physical access to the camera itself. Similar to other work, we formulate an optimization problem that minimizes a two-term objective function: a camera-specific term from a system of equations, and a universal term that bounds the solution space. Different than other work, we utilize publicly available high-quality calibration data to construct both terms. We use the colorimetric mapping matrices provided by the Adobe DNG Converter to formulate the camera-specific system of equations, and constrain the solutions using an autoencoder trained on a database of ground-truth curves. On average, we achieve reconstruction errors as low as those that can arise due to manufacturing imperfections between two copies of the same camera. We provide our code and predicted sensitivities for 1, 000+ cameras at https://github.com/COLOR-Lab-Eilat/Spectral-sensitivity-estimation, and discuss which tasks can become trivial when camera responses are available.

1 Introduction

Consumer digital cameras (i.e., point-and-shoot, DSLR, mirrorless cameras and smartphones) are not tools designed for scientific imaging, i.e., they are not scientific light-measuring devices [ 1 – 3 ]. Yet their outputs—digital photos and videos—constitute major sources of data for image processing, colorimetry, computational photography, computer vision, and machine learning. Typically, research utilizing consumer camera imagery focuses on the development of filters or algorithms that alter the visual appearance of images [ 4 – 7 ], or on the understanding of scene content [ 8 – 10 ] and structure [11 , 12] with downstream goals like recognizing, tracking, or counting objects. For many of these goals, successfully recovering scene reflectance and/or illumination is key (and often the main goal itself), but these tasks are complicated by the fact that consumer cameras do not capture colors in a standardized way [13 , 14] .