RAW Image Reconstruction Using a Self-Contained sRGB-JPEG Image with Only 64 KB Overhead

Most camera images are saved as 8-bit standard RGB (sRGB) compressed JPEGs. Even when JPEG compression is set to its highest quality, the encoded sRGB image has been significantly processed in terms of color and tone manipulation. This makes sRGB-JPEG images undesirable for many computer vision tasks that assume a direct relationship between pixel values and incoming light. For such applications, the RAW image format is preferred, as RAW represents a minimally processed, sensor-specific RGB image with higher dynamic range that is linear with respect to scene radiance. The drawback with RAW images, however, is that they require large amounts of storage and are not well-supported by many imaging applications. To address this issue, we present a method to encode the necessary metadata within an sRGB image to reconstruct a high-quality RAW image. Our approach requires no calibration of the camera and can reconstruct the original RAW to within 0:3% error with only a 64 KB overhead for the additional data. More importantly, our output is a fully selfcontained 100% complainant sRGB-JPEG file that can be used as-is, not affecting any existing image workflow - the RAW image can be extracted when needed, or ignored otherwise. We detail our approach and show its effectiveness against competing strategies.

1. Introduction

The vast majority of images used in computer vision and image processing applications are 8-bit standard RGB (sRGB) images, typically saved using the JPEG compression standard. Virtually all image processing application workflows support sRGB and JPEG images. There are many drawbacks, however, when working with sRGB images, e.g. it is well known that sRGB images have a number of non-linear operations applied that makes it difficult to relate the sRGB values back to scene radiance [3], [8], [13], [17], [22], [24]. Figure 1.

(a) A resolution high-quality SRGB-JPEG with our metadata embedded (original JPEG size of 9,788 KB, new size of 9,852 KB). (b) Original RAW image is 25,947 KB. (c) Our reconstructed RAW image using the data in the self-contained JPEG. (d) Error map between (b) and (c). Overall reconstruction error is 0.2%.