Colorimetric Sensor Reading and Illumination Correction via Multi-Task Deep-Learning

Colorimetric sensors represent an accessible and sensitive nanotechnology for rapid and accessible measurement of a substance’s properties (e.g., analyte concentration) via color changes. Although colorimetric sensors are widely used in healthcare and laboratories, interpretation of their output is performed either by visual inspection or using cameras in highly controlled illumination set-ups, limiting their usage in end-user applications, with lower resolutions and altered light conditions. For that purpose, we implement a set of image processing and deep-learning (DL) methods that correct for non-uniform illumination alterations and accurately read the target variable from the color response of the sensor. Methods that perform both tasks independently vs. jointly in a multi-task model are evaluated. Video recordings of colorimetric sensors measuring temperature conditions were collected to build an experimental reference dataset. Sensor images were augmented with non-uniform color alterations. The best-performing DL architecture disentangles the luminance, chrominance, and noise via separate decoders and integrates a regression task in the latent space to predict the sensor readings, achieving a mean squared error (MSE) performance of 0.811±0.074[°C] and r
²
=0.930±0.007, under strong color perturbations, resulting in an improvement of 1.26[°C] when compared to the MSE of the best performing method with independent denoising and regression tasks.Clinical Relevance— The proposed methodology aims to improve the accuracy of colorimetric sensor reading and their large-scale accessibility as point-of-care diagnostic and continuous health monitoring devices, in altered illumination conditions.