Deep Learning Based Image Segmentation Directly in the JPEG Compressed Domain

Image segmentation is a widely explored research problem in the field of computer vision and image processing. Most of the existing techniques carry out segmentation in the pixel or uncompressed domain. More and more images and image-frames (videos) are archived and accessed in compressed form to provide better storage and transmission efficiency in recent years. As a result, the amount of compressed data has drastically increased and warrants novel techniques for automatically processing compressed data in the compressed domain without decompressing any data. Therefore here in this research paper we propose a novel idea of performing the segmentation straight from the JPEG compressed data by feeding the compressed DCT coefficients into the deep neural network. The proposed model is constructed by modifying an existing model Eff-UNet, where the first 3 layers are removed. The segmentation performance is improved and achieved state-of-the-art accuracy when the proposed model is tested on two benchmark datasets Oxford-IIIT and IDD-lite. One more observation is that the proposed model has achieved the fast encoding of input when it is fed with DCT representation