Fusion of UNet and Criminisi algorithms for endoscopic specular highlight removal

To address the problems of incomplete highlight segmentation and unreasonable repair texture in the current endoscope specular highlight removal algorithm, a fusion of UNet and Criminisi is proposed for the endoscope specular highlight removal algorithm. Highlight removal is mainly divided into two processes: highlight segmentation and inpainting. In the highlight segmentation module, the highlights are segmented based on the improved UNet model in combination with the transfer learning strategy. Firstly, VGG16 is used as the backbone feature extraction network; secondly, in the enhanced feature extraction module, a simple pyramid pooling module is used to aggregate global contextual information to improve the network's ability to obtain global information; finally, we trained the model using a loss function that mixes dice loss and focus loss. In the highlight inpainting module, the Criminisi algorithm with a better reconstruction effect is chosen and the original priority function is optimized to improve the algorithm. The experimental results show that in the highlight segmentation module, the Dice coefficient and IoU of the improved algorithm are improved by 0.97% and 1.37%, respectively; in the highlight inpainting module, the PSRN and SSIM values of the improved algorithm are improved by 0.78 dB and 0.005, respectively, which verifies the effectiveness and feasibility of the improved algorithm and has certain clinical application potential.