An Auto-Focus Method Based on Deep Reinforcement Learning for Microendoscopy

Auto-focus is essential for microendoscopy by achieving clear vision to screen for and prevent cancer. Conventional auto-focus methods utilize the rule-based search algorithm to maximize the sharpness metric across a range of focus distance, but they always stuck in a local optimum due to image distortions in real-world environments. To address this challenge, we propose a novel auto-focus method for microendoscopy, which treats the auto-focus procedures to be Markov Decision Process (MDP) based on deep reinforcement learning. Then we introduce an enhanced Proximal Policy Optimization (PPO) algorithm, called PPO-SOFT, which encourages the agent to adapt to different exploration strategies by clipping constraints, which makes reinforcement learning more stale to the complicated auto-focus environment. We evaluate our method on our auto-focus environment for microendoscopy, the experimental results show that our algorithm outperforms other approaches in terms of average cumulative reward and focus accuracy rate for the auto-focus task. Additionally, we also evaluate the proposed clipping constraint on the MuJoCo benchmark, our method demonstrates robust and stable than other improved PPO methods.