A Multimodal Approach Based on Large Vision Model for Close-Range Underwater Target Localization

Underwater target localization uses real-time sensory measurements to estimate the position of underwater objects of interest, providing critical feedback information for underwater robots in tasks, such as obstacle avoidance, scientific exploration, and environmental monitoring. While acoustic sensing is the most acknowledged and commonly used method in underwater robots and possibly the only effective approach for long-range underwater target localization, such a sensing modality generally suffers from low resolution, high cost, and high energy consumption, thus leading to a mediocre performance when applied to close-range underwater target localization. On the other hand, optical sensing has attracted increasing attention in the underwater robotics community for its advantages of high resolution and low cost, holding a great potential particularly in close-range underwater target localization. However, most existing studies in underwater optical sensing are restricted to specific types of targets, thus lacking generalization capabilities. In addition, these studies typically focus on the design of estimation algorithms and ignore the influence of illumination conditions on the sensing performance, thus hindering wider applications in the real world. To address the aforementioned issues, this article proposes a novel target localization method that assimilates both optical and acoustic sensory measurements to estimate the 3-D positions of close-range underwater targets. The proposed sensing method integrates a large vision model with unique acoustic-based model prompt design to process multimodal sensor measurements, ensuring the generalizability and robustness of underwater target localization. A test platform with controllable illumination conditions is developed. Extensive experiments are conducted, the results of which validate the effectiveness of the proposed method.