Transparent Object Depth Reconstruction Framework for Mixed Scenes with Transparent and Opaque Objects

The increasing demand for autonomous robots necessitates the recognition and handling of transparent objects commonly found in daily life. Recognizing transparent objects through sensors is challenging due to their partial transmission and refraction of light, resulting in inaccurate depth measurements. Through testing, we checked that previous studies on transparent object depth reconstruction yielded inaccurate results in scenes where transparent and opaque objects are mixed, as well as false-positive results in scenes without transparent objects. We propose a framework that performs transparent object depth reconstruction even in scenes where both transparent and opaque objects coexist, while avoiding false-positive results in scenes without transparent objects. We utilize the structure of ClearGrasp, which has separate networks for different roles, allowing easy modifications. The Translab network, which is trained using the Trans10K-v2 dataset, is used for transparent object segmentation during the inference process. For the Depth Completion network, we utilized a network with a self-attention mechanism that effectively completes significant depth differences in the surroundings. Additionally, we design and apply the Depth Modification module to enhance depth input of the Depth Completion network by retaining accurate depth values in transparent object regions. The experimental results showed that our network achieved a lower depth estimation RMSE of 0.019 for real-novel objects compared to existing state-of-the-art networks. Furthermore, the experimental results showed a success rate of 94.1% for bin picking with cluttered objects.