POE: Acoustic Soft Robotic Proprioception for Omnidirectional End-effectors

Shape estimation is crucial for precise control of soft robots. However, soft robot shape estimation and proprioception are challenging due to their complex deformation behaviors and infinite degrees of freedom. Their continuously deforming bodies complicate integrating rigid sensors and reliably estimating its shape. In this work, we present Proprioceptive Omnidirectional End-effector (POE), a tendon-driven soft robot with six embedded microphones. We first introduce novel applications of 3D reconstruction methods to acoustic signals from the microphones for soft robot shape proprioception. To improve the proprioception pipeline’s training efficiency and model prediction consistency, we present POE-M. POE-M predicts key point positions from acoustic signal observations and uses an energy-minimization method to reconstruct a physically admissible high-resolution mesh of POE. We evaluate mesh reconstruction on simulated data and the POE-M pipeline with real-world experiments. Ablation studies suggest POE-M’s guidance of the key points during the mesh reconstruction process provides robustness and stability to the pipeline. POE-M reduced the maximum Chamfer distance error by 23.1 % compared to the state-of-the-art end-to-end soft robot proprioception models and achieved 4.91 mm average Chamfer distance error during evaluation. Supplemental materials, experiment data, and visualizations are available at sites.google.com/view/acoustic-poe.