Sonar Object Detection from Synthetic Sonar Data Generated by a Sonar Simulator and an Adversarial Neural Network

The lack of suitable sonar datasets has made it difficult to train robust deep-learning models for object detection with sonar. Using synthetic sonar data to expand existing or create new data sets can facilitate robust training of machine learning based detectors for underwater sonar applications. In this work a pipeline is presented that enables the generation of synthetic sonar images from 3D models for both forward-looking sonar (FLS) and side scan sonar (SSS) by combining 3D renderings with the style transfer capabilities of generative adversarial neural networks. Images generated by this pipeline are used to train object detectors which are evaluated on two data sets, one with shipwrecks and the other with subsea infrastructure collected with a synthetic-aperture sonar and forward-looking sonar respectively. The results of this study show that an object detector trained purely on the synthetic FLS data generated by this approach, was able to detect and correctly classify the structures observed in the majority of the subsea infrastructure images. A detector trained on real images of the same structures performed better with a recall of 87.4% against 64.8% . The object detector trained purely on synthetic side scan sonar data yielded a recall of 83% on our shipwreck dataset at 80% confidence, but yielded a high rate of false positives when the confidence threshold was lowered. The results indicate that an object detector trained on synthetic data of specific objects generated with this approach is able to learn features that translate to detection in real images.