Road Roughness Estimation for Intelligent Vehicles Based on SNE and Semantic Segmentation

The rapid development of unmanned systems benefits from the improvement of perceptual capabilities. However, there is a lack of road roughness estimation for intelligent vehicles. This paper proposes a real-time road roughness estimation method, which can be applied by intelligent vehicles to obtain the specific position and height of the road patches. The binocular camera mounted on the intelligent vehicle can capture RGB images and parallax information. The images are input into the devised lightweight semantic segmentation model, which can accurately recognize each category on the road. Based on the Surface Normal Estimator (SNE), the height of each point in the Region of Interest (ROI) above the road is estimated. After multi-frame fusion, filtering processing and other operations, the specific height and position of the bumps in the road are obtained. Based on the combination of VGG and FPN, the proposed method converts an image to several images with different sizes as inputs and utilizes the attention mechanism and the adaptive loss function of multiple branches. We further validate our approach on a real-world dataset. The experimental results indicate that the mean Intersection over Union (mIoU) of the devised semantic segmentation model can achieve 0.686, and the frames per second (FPS) of the integrated method can reach 78. In addition, the proposed method can estimate the height of the road bumps with 91% accuracy, which can be leveraged to estimate the road roughness for intelligent vehicles accurately in real-time.

I. Introduction

For intelligent vehicles, it is essential to accurately recognize surrounding road conditions and make corresponding decisions and planning according to the specific situations. The safety and comfort of driving also needs to be considered. While going on bumpy roads, intelligent vehicles should not pass at an excessive speed, otherwise, it will make passengers uncomfortable, wear out the life of some vehicle parts and even cause traffic accidents. But if intelligent vehicles drive at the lowest speed all the time while sensing bumps in the road, it would cause traffic jams and reduce driving efficiency. If the accurate height of the road bump can be estimated in real-time, the control module of the intelligent vehicle can automatically formulate the appropriate strategy to adjust its speed. Consequently, this method can not only ensure the safety, stability and comfort of the vehicle but also improve the efficiency of driving and alleviate traffic jams.