CombineNet: A Stereo Network Combining Local and Semi-global Aggregation

Cost aggregation is a critical component in the stereo matching task, as it largely determines the quality of the resulting disparity map. However, most existing stereo networks rely solely on 3D convolution, a local aggregation method. In this study, we propose incorporating the traditional semi-global aggregation method, which optimizes an energy function, as a layer in CNN models by backpropagating the gradient from the disparity map to the cost volume. This approach offers the advantage of not requiring any learned parameters in the layer. Furthermore, the same method can be used in the Winner-Take-All (WTA) strategy instead of soft-argmin for disparity prediction. We also introduce an end-to-end stereo network (CombineNet) that combines both local and semi-global aggregation. Our results demonstrate that this network achieves competitive performance on Scene Flow and KITTI datasets, and outperforms advanced networks for foreground disparity prediction.