Reducing Language Bias for Robust VQA Model with Multi-Branch Learning

Although existing models perform well in Visual Question Answering tasks, they still suffer from the problem of language prior. Most models tend to output answers that are often found in the dataset, thus ignoring image content. To alleviate the above problem, we propose a novel multi-branch learning model (MBL). First, we propose a robust backbone architecture for multimodal interaction and inference. Second, we design a novel language debiasing module to improve the model generalization performance by directly capturing language biases and eliminating them. Finally, we design a global context re-interaction module, which aims to further enhance inter-modal interaction and thus improve model robustness. In addition, we propose to construct two types of negative samples to assist in training the model without introducing additional annotation information. Extensive experiments on the VQA-CP v2 and VQA v2 datasets demonstrate the effectiveness of our MBL method.