Knowledge-guided Aspect-based Summarization

Contextualized pre-trained models, such as BERT [1] and BART [2], have shown great potential in various NLP tasks, pushing the state-of-the-art results to a new level. Although studies have shown that those pre-trained models have captured different kinds of knowledge due to the massive corpus they have been trained on [3], injecting task-specific external knowledge often shows further improvement [4]. Here we choose aspect-based abstractive summarization as a case study and explore two different ways to inject external knowledge into BART. One is through a knowledge graph, the other is through human-defined sequence-level scores. Experiment results show that both methods can get an improvement over vanilla BART.