A Prompt-Based Hierarchical Pipeline for Cross-Domain Slot Filling

In task-oriented dialogue systems, slot filling aims to identify the semantic slot types of each token in user utterances. Due to the lack of sufficient supervised data in many scenarios, it is necessary to transfer relevant knowledge by using cross-domain slot filling. Previous studies rely on manually additional meta-information to build the relationships among similar slots across domains, yet not fully utilizing the knowledge learned by language models in the pre-training stage. In this study, we propose a prompt-based hierarchical pipeline (PHP) with three innovations. First, we design a hierarchical pipeline to separately model domain-independent syntactic structures and domain-specific semantic structures, i.e., span detection and slot prediction. Second, we improve the prompt paradigm with discriminative structure to fully utilize pre-trained language models, which reformulates downstream tasks into pre-trained tasks. Finally, we polish our template and verbalizer to effectively utilize task-specific prior knowledge, adding some meta-information and updating their additional trainable parameters. We conducted extensive experiments on three datasets to evaluate our method, and experimental results show that our method significantly outperforms the previous state-of-the-art results.