Labels Need Prompts Too: Mask Matching for Natural Language Understanding Tasks

Autor:	Li, Bo, Ye, Wei, Wang, Quansen, Zhao, Wen, Zhang, Shikun
Rok vydání:	2023
Předmět:	Computer Science - Computation and Language Computer Science - Artificial Intelligence
Druh dokumentu:	Working Paper
Popis:	Textual label names (descriptions) are typically semantically rich in many natural language understanding (NLU) tasks. In this paper, we incorporate the prompting methodology, which is widely used to enrich model input, into the label side for the first time. Specifically, we propose a Mask Matching method, which equips an input with a prompt and its label with another, and then makes predictions by matching their mask representations. We evaluate our method extensively on 8 NLU tasks with 14 datasets. The experimental results show that Mask Matching significantly outperforms its counterparts of fine-tuning and conventional prompt-tuning, setting up state-of-the-art performances in several datasets. Mask Matching is particularly good at handling NLU tasks with large label counts and informative label names. As pioneering efforts that investigate the label-side prompt, we also discuss open issues for future study. Comment: AAAI2024, Regular Paper
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2312.08726 Zobrazit plný text záznamu View this record from Arxiv