Pre-training with Meta Learning for Chinese Word Segmentation
| Autor: | Erli Meng, Bin Wang, Liang Shi, Zhen Ke, Xipeng Qiu, Songtao Sun |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2020 |
| Předmět: |
FOS: Computer and information sciences
Computer Science - Computation and Language Meta learning (computer science) Computer science business.industry 02 engineering and technology 010501 environmental sciences Machine learning computer.software_genre 01 natural sciences Task (computing) 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Segmentation Language model Artificial intelligence Chinese word business computer Computation and Language (cs.CL) 0105 earth and related environmental sciences |
| Zdroj: | NAACL-HLT |
| Popis: | Recent researches show that pre-trained models (PTMs) are beneficial to Chinese Word Segmentation (CWS). However, PTMs used in previous works usually adopt language modeling as pre-training tasks, lacking task-specific prior segmentation knowledge and ignoring the discrepancy between pre-training tasks and downstream CWS tasks. In this paper, we propose a CWS-specific pre-trained model METASEG, which employs a unified architecture and incorporates meta learning algorithm into a multi-criteria pre-training task. Empirical results show that METASEG could utilize common prior segmentation knowledge from different existing criteria and alleviate the discrepancy between pre-trained models and downstream CWS tasks. Besides, METASEG can achieve new state-of-the-art performance on twelve widely-used CWS datasets and significantly improve model performance in low-resource settings. Accepted by NAACL 2021 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|