JuriBERT: A Masked-Language Model Adaptation for French Legal Text

Autor:	Douka, Stella, Abdine, Hadi, Vazirgiannis, Michalis, Hamdani, Rajaa El, Amariles, David Restrepo
Rok vydání:	2021
Předmět:	Computer Science - Computation and Language
Druh dokumentu:	Working Paper
Popis:	Language models have proven to be very useful when adapted to specific domains. Nonetheless, little research has been done on the adaptation of domain-specific BERT models in the French language. In this paper, we focus on creating a language model adapted to French legal text with the goal of helping law professionals. We conclude that some specific tasks do not benefit from generic language models pre-trained on large amounts of data. We explore the use of smaller architectures in domain-specific sub-languages and their benefits for French legal text. We prove that domain-specific pre-trained models can perform better than their equivalent generalised ones in the legal domain. Finally, we release JuriBERT, a new set of BERT models adapted to the French legal domain. Comment: 7 pages
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2110.01485 Zobrazit plný text záznamu View this record from Arxiv