Methods for Estimating and Improving Robustness of Language Models

Autor:	Štefánik, Michal
Rok vydání:	2022
Předmět:	Computer Science - Computation and Language Computer Science - Artificial Intelligence
Druh dokumentu:	Working Paper
Popis:	Despite their outstanding performance, large language models (LLMs) suffer notorious flaws related to their preference for simple, surface-level textual relations over full semantic complexity of the problem. This proposal investigates a common denominator of this problem in their weak ability to generalise outside of the training domain. We survey diverse research directions providing estimations of model generalisation ability and find that incorporating some of these measures in the training objectives leads to enhanced distributional robustness of neural models. Based on these findings, we present future research directions towards enhancing the robustness of LLMs. Comment: Thesis proposal, accepted & to appear in NAACL SRW 2022
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2206.08446 Zobrazit plný text záznamu View this record from Arxiv