Preventing RNN from Using Sequence Length as a Feature

Autor:	Baillargeon, Jean-Thomas, Cossette, Hélène, Lamontagne, Luc
Rok vydání:	2022
Předmět:	Computer Science - Machine Learning
Druh dokumentu:	Working Paper
Popis:	Recurrent neural networks are deep learning topologies that can be trained to classify long documents. However, in our recent work, we found a critical problem with these cells: they can use the length differences between texts of different classes as a prominent classification feature. This has the effect of producing models that are brittle and fragile to concept drift, can provide misleading performances and are trivially explainable regardless of text content. This paper illustrates the problem using synthetic and real-world data and provides a simple solution using weight decay regularization. Comment: 6 pages, but my overleaf generrates 5 pages. I have no error, the font size seems different
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2212.08276 Zobrazit plný text záznamu View this record from Arxiv