Zobrazeno 1 - 10
of 19
pro vyhledávání: '"Kulmizev, Artur"'
Recent studies have shown that language models pretrained and/or fine-tuned on randomly permuted sentences exhibit competitive performance on GLUE, putting into question the importance of word order information. Somewhat counter-intuitively, some of
Externí odkaz:
http://arxiv.org/abs/2203.10995
Autor:
Kulmizev, Artur, Nivre, Joakim
In the last half-decade, the field of natural language processing (NLP) has undergone two major transitions: the switch to neural networks as the primary modeling paradigm and the homogenization of the training regime (pre-train, then fine-tune). Ami
Externí odkaz:
http://arxiv.org/abs/2110.08887
Autor:
Abdou, Mostafa, Kulmizev, Artur, Hershcovich, Daniel, Frank, Stella, Pavlick, Ellie, Søgaard, Anders
Pretrained language models have been shown to encode relational information, such as the relations between entities or concepts in knowledge-bases -- (Paris, Capital, France). However, simple relations of this type can often be recovered heuristicall
Externí odkaz:
http://arxiv.org/abs/2109.06129
Publikováno v:
EACL 2021
Since the popularization of the Transformer as a general-purpose feature encoder for NLP, many studies have attempted to decode linguistic structure from its novel multi-head attention mechanism. However, much of such work focused almost exclusively
Externí odkaz:
http://arxiv.org/abs/2101.10927
In this work, we demonstrate that the contextualized word vectors derived from pretrained masked language model-based encoders share a common, perhaps undesirable pattern across layers. Namely, we find cases of persistent outlier neurons within BERT
Externí odkaz:
http://arxiv.org/abs/2011.04393
We present K{\o}psala, the Copenhagen-Uppsala system for the Enhanced Universal Dependencies Shared Task at IWPT 2020. Our system is a pipeline consisting of off-the-shelf models for everything but enhanced graph parsing, and for the latter, a transi
Externí odkaz:
http://arxiv.org/abs/2005.12094
Recent work on the interpretability of deep neural language models has concluded that many properties of natural language syntax are encoded in their representational spaces. However, such studies often suffer from limited scope by focusing on a sing
Externí odkaz:
http://arxiv.org/abs/2004.14096
Representational Similarity Analysis (RSA) is a technique developed by neuroscientists for comparing activity patterns of different measurement modalities (e.g., fMRI, electrophysiology, behavior). As a framework, RSA has several advantages over exis
Externí odkaz:
http://arxiv.org/abs/1909.00303
Transition-based and graph-based dependency parsers have previously been shown to have complementary strengths and weaknesses: transition-based parsers exploit rich structural features but suffer from error propagation, while graph-based parsers bene
Externí odkaz:
http://arxiv.org/abs/1908.07397
We investigate the effects of multi-task learning using the recently introduced task of semantic tagging. We employ semantic tagging as an auxiliary task for three different NLP tasks: part-of-speech tagging, Universal Dependency parsing, and Natural
Externí odkaz:
http://arxiv.org/abs/1808.09716